Every embedded project we scope hits the same question: the model needs to be INT8 to hit the latency and memory budget — so what does that do to accuracy? The literature says "usually under a point." Vendors say "negligible." Both are broadly right and completely useless for your model, which is why our edge pipeline produces an accuracy regression report at every compression step instead of quoting folklore.

What we measure, and how

The setup is deliberately unexciting. We hold out an evaluation set drawn from the client's production distribution — not the clean training split — and score the full-precision model against its quantized version on the task metric that matters (mAP for detection, F1 for classification, task-level accuracy for pipelines). Calibration for post-training quantization (PTQ) uses a few hundred real production samples. Then we diff.

The pattern the numbers keep showing

  • The typical case is boring — happily. Well-conditioned convolutional vision models lose a fraction of a point going to INT8 with decent calibration data. Users of the deployed system cannot tell the difference; the latency graph very much can.
  • Small models bruise more easily. A network that is already compact has less redundancy to absorb rounding error. Aggressively pruned models plus PTQ is where we most often see losses worth arguing about.
  • Outliers are the real villain. A handful of activation channels with extreme ranges can wreck a per-tensor quantization scheme. Per-channel quantization and smarter calibration usually recover most of it.
  • Pipelines compound. A detector feeding a classifier feeding business rules multiplies small degradations. We evaluate end-to-end task accuracy, not per-model metrics — the client ships the pipeline, not the layer.
Compression without a regression report is guesswork. — from our edge delivery pipeline

When PTQ isn't enough

When post-training quantization costs more than the accuracy budget allows, the escalation path is quantization-aware training — fine-tuning with quantization simulated in the loop, so the network learns around the rounding. It costs training time and pipeline complexity, which is exactly why we only reach for it when the measurements say so, not by default.

Practice note The order of operations matters: quantize then measure then decide — per step, with the client seeing every trade-off. An accuracy budget agreed up front turns "is this loss acceptable?" from a debate into a lookup.

None of this is novel research — it is the discipline of applying known results carefully, per project, on real data. That is the point of these field notes: the literature tells you what usually happens. The regression report tells you what happened to you.