The distance between a model that works on a GPU workstation and the same model working next to the machine it monitors is where most edge projects quietly die. The model was never the hard part. The hard part is that the target is a Jetson in a hot enclosure, an STM32 on a coin cell, or a RISC-V board the toolchain has barely heard of, and none of them care how well the demo ran.

This is the checklist we work through when a model has to leave the workstation. It's written as a sequence because the order matters: each step constrains the next.

1. Set the budget before touching the model

Latency, memory, and power: written down, agreed with the people who own the machine, before any optimisation starts. A vision model on a Jetson-class device might get tens of milliseconds and a few gigabytes; an anomaly detector on an STM32 gets kilobytes and a power draw measured in milliwatts. Everything that follows is a negotiation against this budget. Without it, "fast enough" is decided by whoever is in the room, and it changes weekly.

Budget the whole pipeline, not just the forward pass. Capture, preprocessing, inference, and acting on the result all spend from the same account.

2. Choose the compression path deliberately

We try the cheapest thing first. Post-training quantisation to INT8 costs an afternoon and a calibration set, and for many convolutional models it's all you need. It fails predictably: layers with wide activation ranges, attention blocks in small transformers, and models so small that every weight already carries signal. When accuracy drops off a cliff, per-channel quantisation and a better calibration set are the next moves, not panic.

Quantisation-aware training buys the accuracy back by simulating low precision during training. It costs a full training loop and requires access to the training data, which is why we confirm that access exists in week one, not the week the accuracy cliff appears.

When the architecture itself is wrong for the target, no amount of quantisation saves it. Distillation into a smaller, hardware-friendly architecture is the honest fix: the large model becomes the teacher and the label generator, and the small model is designed around the ops the target actually accelerates.

3. Measure on the device, not the devkit

The devkit on your desk has a heatsink the production enclosure doesn't, a power supply the installation won't match, and none of the other workloads that will share the device.

The devkit on your desk is not the device on the wall.

We measure p99 latency, not the mean, on production hardware, inside the real enclosure, at operating temperature, after at least thirty minutes of sustained load, with the real preprocessing in the loop. Anything else is a benchmark of a machine you're not shipping.

4. Design for disconnection

A factory network is not the internet, and it shouldn't need to be. The device works offline first: inference is local, results are queued when the network is down, and the failure mode is degraded-but-predictable, never frozen. Updates arrive over the air as signed artifacts with staged rollout and a tested rollback path, because the alternative is a technician with a laptop walking the line, and that's how devices end up running three-year-old models forever.

What breaks on the way

The recurring failures, so you can look for them early:

  • Preprocessing drift. Training used one library's resize and normalisation; the device uses another. The model is fine, the pixels are different, accuracy falls quietly. Pin the preprocessing into the exported graph or test it byte-for-byte.
  • Unsupported ops. The exporter emits an operation the runtime doesn't accelerate, and it falls back to CPU, silently. Check the runtime's placement report, not just the output.
  • Thermal throttling. The board passes a five-minute benchmark and throttles after thirty. This is why sustained-load measurement is on the list.
  • Accuracy cliffs in rare classes. Quantisation error concentrates in exactly the defect classes you built the system to catch. Evaluate per class, never on the aggregate alone.

None of this is exotic. That's the point: it's a checklist precisely because the failures repeat, and a model that survives it earns its place on the floor.