Most edge projects die between a model that works on a GPU workstation and the same model working next to the machine it monitors. The target is a Jetson in a hot enclosure, an STM32 on a coin cell, or a RISC-V board the toolchain has barely heard of. This is the checklist we work through when a model has to leave the workstation. Each step constrains the next.
The checklist
Set the budget first. Latency, memory and power, written down and agreed with the people who own the machine. A Jetson-class device might get tens of milliseconds and a few gigabytes; an STM32 gets kilobytes and milliwatts. Budget the whole pipeline, capture and preprocessing included.
Choose the compression path deliberately. Post-training quantisation to INT8 costs an afternoon and is often enough. It fails predictably on wide activation ranges, small attention blocks and tiny models. Quantisation-aware training buys the accuracy back but needs the training data, so confirm that access in week one. When the architecture is wrong for the target, distillation into a smaller, hardware-friendly model is the honest fix.
Measure on the device, not the devkit. p99 latency, on production hardware, inside the real enclosure, at operating temperature, after thirty minutes of sustained load, with the real preprocessing in the loop.
The devkit on your desk is not the device on the wall.
Design for disconnection. Inference is local, results queue when the network is down, and the failure mode is degraded but predictable. Updates arrive over the air as signed artifacts with staged rollout and a tested rollback, or devices end up running three-year-old models forever.
What breaks
- Preprocessing drift. Training used one library's resize, the device uses another. Pin the preprocessing into the exported graph.
- Unsupported ops falling back to CPU silently. Check the runtime's placement report.
- Thermal throttling after thirty minutes, not five.
- Accuracy cliffs in rare classes, the defect classes you built the system to catch. Evaluate per class.