AI Model Accurate on PC but Wrong on the Device: Fixing Accuracy Loss After Deployment
The model scored well in training and misses obvious detections on the product. Why that happens and how I recover the accuracy.
The problem clients bring: "Our model works great on the laptop. On the device it misses obvious objects and gives strange results, and the data science team cannot reproduce it."
What is really going wrong
- Quantisation to INT8 was calibrated on data that does not look like the real camera.
- Image pre-processing on the device differs from training: colour order, resize method or normalisation.
- The camera's exposure, lens and noise are different from the training images.
- Some layers fall back to a slower path with different numerical behaviour.
How I solve it
- Run the same input through the PC model and the device model and compare layer outputs.
- Match pre-processing exactly between training and device.
- Recalibrate quantisation with frames captured from the real device.
- Tune camera settings for consistent images.
- Feed device-captured images back into training when needed.
What the client gets
On-device accuracy close to the PC results, and a repeatable process to check every new model before it ships.