- Client
- Consumer product company (under NDA)
- Constraint
- Processor under $4 at volume
- Task
- Detect and track several objects in video
- Result
- Real-time tracking on a microcontroller-class chip
The problem
The product needed to follow several moving objects in front of a camera and react to them. Every off-the-shelf approach assumed a Linux board with an NPU, which would have tripled the product cost. The client asked whether it could be done on a much cheaper chip.
How I solved it
Simplify the problem
Instead of a large general detector, I used the product's fixed camera position and controlled scene to reduce what the model needed to learn.
A tiny detector
A small quantised model finds candidate objects at low resolution, using the chip's vector instructions.
Classic tracking
A lightweight tracker associates detections between frames and predicts motion, so the expensive detector does not need to run on every frame.
Memory discipline
Frame buffers, model weights and tracker state were laid out carefully to fit in the chip's on-chip and external RAM.
Field testing
Tuned on footage from the actual product, under real lighting.
What the client got
Results
- Real-time multi-object tracking on a chip under $4.
- A product cost that made the business case work.
- A pipeline that can be retrained as the product evolves.
Embedded systems consultant at TechProgeny, Bengaluru. 7+ years and 29+ shipped products across firmware, embedded Linux, edge AI, apps and IoT servers.