Tracking objects in video sounds like a job for a powerful board. In many products it is not, because the camera is fixed, the scene is controlled and only a few object types matter.
Three building blocks
- Detection: finds objects in a frame. Neural detectors are accurate but expensive to run.
- Tracking: links detections across frames and predicts motion. Cheap to run.
- Classic vision: background subtraction, blobs and optical flow. Very cheap, works well in stable scenes.
Designing for a small chip
Constrain the problem
Use what you know: fixed camera angle, known object sizes, limited classes.
Lower the resolution
Detect at low resolution and only look closer where needed.
Detect less often
Run the detector every few frames and let the tracker fill the gaps.
Quantise
Use 8-bit models and the chip's vector or DSP instructions.
Budget memory
Frame buffers usually dominate. Plan them before choosing the chip.
| Scenario | Realistic on a low-cost chip? |
|---|---|
| Counting people at a doorway | Usually yes |
| Tracking a few parts on a conveyor | Usually yes |
| Many object types in a busy street | No, use an NPU board |
| High resolution with fine detail | No, use an NPU board |
Frequently asked questions
Which chips are suitable?
Microcontrollers with camera interfaces and vector or DSP instructions, such as the ESP32-S3, are common choices.
How many objects can be tracked?
Often around ten in a controlled scene, depending on resolution and frame rate.
Will accuracy suffer?
Within a constrained scene, a well-designed pipeline can perform close to a larger system.
Embedded systems consultant at TechProgeny, Bengaluru. 7+ years and 29+ shipped products across firmware, embedded Linux, edge AI, apps and IoT servers.