Vision Field Notes

How Many Images Do You Need to Train an Object Detector?

Everyone asks this early, and everyone wants a number. The honest answer is that no single number is correct. A detector for one clean object on a fixed camera needs far fewer images than a detector for ten similar-looking classes in changing weather. What you can do is build a method that tells you, with evidence, whether you have enough.

This article gives rough starting points, explains what really drives the requirement, and shows a simple way to decide when to stop collecting.

Why there is no fixed number

The count of images is a poor proxy for what the model needs to see. What matters is the variety of examples per class, and how closely your training data matches the footage the model will run on.

Four things push the requirement up or down:

Also count instances, not just images. One image with twenty labelled boxes is more training signal for that class than one image with a single box. A dataset of 500 images can be rich or poor depending on how many objects per class it contains.

Rough starting points

These are working rules of thumb from general practice, not guarantees. Treat them as a first target to reach before your first real test, not as a promise of quality.

Situation Starting target Notes
One class, fixed camera, stable scene A few hundred images Variety across time of day still matters
A few classes, one site Several hundred per class Cover each lighting condition
Many classes or similar-looking classes Often a thousand or more per class Hard classes need more
Multiple sites and cameras Add data from every site Do not assume one site generalises

Pretrained weights change the picture. Starting from a model trained on a large public dataset means you are adjusting an existing detector, not teaching one from nothing. That is why a few hundred images can give a usable first result. See Fine-Tuning YOLO: Transfer Learning Explained.

Collect for variety, not volume

A thousand frames from the same ten seconds of video are worth little more than ten frames. Consecutive video frames are nearly identical. If you extract every frame, you inflate the count without adding information, and you also risk leaking near-duplicates into your validation set.

Better habits:

Two rows of three frames from a store aisle camera with detection boxes. The top row shows three consecutive frames at 16.7, 16.8 and 16.9 seconds that look identical, each with 5 boxes. The bottom row shows frames at 1.2, 10.8 and 19.2 seconds with different shoppers and 4, 3 and 6 boxes.
Consecutive frames show the same people in the same places, so they add count but not information. Frames taken seconds apart change who is in the scene and where. Footage: Intel IoT DevKit sample videos (CC BY 4.0). Detections: YOLOv8n.

In production video analytics, a common problem is a model that works well on the footage it was trained on and degrades on a new camera. The fix is almost always more varied data from the new setting, not more of the old data.

How to decide when to stop: learning curves by data size

The most reliable method is empirical. Train on increasing fractions of your data and watch the validation result.

  1. Fix a validation set first, and never train on it.
  2. Train on 25 percent of your training images. Record the validation metric you care about.
  3. Repeat with 50 percent, 75 percent and 100 percent. Keep every other setting identical.
  4. Plot metric against dataset size.

Read the curve:

Line chart of a validation metric against the share of training images used, at 25, 50, 75 and 100 percent. Project A's line climbs steeply all the way to 100 percent. Project B's line rises early and then levels off between 75 and 100 percent.
Illustrative curves, not measurements. A line still climbing at 100 percent says more data will help; a line that has levelled off says to study the errors instead.

This costs a few extra training runs, but it replaces guessing with a measurement. For how to interpret the metric itself, read mAP, Precision and Recall: How to Read Detection Results.

Look at errors, not only the score

A single mAP value hides where the model fails. After each training run, open the validation images with predictions drawn and sort the mistakes into groups:

Targeted collection beats blanket collection. Fifty images that fix a known failure are usually worth more than five hundred random ones.

Getting more from less

When collecting is expensive, these help:

Synthetic images can fill gaps for rare cases, but they rarely replace real footage. Always validate on real images.

Common mistakes

Checklist

Next steps: set up the data layout and run a first training in Train YOLO on a Custom Dataset: A Step-by-Step Guide, then audit your labels with Annotation Mistakes That Quietly Ruin Your Model.