Vision Field Notes

Class Imbalance and Small Objects: Practical Fixes

Two problems appear in almost every real detection project. Some classes have far fewer examples than others, and some objects are tiny in the frame. Both make a model look fine on average while failing at the cases you care about. The rare class is often the important one, such as a safety violation, and the tiny object is often a distant person or vehicle.

This article explains why each problem happens and gives fixes in a sensible order.

Class imbalance

Why it hurts

Training minimises an overall loss. If 90 percent of the boxes belong to one class, the model gets most of its reward from that class. The rare class contributes little to the loss, so the model can do poorly on it and still look good overall.

Averaged metrics hide this. A mean over classes weights each class equally, but a plain count of detections does not. Always read per-class results.

Step 1: measure it

Count instances per class in the training and validation sets, not just images. Also check how many distinct situations each class covers. A class with 300 boxes taken from three videos is weaker than it looks.

Then look at per-class precision and recall after training. A class with low recall and few examples is a candidate for the fixes below.

Horizontal bar chart of labelled boxes per class: person 4,200, hard hat 3,100, safety vest 2,700, forklift 900, and no hard hat 140, highlighted in orange with a note saying it is the class the alert depends on and under 2 percent of all boxes.
Illustrative counts for a typical site-safety dataset, not a real one. The class that triggers the alert is the smallest bar, so it contributes little to the training loss unless you act on it.

Step 2: collect targeted data

The most reliable fix is more real examples of the rare class, especially in varied conditions. Search your footage for the rare event, take frames from different cameras and times, and label them. If events are rare in live footage, consider running a rough model to find candidate clips, then review by hand.

Remember that rare events may be rare in validation too. Make sure the validation set contains enough examples of the rare class to give a meaningful number.

Step 3: rebalance what the model sees

Options when you cannot collect more:

Be careful that oversampling does not change the real-world frequency the model expects. If the rare class is truly rare in deployment, the model may produce more false alarms for it after rebalancing, and you will need to tune its threshold. See Confidence Thresholds: Tuning a Detector to Cut False Alarms.

Step 4: consider the class definition

Sometimes imbalance is a sign of a bad class design. Two visually identical classes can be merged. A class that mixes very different appearances may need splitting. If a class cannot be told apart by looking at the pixels, no amount of data will fix it.

Step 5: tune per class

After training, pick the confidence threshold for each class separately. A rare, important class often deserves a lower threshold than a common one, trading more false alarms for more catches. Decide this from the precision-recall curve and the cost of each error type.

Small objects

Why they are hard

A detector looks at the image through a stack of layers that reduce resolution. By the time features reach the deeper layers, a tiny object may occupy only a pixel or two of the feature map. Fine detail is lost, and there is simply less information to learn from. Input images are also commonly resized for training and inference, which shrinks tiny objects further.

Metrics usually suffer on small objects too, because a box that is a few pixels off has a low IoU for a tiny box. See IoU and Non-Maximum Suppression, Explained with Examples.

Step 1: find out how small is small

Compute the width and height of your boxes in pixels at the size the model sees. If many boxes are only a handful of pixels in the resized input, you have a resolution problem. It is better to know this from the numbers than to assume.

Step 2: raise the input resolution

Training and running at a larger imgsz keeps more pixels on each object. This often helps the most for small objects, but it costs memory and compute, so inference gets slower. Check the cost on your target hardware. See Measuring FPS and Latency Correctly.

Train and run inference at the same resolution, or you may lose the benefit.

Step 3: use tiling

Instead of shrinking a large frame to fit the model, cut it into overlapping tiles and run the detector on each tile at full detail. Then map the boxes back to full-frame coordinates and merge duplicates from the overlap regions with NMS.

Key points for tiling:

Two copies of a store aisle full of white crockery. Left, the whole frame run once: 4 boxes, three bowls on the top-right shelf and one large refrigerator box. Right, the frame cut into four overlapping tiles shown as dashed coloured rectangles: 11 boxes, including several more bowls and a cup, but also a false toilet detection on the crockery table and two refrigerator boxes split at a tile edge.
The same frame and model, run whole and as four overlapping tiles. Tiling found more of the small bowls, but also produced a false toilet and an object split into two boxes at a tile seam, so tiles need their own threshold check and careful merging. Footage: Intel IoT DevKit sample videos (CC BY 4.0). Detections: YOLOv8n.

Tiling is powerful for high-resolution frames with small targets, such as aerial or wide-area cameras. For a camera where objects are already large, it is wasted work.

Step 4: improve the camera setup

Often the cheapest fix is not software. A tighter lens, a camera closer to the area of interest, a higher-resolution sensor, or a region-of-interest crop can put more pixels on the target. Review Choosing a Camera for Computer Vision Projects before spending effort on model changes.

Step 5: label carefully

Tiny objects are easy to label loosely or skip. A box that is two pixels too wide matters when the object is ten pixels wide. Set a rule for the minimum size to label and apply it consistently. See Annotation Mistakes That Quietly Ruin Your Model.

Step 6: be careful with augmentation

Strong downscaling and blur can erase small objects entirely, leaving a box around nothing. Keep scale augmentation moderate when targets are tiny.

Measuring progress honestly

Checklist

Next steps: read How Many Images Do You Need to Train an Object Detector? to plan targeted collection, and Confidence Thresholds: Tuning a Detector to Cut False Alarms to set per-class thresholds.