Vision Field Notes

Fine-Tuning YOLO: Transfer Learning Explained

Almost nobody trains a detector from scratch. You start from a model that has already learned to see on a large public dataset, then adapt it to your classes and cameras. This is transfer learning, and fine-tuning is the adaptation step. It is the reason a few hundred images can produce a usable detector.

This article explains why it works, how to do it with Ultralytics, which settings matter, and where it breaks down.

Why pretrained weights help

A detector has two broad jobs. The first is to extract visual features: edges, textures, shapes, parts of objects. The second is to turn those features into boxes and class labels.

The feature-extraction part is largely general. Edges and textures are similar whether you detect cats or hard hats. A model trained on a large, varied dataset has already learned good general features. When you fine-tune, you keep that knowledge and mainly teach the model your specific classes and your specific look.

Diagram of a detector as four boxes in a row: your image, backbone (edges, textures, object parts), neck (combines scales) and head (boxes plus your class names). A blue line over the backbone and neck says reused from the COCO checkpoint, yolo11n.pt. An orange line over the head says re-created for your class count. A dashed outline around the backbone is labelled freeze: keep these early layers fixed, worth testing on small datasets.
The backbone and neck carry over from the pretrained checkpoint; only the head is rebuilt for your classes. Freezing keeps the early layers fixed while the rest adapts.

Starting from scratch needs far more data and training time, and with a small dataset it usually gives worse results.

The pretrained checkpoint and its classes

Ultralytics checkpoints such as yolo11n.pt are trained on the COCO dataset, which has 80 common classes. When you train on your own data with different classes, the final prediction layers are re-created to match your class count, while the earlier layers are reused. You do not need to do anything special for this; the library handles it.

A warehouse frame where a worker in a yellow hard hat and orange safety vest is boxed as person with score 0.89. Arrows point to the hard hat and the vest with notes saying each is not a COCO class.
A pretrained COCO model already finds the person reliably, which is the general knowledge fine-tuning reuses. It cannot report the hard hat or vest until you fine-tune it on labelled examples of those classes. Footage: Intel IoT DevKit sample videos (CC BY 4.0). Detections: YOLOv8n.

Check the current licence of any pretrained weights before commercial use. Ultralytics models are AGPL-3.0 with a commercial licence available. See YOLO Licenses Explained: AGPL, GPL and Commercial Use.

A basic fine-tuning run

from ultralytics import YOLO

model = YOLO("yolo11n.pt")          # pretrained weights
model.train(
    data="data.yaml",
    epochs=50,
    imgsz=640,
)

That is already fine-tuning. Loading a .pt checkpoint means training starts from learned weights. By contrast, loading a model definition file (a .yaml) builds the architecture with random weights. Prefer the checkpoint unless you have a specific reason not to.

For the full dataset setup, see Train YOLO on a Custom Dataset: A Step-by-Step Guide.

Freezing layers

Freezing means keeping some layers fixed so training does not change them. Ultralytics exposes a freeze argument for this, which takes a number of layers (or a list of layer indices). Check the documentation for the exact behaviour in your version.

model.train(data="data.yaml", epochs=50, freeze=10)

Why freeze:

Why not freeze:

A sensible approach: try full fine-tuning first as a baseline. If you see signs of overfitting on a small dataset, try freezing the early part of the network and compare on validation. Do not assume freezing helps. Test it.

Learning rate and epochs

Fine-tuning generally uses gentler updates than training from scratch, because the weights are already good and large steps can damage them.

Warm-up epochs at the start gradually raise the learning rate and help avoid early damage to pretrained weights. This is a default behaviour in Ultralytics; check the docs for current settings.

Choosing the starting point

Using fine-tuning to add data over time

A practical production pattern: deploy a first model, collect the cases where it fails on real footage, label them, and fine-tune again from the previous best weights with old and new data combined. Mixing old data in matters. If you train only on new data, the model may drift toward the new conditions and lose performance on the old ones. This is called catastrophic forgetting.

Keep a stable validation set that spans old and new conditions, so you can see if an update improves one situation and harms another.

When fine-tuning fails

Practical recipe

  1. Fine-tune a small pretrained model with default settings. Record the baseline.
  2. Inspect validation metrics per class and look at predictions on real footage.
  3. Fix data problems first.
  4. If overfitting on a small dataset, test freezing early layers.
  5. If underfitting, try a larger model or higher resolution.
  6. Only then adjust learning rate and other fine details.
  7. Change one thing at a time and keep a log of settings and results.

Checklist

Next steps: put this into practice with Train YOLO on a Custom Dataset: A Step-by-Step Guide, and check the result with mAP, Precision and Recall: How to Read Detection Results.