Vision Field Notes

Dockerize a Computer Vision App for Edge and Cloud

"It works on my machine" is a bigger problem in computer vision than in most fields. Your app depends on OpenCV builds, codec libraries, Python packages, CUDA versions and drivers. A container packages the user-space parts of that stack, so the same image runs on your laptop, a cloud server and an edge device.

This article gives practical Dockerfile patterns for vision apps, explains GPU access and shows how to keep images small and reproducible.

What Docker does and does not solve

A container shares the host's kernel. It packages your code, Python, libraries and system packages. It does not package the GPU driver. For GPU use, the host needs a compatible NVIDIA driver, and a runtime component passes the GPU into the container.

Docker also does not make a model faster. It makes deployment repeatable, simplifies rollbacks and isolates apps from each other on a shared device.

A minimal CPU image

For an app using OpenCV and ONNX Runtime on CPU, start from a slim Python image:

FROM python:3.11-slim

ENV PYTHONUNBUFFERED=1 \
    PIP_NO_CACHE_DIR=1

WORKDIR /app

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY src/ ./src/
COPY models/best.onnx ./models/best.onnx

CMD ["python", "src/main.py"]

With a requirements.txt such as:

opencv-python-headless==<pinned version>
onnxruntime==<pinned version>
numpy==<pinned version>

Replace the placeholders with the exact versions you tested. Pinning is the key to reproducibility, and pip freeze from a working environment is a good source.

Use the headless OpenCV package

The standard opencv-python wheel expects GUI libraries that a slim image does not include, and you may see errors about missing shared libraries such as libGL. In a container with no display, use opencv-python-headless. You cannot call cv2.imshow in it, which is fine for a server or edge device. Save frames or stream results instead.

Layer order matters

Docker caches layers. Copy requirements.txt and install dependencies before copying your source code. Then a code change does not trigger a slow reinstall. This single habit saves a lot of time.

Two Dockerfile layer stacks. Left, dependencies first: FROM, COPY requirements.txt and RUN pip install are reused from cache, and only COPY src and COPY model rebuild after a code change. Right, code copied first: COPY . . and RUN pip install both rebuild after a code change.
Docker rebuilds every layer above the first one that changed. Copying code before installing dependencies makes every small edit re-run pip install.

Video input inside a container

How the container gets frames depends on the source:

Example run command:

docker run --rm --device /dev/video0 -v "$PWD/out:/app/out" cv-app:latest

GPU access

For NVIDIA GPUs on a Linux host:

  1. Install the NVIDIA driver on the host.
  2. Install the NVIDIA Container Toolkit.
  3. Run containers with --gpus all, or the subset you need.
docker run --rm --gpus all cv-app-gpu:latest

For the image, choose a base that matches your needs:

The usual failure is a version mismatch between the host driver, the CUDA version in the image and the library you installed. When the GPU is not used, check this first. You can confirm in code: ONNX Runtime's ort.get_available_providers() lists providers, and torch.cuda.is_available() tells you about PyTorch.

Diagram of two stacked boxes. The container image holds your app with headless OpenCV, NumPy and the model, plus onnxruntime-gpu and CUDA and cuDNN libraries. The host holds the Linux kernel, the NVIDIA driver and the NVIDIA Container Toolkit. A red double arrow links the CUDA libraries in the image to the host driver, noting the versions must be compatible.
The image brings CUDA libraries, but the GPU driver lives on the host. The driver, the CUDA version in the image and the onnxruntime-gpu build must all be compatible.

Jetson devices

Jetson boards use NVIDIA's JetPack stack and require container images built for that Arm platform. Use the base images NVIDIA publishes for Jetson, and match them to your JetPack version. You cannot use ordinary x86 CUDA images there. See TensorRT for YOLO for notes on engines, which should be built on the target.

Raspberry Pi

Pi images are Arm builds. Most Python wheels for Arm exist, but check availability for each package. Build for the right platform. On a laptop you can use docker buildx to build multi-platform images, although building on the target is simpler for small projects.

Keep images small

Smaller images pull faster over weak connections and start sooner. Practical steps:

Should the model be inside the image? For small models, including them gives a self-contained, versioned unit. For larger or frequently updated models, mounting a volume or downloading at start avoids rebuilding. Choose based on how often models change. Whatever you choose, log the model's file name and hash at startup so you know what is running.

Reproducibility

For local environment setup outside containers, see setting up a clean Python environment for computer vision.

Runtime settings that matter on the edge

Common mistakes

Checklist

Next steps

Read Setting Up a Clean Python Environment for Computer Vision, then Power, Cooling and Storage for a Reliable Edge AI Box.