Vision Field Notes

Running Object Detection in the Browser with ONNX Runtime Web

You can run a trained detector entirely inside a web page. The model file downloads to the visitor's browser, and inference runs on their device. No server receives the images. This is useful for demos, privacy-sensitive tools and cheap hosting. It also has real limits, which you should know before you start.

This article explains how ONNX Runtime Web works, what to expect, how to prepare a model and what can go wrong.

How it works

ONNX Runtime Web (the onnxruntime-web package) is a JavaScript build of ONNX Runtime. It loads an .onnx model and runs it using one of several execution backends:

You choose providers in order of preference, and the runtime falls back if one is unavailable. Which one wins depends on the browser, device and model, so test on the devices your users have.

Flowchart starting from executionProviders webgpu then wasm. If WebGPU is available, the GPU path is used. If not, WASM runs on the CPU: if the page is cross-origin isolated it is multi-threaded, otherwise single-threaded and slowest. A note says cross-origin isolation needs COOP and COEP headers that some static hosts cannot set.
The same page can run on the GPU, multi-threaded WASM or single-threaded WASM depending on the visitor's browser and the headers your host sends.

The pipeline

The steps are the same as in any ONNX deployment:

  1. Get an image from a file input, an image element or the camera.
  2. Pre-process it: letterbox resize, RGB, scale to 0-1, channel-first layout.
  3. Run the model.
  4. Decode the output and apply non-maximum suppression.
  5. Draw boxes on a canvas.

The model export and decoding details are the same as in exporting YOLO to ONNX and running it with ONNX Runtime. The big difference is that you now write pre- and post-processing in JavaScript, with no NumPy or OpenCV. Canvas APIs handle resizing and pixel access, and you write NMS yourself or use a small library.

Diagram: a static web host with HTML, JavaScript, the WASM runtime and model.onnx sends files down once to the visitor's browser. Inside the browser, camera or image file, pre-process on a canvas, ONNX Runtime Web, decode and NMS in JavaScript, and draw boxes on a canvas all run on the visitor's device.
Only static files travel, and only from the host to the browser. The whole detection pipeline runs on the visitor's device, so the image does not need to be uploaded.

A skeleton for loading and running a model:

import * as ort from "onnxruntime-web";

const session = await ort.InferenceSession.create("model.onnx", {
  executionProviders: ["wasm"],
});

// input: Float32Array in NCHW order, e.g. [1, 3, 640, 640]
const tensor = new ort.Tensor("float32", input, [1, 3, 640, 640]);
const results = await session.run({ [session.inputNames[0]]: tensor });
const output = results[session.outputNames[0]];

Read session.inputNames and session.outputNames instead of hard-coding names, and check output.dims against what you expect. You can put "webgpu" before "wasm" in the provider list to try the GPU path first.

Preparing the model

Keep it small

The visitor must download the model. A large file means a slow first load and a poor experience on mobile data. Options:

Browsers also limit memory per tab. Very large models can fail on phones. Test on a low-end device, not just your laptop.

Export settings

Use a fixed input shape unless you need otherwise, since fixed shapes are simpler and often better supported. Check that the exported graph uses only operators that the web runtime supports. If a model fails to load, the error message usually names the unsupported operator or shape problem.

Limits to expect

Privacy benefits

Client-side inference means the image never leaves the device. For tools that handle photos of people, documents or private spaces, this is a strong benefit and a simple thing to explain to users: "Your image is processed in your browser and is not uploaded."

Be accurate about it, though. State this only if it is true. If your page also sends analytics, error reports or results to a server, say so. Loading the page itself still lets the host see ordinary web traffic. For the wider picture on responsible camera systems, see privacy and ethics in camera analytics.

Performance tips

Hosting

Because everything is static files (HTML, JavaScript, the runtime's WASM files and the model), you can host it on static hosting. A free option is described in hosting computer vision demos on GitHub Pages. If you want your own address, a domain name and hosting service can serve the same files. Check file size limits and header options on any host you choose.

When to use something else

Browser inference is a poor fit when you need to process many camera streams, run large models, keep strict accuracy on small objects or control the hardware. In those cases, run inference on a server or an edge device. Use the browser for demos, education, lightweight tools and privacy-first apps.

Key points

Next steps

Read Host Computer Vision Demos for Free on GitHub Pages to publish your demo, and Export YOLO to ONNX for the model side.