Vision Field Notes

Thermal Throttling on Edge Devices: Detecting It and Planning for It

An edge box that runs at full speed on the desk can run slower in the field. Small boards protect themselves from heat by lowering clock speeds, and when the clocks drop, inference takes longer. If the pipeline was sized for the desk number, frames start to queue or drop, and nothing in the application log says why.

This article explains how throttling shows up, how to detect it on a Raspberry Pi and on NVIDIA Jetson, and how to test and size a device so it does not catch you out.

What throttling looks like from the application

Throttling rarely announces itself. What you see instead:

Two schematic curves against time under load. Top: SoC temperature rises and crosses a dashed throttle threshold after several minutes; a shaded band marks the first two minutes as the length of a short benchmark. Bottom: latency per frame is flat until the threshold is crossed and then steps up and keeps rising.
Schematic shape, not a measurement. A short benchmark measures a cool board. How long it takes to reach the limit depends on the board, the cooling, the enclosure and the room.

The key point is the time scale. A board can take many minutes to reach its limit, especially with a heatsink. A benchmark of a minute or two measures a cool board and tells you little about sustained speed. See Measuring FPS and Latency Correctly for the rest of the benchmarking method.

Detecting it on a Raspberry Pi

Raspberry Pi firmware reports both temperature and throttling state:

vcgencmd measure_temp
vcgencmd get_throttled

get_throttled returns a bit field. The bits that matter most:

Bit Meaning
0 Under-voltage detected now
1 ARM frequency capped now
2 Currently throttled
3 Soft temperature limit active now
16 to 19 The same four conditions, but "has occurred since boot"

A value of 0x0 means none of these has happened. A value like 0x80000 means the soft temperature limit was hit at some point since boot, even if it is not active now. Log this value periodically; the "has occurred" bits are what catch an episode that happened at 3 p.m. while nobody was watching.

Note that under-voltage also causes throttling. A weak power supply or a long thin cable can look exactly like a heat problem. If bit 0 or bit 16 is set, fix the power before looking at cooling. Power, Cooling and Storage for a Reliable Edge AI Box covers both.

On any Linux board you can also read the kernel's view directly:

cat /sys/class/thermal/thermal_zone*/temp           # millidegrees Celsius
cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_cur_freq   # current CPU clock in kHz

A CPU clock that falls under steady load is the clearest sign.

Detecting it on Jetson

On Jetson boards, tegrastats prints CPU and GPU load, clock frequencies, temperatures and power rails once per interval. The community tool jtop (from the jetson-stats package) shows the same in a friendlier view. Watch the GPU frequency under load: if it falls while load stays high, the board is being limited, by temperature or by the power budget of the current mode.

Jetson also has power modes, set with nvpmodel, that cap clocks and power by design. A benchmark in one mode is not comparable with another, so record the mode with every result. jetson_clocks pins clocks at the maximum for the current mode, which is useful for repeatable benchmarks, but a box in a hot enclosure may not be able to sustain it.

Testing properly

To know what a box will sustain in the field:

  1. Run the real pipeline, with the real number of streams, not the model alone. Decoding and pre-processing heat the CPU too.
  2. Run it for long enough. At least until temperature has been flat for a good while; an hour is a reasonable minimum, and an overnight run is better.
  3. Run it in the real enclosure. A sealed plastic box on a wall in the sun is a very different environment from an open desk.
  4. Match the ambient temperature. If the site is a warehouse without air conditioning, test somewhere similar, or at least note the room temperature with the results.
  5. Log temperature, clocks and throttle flags alongside latency, so you can line them up afterwards.

Report the steady-state number, after the board has warmed up, as the capacity of the box.

Planning for it

Even with good cooling, leave headroom. A box that needs 100% of its sustained capacity on a cool day will fall behind on a hot one. Some practical habits:

Cheaper fixes before buying hardware

If a box throttles, a few changes can bring the load down without new hardware:

Each of these reduces heat as well as compute, because the board does less work per second.

Checklist

Next steps: Power, Cooling and Storage for a Reliable Edge AI Box and Raspberry Pi vs Jetson Orin Nano for Object Detection.