# Cutting drone video latency from 333 ms to 89 ms on a Raspberry Pi Zero 2 W

> How we measured camera-to-screen delay on a Pi Zero 2 W video link, where the time actually went, and the changes that took it from a third of a second to under 90 ms. Most of the gain came from places no tool was watching.

Level: Advanced. Published 2026-11-14 by Goodchild Engineering. Web version: https://goodchildengineering.com/blog/cutting-drone-video-latency/

Products: https://goodchildengineering.com/products/flighthat/

![Two columns of clock readings. Each row pairs the millisecond clock on the screen with the same clock as seen through the camera and video link. On the left (start) the video trails the clock by about a third of a second; on the right (after) by about a tenth.](https://goodchildengineering.com/images/blog/cutting-drone-video-latency.jpg)

When you fly a drone from its camera, every millisecond between the scene and your screen is time you
spend reacting to the past. At 10 m/s, 300 ms of video delay means you see the vehicle 3 m behind where it
really is. We wanted a low-latency video link from a Raspberry Pi Zero 2 W, the smallest host the
[flightHAT](https://goodchildengineering.com/products/flighthat/) supports, to a ground station running QGroundControl.

We started at **333 ms** and ended at **89 ms** on a Steam Deck. The biggest wins weren't encoder tuning.
They came from a decoder setting on the PC, from picking a sensor mode the Pi could actually keep up with,
and from removing a one-frame wait hiding in a pipe.

## The setup

| Part | What we used |
| --- | --- |
| Camera host | Raspberry Pi Zero 2 W (4 × Cortex-A53 at 1 GHz, 512 MB, VideoCore IV ISP and H.264 encoder) |
| Cameras | Sony IMX415 module (8 MP), then a Sony IMX335 module (5 MP, with a 2 × 2 binned mode) |
| Encoding | The Pi's hardware H.264 encoder, RTP over UDP |
| Links | USB networking to a Windows PC; the Pi's own Wi-Fi access point to the Steam Deck |
| Ground stations | Windows 11 PC with a 60 Hz monitor; Steam Deck OLED (90 Hz) with a GStreamer viewer and with QGroundControl |

The video path has stages on both sides of the link, and each one can hold a frame:

```
camera host (Pi):  exposure → sensor readout → ISP → H.264 encode → RTP/UDP
link:              USB or Wi-Fi
ground station:    jitter buffer → decode → compositor → screen
```

## How we measured it

We measured two ways, and needed both.

**Glass to glass** is the number that matters: light in at the lens, light out of the screen. We put a
millisecond clock on the ground station's screen, pointed the camera at it, and showed the video next to
it. One screenshot then holds both the live clock and the clock as the camera saw it some time ago. The
difference is the latency.

![The test bench: a Raspberry Pi Zero 2 W on a wooden desk with its camera module held in a small vise, pointed at a Steam Deck about half a metre away. The Deck's screen shows a large clock reading 0:00:39.286 and, below it, the camera's live video of the Deck showing the clock about a tenth of a second earlier.](https://goodchildengineering.com/images/blog/cutting-drone-video-latency/bench-setup.jpg)

The bench during the Steam Deck tests: the Pi Zero 2 W (right) with its camera held in a vise, streaming
over the Pi's own Wi-Fi to the Deck. The Deck shows the clock at the top and the camera's video of itself
below, so every video frame holds an older copy of the clock. A screenshot on the Deck captures both at the
same instant:

![A Steam Deck screen split in two: at the top a large clock reading 0:05:37.653; below it the live video from the Pi's camera, which is pointed at the same screen and shows the clock as 0:05:37.58, about 70 ms behind.](https://goodchildengineering.com/images/blog/cutting-drone-video-latency/steam-deck-glass-to-glass.jpg)

Each reading is only good to one screen refresh (±17 ms at 60 Hz), so we took 20 screenshots per setup
and used the median. The hero image above shows five of the twenty from the start and the end of this
work.

**Per-stage timing** tells you where to look. On the Pi we timestamped each frame from the sensor's own
timestamp through the encoder output, all on the same monotonic clock. On the PC we used GStreamer's latency
tracer for the receive side, and ping for the link.

Neither is enough alone. Early on, the stages we could time added up to 140 ms while glass to glass was
283 ms. The missing time was in places no tool watched: a wait inside a pipe, the desktop compositor and the
monitor.

## The results

![Bar chart of median glass-to-glass latency at each step: 333 ms with rpicam-vid piped into GStreamer, 283 ms with a GStreamer-only sender, 167 ms with the IMX335 in a binned 720p30 mode, 108 ms with whole frames fed into GStreamer from Picamera2, and 89 ms for the same stream on a Steam Deck.](https://goodchildengineering.com/images/blog/cutting-drone-video-latency/glass-to-glass-steps.svg)

| Step | Camera and mode | Ground station | Glass to glass, median (range) |
| --- | --- | --- | --- |
| Start: `rpicam-vid` piped into GStreamer | IMX415, 1080p15 | PC | 333 ms (267–384) |
| GStreamer-only sender | IMX415, 1080p15 | PC | 283 ms (233–316) |
| Binned sensor mode | IMX335, 720p30 | PC | 167 ms (150–184) |
| Whole frames into GStreamer | IMX335, 720p30 | PC | 108 ms (83–133) |
| Same stream | IMX335, 720p30 | Steam Deck, Wi-Fi | **89 ms (60–122)** |

All of these already include the first fix below, on the PC's decoder.

## What made the difference

### 1. The ground station's decoder was holding frames (−190 ms)

The first surprise was on the PC. Windows' hardware H.264 decoders (Direct3D 11, Direct3D 12 and NVIDIA's)
each held two or three frames before showing anything, whatever we set. At 15 fps that was 134–204 ms.

A plain software decoder, FFmpeg's `avdec_h264` limited to **one thread**, shows each frame as soon as it's
decoded: about 4 ms for 1080p on a desktop CPU. The single thread matters, because FFmpeg's frame threading
adds a frame of delay per thread.

| PC decoder | Delay it added |
| --- | --- |
| Direct3D 11 / 12 or NVDEC hardware decoders | 134–204 ms (2–3 frames held) |
| `avdec_h264`, default threads | about one frame per thread |
| **`avdec_h264 max-threads=1`** | **≈ 4 ms** |

H.264's Baseline profile only helps a decoder that reorders frames. With the software decoder, High profile
is just as fast and looks better.

### 2. Budget the ISP in sensor pixels per second (−165 ms)

The Pi's image signal processor (ISP) turns the sensor's raw data into a picture. On the Zero 2 W it
managed about **130 megapixels per second** at its stock clock. What it has to process is the *sensor*
frame, not the output frame:

$$
\text{ISP load} = W_\text{sensor} \times H_\text{sensor} \times \text{fps}
$$

The IMX415's driver only offers its full 3864 × 2192 mode. At 15 fps that's 127 Mpixel/s, just inside the
budget. Asking for 30 fps didn't give 30 fps. It gave 17 fps and *more* latency, because frames queued behind
the overloaded ISP. Cropping didn't help either: the ISP's first stages run over the whole frame before any
crop.

![Bar chart of sensor pixel rate against the Zero 2 W ISP limit of about 130 Mpixel/s. IMX415 full frame at 15 fps is 127 Mpixel/s; at 30 fps it would be 254 Mpixel/s and only 17 fps came out; the IMX335 2×2 binned mode is 38 Mpixel/s at 30 fps and 77 at 60 fps.](https://goodchildengineering.com/images/blog/cutting-drone-video-latency/isp-pixel-budget.svg)

The IMX335 has a 2 × 2 binned mode (1312 × 972) that keeps the full field of view at a quarter of the
pixels: 38 Mpixel/s at 30 fps. Readout plus ISP time dropped from 128 ms to 33 ms. Overclocking the ISP
wasn't the answer: 400 MHz took the IMX415 only to about 20 fps and isn't trustworthy without overvolting.

Using the IMX335 on the Pi needed one extra piece. libcamera needs a small sensor-specific "camera helper"
compiled into its image-processing module, and the stock build didn't include one for this sensor. We
cross-compiled it in Docker in about 3 minutes. Building it on the Pi itself took an hour and ran it out of
memory.

### 3. Keep frames whole from the encoder to the network (−59 ms)

The common recipe pipes the encoder's output into GStreamer:

```bash
rpicam-vid -o - | gst-launch-1.0 fdsrc ! h264parse ! rtph264pay ! udpsink ...
```

The pipe turns frames into a plain byte stream, so `h264parse` can't tell a frame has ended until the
*next* one starts. That costs one frame, and the pipe adds buffering of its own. We replaced it with a small
Python sender: Picamera2 drives the camera and the hardware encoder, and each encoded frame goes into
GStreamer as one complete buffer (`appsrc`), straight to the RTP payloader. That took 167 ms to 108 ms.

### 4. The rest: faster readout, and things that didn't matter

- **A faster sensor link helps even at the same frame rate.** Doubling the IMX415's CSI link from 360 to
  720 MHz cut readout plus ISP from 128 to 91 ms at the same 15 fps.
- **The jitter buffer cost nothing.** GStreamer's `rtpjitterbuffer` only holds packets that are missing or
  out of order, so on a clean link 0 ms and 50 ms settings measured the same. We kept 50 ms for Wi-Fi.
- **The USB link** was under 1 ms.
- **The Pi's CPU** was 85 % idle at 30 fps. The limits were all in the VideoCore's ISP and encoder blocks.

## Where the 108 ms goes now

![Timeline of the 108 ms path: half the exposure 13 ms, sensor readout and ISP 33 ms, H.264 encode 18 ms on the Pi; link and jitter buffer about 1 ms; software decode 4 ms; and about 39 ms of display time, the remainder after the timed stages.](https://goodchildengineering.com/images/blog/cutting-drone-video-latency/latency-budget.svg)

The clock starts at the middle of the exposure, the moment the picture is "taken" on average. The
camera host now accounts for about 64 ms, the link almost nothing, and the ground station about 43 ms.
Most of that is the last stretch, compositor, vsync and the monitor itself, which we inferred as the
remainder rather than timed directly. The Steam Deck's 90 Hz screen is one reason it measured 89 ms
instead of 108.

## Above 30 fps, frame rate stopped mattering

Most latency is counted in frames: readout, ISP, encode and every queue each cost about one frame period. So
we expected 60 fps to roughly halve it. It didn't.

![Line chart of glass-to-glass latency at 30, 50 and 60 fps. PC with a 60 Hz monitor: 108, 100, 117 ms. Steam Deck with a 90 Hz screen: 89, 92, 100 ms.](https://goodchildengineering.com/images/blog/cutting-drone-video-latency/frame-rate-vs-latency.svg)

| Frame rate | PC, 60 Hz monitor | Steam Deck, 90 Hz |
| --- | --- | --- |
| 30 fps | 108 ms | **89 ms** |
| 50 fps | 100 ms | 92 ms |
| 60 fps | 117 ms | 100 ms |

The IMX335's binned mode always reads out at its 60 fps speed (a lower frame rate just waits longer between
frames), and the ISP and encoder take the same time per frame at any rate. Only the exposure shrinks. At
60 fps the Pi was at its limit (58 fps delivered, a few drops), and latency got slightly worse. Once the
sensor reads out fast, pick the frame rate for smoothness and load, not latency. We settled on 720p30.

## The ground-station app has its own buffer

Then we tried the app you actually fly with. QGroundControl played the stream with its default video
settings (UDP H.264 on port 5600), with nothing to configure on the ground station. But by default it
syncs frames to a clock and buffers them for smoothness.

![Two frames from QGroundControl on the Steam Deck, each showing the live clock above the camera's view of it. Left, default settings: the clock reads 24.540 s and the video shows 24.162 s, 378 ms behind. Right, Low Latency Mode on: 47.829 s against about 47.70 s, about 120 ms behind.](https://goodchildengineering.com/images/blog/cutting-drone-video-latency/qgroundcontrol-default-vs-low-latency.jpg)

![Bar chart: a bare GStreamer viewer shows the stream 89 ms behind; QGroundControl with Low Latency Mode on 113 ms; QGroundControl with its default video settings 355 ms.](https://goodchildengineering.com/images/blog/cutting-drone-video-latency/qgroundcontrol-low-latency.svg)

**Turn on Low Latency Mode** (Application Settings → Video). It took QGroundControl from 355 ms to 113 ms,
only 24 ms more than a bare viewer. Whatever viewer you use, measure glass to glass in the app you fly with.

## A checklist for your own camera link

1. **List the sensor modes** and work out `width × height × fps` against the host ISP's throughput. A binned
   mode is often the answer.
2. **Check the delivered frame rate**, with even frame spacing (`rpicam-vid --save-pts`), before measuring
   latency. A stage that can't keep up makes latency worse, not just the frame rate.
3. **Use the fastest sensor link** (CSI lane rate) the driver offers.
4. **Keep encoded frames whole** from the encoder to the RTP payloader. No byte-stream pipes.
5. **On the ground station, use a decoder that doesn't hold frames.** Check it with GStreamer's latency
   tracer.
6. **Don't run the host at its limit.** Above about 30 fps, extra frames buy smoothness, not latency.
7. **Measure glass to glass** with 20 samples and take the median, in the ground-station app itself.
   In QGroundControl, turn on Low Latency Mode.

## Further reading

- [Raspberry Pi camera software documentation](https://www.raspberrypi.com/documentation/computers/camera_software.html) (rpicam-apps and libcamera)
- [Picamera2 library](https://github.com/raspberrypi/picamera2)
- [GStreamer latency tracer](https://gstreamer.freedesktop.org/documentation/additional/design/tracing.html)
- [QGroundControl video settings](https://docs.qgroundcontrol.com/master/en/qgc-user-guide/settings_view/general.html)
- Our earlier post on [flight controllers and companion computers](https://goodchildengineering.com/blog/flight-controller-vs-companion-computer/), which covers why the video runs on the companion computer