Receiving analog video over FM

The carrier

A carrier is a pure sine wave at one fixed frequency. On its own it carries no information, because every cycle looks like the last.

The arrow on the right is the same wave drawn as a rotating vector, called a phasor. Its length is the amplitude and it turns one full circle per cycle. The height of its tip, traced over time, draws the wave. I/Q sampling measures this arrow directly, so it will come back.

The message

The message is the information to send: audio for broadcast radio, or composite video for an analog FPV link. Here it is a single slow tone.

Its frequency is far below the carrier's. That separation is what lets the two be combined for transmission and pulled apart again at the receiver.

Frequency modulation

FM lets the message steer the carrier's frequency. When the message is high the carrier speeds up, and when it is low the carrier slows down. The amplitude never changes, so all of the information is in timing.

\Delta f is the peak deviation: how far the frequency swings at the message's extremes. Frequency is the rate of change of phase, so the phase is the running total of the frequency:

For a single tone \class{c-message}{m(t) = \sin(2\pi f_m t)}, the integral works out to a cosine, up to a constant:

\beta is the modulation index. It is also the largest angle, in radians, that the FM signal's phase gets ahead of or behind an unmodulated carrier. Turn on the reference carrier to watch that lead and lag.

The arrow stays the same length but speeds up and slows down with the message. Demodulation is the job of measuring that rotation speed.

Mixing with a local oscillator

The FM signal's information is in its phase, but the receiver can't read phase from a single wave with nothing to compare it to. So it makes its own sine wave near the carrier frequency, called the local oscillator (LO), and multiplies the incoming signal by it. Multiplying two signals is called mixing, and the circuit that does it is a mixer.

The product of two sine waves is two new waves, one at the difference of their frequencies and one at their sum:

With the signal near the LO frequency, the difference is small and the sum is large, around twice the carrier. The slow part is the useful one: it depends only on how the signal's phase compares with the LO's.

Set both frequencies to the same value and the slow part stops moving. It becomes a constant level, called DC (after direct current), equal to the cosine of the phase difference. It reads 1 when the two waves are in step, 0 when they are a quarter cycle apart, and −1 when they are opposite. Drag the phase slider to see it.

Low-pass filtering

To keep the slow part and throw away the fast part, the receiver uses a low-pass filter (LPF). A low-pass filter lets low frequencies through and blocks high ones.

The simplest low-pass filter is a moving average: each output is the average of the input over the last T seconds.

A fast wiggle averages to nearly zero, because its ups and downs cancel inside the window. A slow wave barely changes across the window, so its average is close to its current value.

How much of each frequency survives depends on the window length, and the panel on the right plots that gain against frequency. A longer window blocks more of the fast part, but it also shrinks the slow part and delays it, because the average lags half a window behind. The fast part cancels completely whenever the window holds a whole number of its cycles, which is where the gain curve touches zero.

Two mixers: I and Q

One mixer and a filter give the cosine of the phase difference. That isn't enough: the cosine of +45° and −45° are equal, so the output can't tell whether the signal is ahead of the LO or behind it.

The fix is a second mixer whose LO is shifted a quarter cycle ahead. The first mixer's filtered output is called I, for in-phase, and the second's is Q, for quadrature (a quarter turn apart).

I is the cosine of the phase difference and Q is its sine, so together they are the coordinates of the FM signal's phasor, drawn on a plane called the IQ plane. It is the same arrow as before, but seen from a viewpoint that rotates along with the LO:

When the LO sits exactly on the carrier, the carrier's rotation cancels out and the arrow only swings back and forth by \pm\beta. A tuning error adds a slow, steady spin on top of the swing.

Two numbers also settle the direction of rotation. With I alone, an arrow turning counterclockwise looks the same as one turning clockwise, which is the difference between a signal above the LO and one below it. Q tells them apart.

Reading the phase

The IQ plane turns demodulation into geometry. The message is in how fast the arrow turns, so the first step is to find which way the arrow points: its angle \class{c-modulated}{\theta}, measured counterclockwise from the positive I axis.

atan2 is the arctangent extended to all four quadrants. The plain arctangent of Q/I can't tell the point (1, 1) from (−1, −1), because the ratio is the same, so atan2 also looks at the signs of I and Q. It returns an angle between −180° and +180°.

That range has a catch. When the arrow turns past 180°, the angle jumps to −180°, even though the arrow barely moved. This is called wrapping. Set a tuning error so the arrow keeps spinning, and the phase trace becomes a sawtooth of these jumps.

Frequency is the slope of the phase

Frequency is how fast phase changes, so the instantaneous frequency is the slope of \theta over time. For now, set the wrapping aside and look at the phase as one continuous curve, which is called unwrapped phase.

The phase curve is steepest where the message is at its peaks, and flat where the message crosses zero. With the LO exactly on the carrier, the slope is the message scaled by \Delta f, so measuring the slope gives the message back. That is FM demodulation.

Measuring between samples

A digital receiver doesn't have a continuous \theta(t). It has samples of I and Q taken f_s times per second, called the sample rate. So it approximates the slope with the change in angle from one sample to the next:

Here n numbers the samples and \hat f is the estimated frequency. This is called a phase-difference discriminator.

The wrap step deals with the jumps from before. If the difference comes out as +350°, the arrow really turned −10°, so wrap adds or subtracts 360° to bring every difference into the range ±180°.

That same step sets a speed limit. A turn of +200° between samples looks exactly like a turn of −160°, so the arrow must turn less than half a circle per sample. Any frequency beyond \pm f_s/2 folds back inside that range, which is called aliasing. Lower the sample rate or raise \Delta f and watch the green samples leave the orange curve at the dashed limits.

A tuning error \delta adds -\delta to every sample, so it shifts the output up or down without changing its shape. A steady shift like this is called a DC offset. One way to remove it is to measure it and subtract it: average the output over a window long enough to hold several message cycles, where the message itself averages to zero, and what remains is the offset. Subtracting a slow average like this is a high-pass filter, the opposite of the low-pass filter from before. Set a tuning error and turn on "Remove DC offset" to see the yellow estimate find the offset and the output recenter.

This has two limits. First, it only works when the message averages to zero. A tone does, but composite video doesn't, because its brightness levels carry meaning, so video receivers instead line the output up against a known reference level in the signal. Second, subtracting the offset doesn't restore headroom. The offset pushes the output toward one of the \pm f_s/2 limits, so aliasing starts at a smaller \Delta f, and the folding has already happened by the time the offset is subtracted. The dashed limits move with the subtraction to show this. Fixing that means retuning the LO itself, which is called automatic frequency control (AFC).

Noise pushes each sample off the circle. Only the angle matters, so the error depends on how large the noise is compared with the arrow's length. Raise the noise to see the samples scatter. Averaging neighboring samples, which is the same moving-average low-pass filter as before, smooths them back out. The cost is detail and delay: the average lags by half its length, and it also flattens fast changes in the message.

Composite video

The message in a 5.8 GHz analog video link is a video signal. An analog camera sends its picture as one voltage that varies over time, called composite video or CVBS (color, video, blanking and sync). It is "composite" because brightness, color and timing all share that one wire. This page uses NTSC, the standard used in North America and Japan.

The picture is sent one horizontal line at a time, left to right, top to bottom. Each line lasts 63.6 µs, so there are about 15,734 lines per second. A full frame is 525 lines, about 30 frames per second.

Within each line, the voltage means different things at different times. Levels are measured in IRE units, where 0 is the blanking level and 100 is white. The whole range from −40 to 100 IRE spans 1 V.

• Horizontal sync (4.7 µs) drops to −40 IRE, the sync tip. Nothing in the picture goes that low, so the receiver can always find it. It marks the start of a new line.
• The back porch (4.7 µs) sits at blanking level after sync. It gives the receiver a known reference level, and later holds the color burst.
• Active video (52.7 µs) is the picture itself. Higher voltage is brighter, from black at 7.5 IRE to white at 100 IRE.
• The front porch (1.5 µs) returns to blanking level before the next sync, so the end of the picture can't blur into the sync edge.

Sync, the porches and anything between black and blanking all show as black on screen. The gap between blanking (0) and black (7.5) is called setup or pedestal. It is specific to North American NTSC; Japanese NTSC and PAL put black at blanking.

This whole waveform is the message m(t) from the start of the page. An FM video transmitter swings its frequency with this voltage, so the sync tip sits at one end of the deviation and white at the other, and the demodulator's output should reproduce it.

Scanning a picture

A receiver turns the waveform back into a picture by sweeping a spot across the screen in step with it. The spot's brightness follows the voltage during active video. Each sync pulse sends it back to the left edge of the next line, and the blanking around sync covers the time that return takes.

The figure below does this with a small greyscale picture. The magenta spot on the picture and the magenta cursor on the waveform are the same moment in time. The panel on the right spans a whole line of time, so the spot spends part of each line in the blanking strips either side of the picture.

The figure simplifies a real frame. It has 60 lines instead of 525, it runs millions of times slower, and it jumps straight from the bottom line back to the top. A real signal pauses at the bottom of each picture, which the next section covers.

Vertical blanking

At the bottom of each picture the spot has to return to the top, which takes much longer than returning to the start of a line. The signal spends about 20 lines on this, called the vertical blanking interval, and nothing in those lines is drawn.

The receiver needs a way to tell this apart from an ordinary line. That is the vertical sync pulse: instead of a 4.7 µs dip, the signal stays at the sync tip for most of three lines. A receiver that smooths the signal sees a much longer stretch of sync and knows to return to the top. The sync is cut into six broad pulses by short gaps, called serrations, so horizontal timing keeps going through it.

Either side of vertical sync are three lines of equalizing pulses, narrow pulses at twice the line rate. They exist because of interlacing. A 525-line frame is sent as two fields of 262.5 lines each, the first carrying the odd lines and the second the even lines. That half line shifts the second field's lines down so they land between the first field's. Toggle the second field to see its horizontal sync start half a line later relative to vertical sync. The equalizing pulses make the interval look the same to the vertical sync detector either way, so both fields start at the same height.

After vertical sync come blank lines with ordinary horizontal sync, which give the spot time to settle before the picture resumes around line 21. Each field takes 1/60 s (59.94 Hz exactly), so the screen is refreshed 60 times a second while the full picture updates 30 times.

The color subcarrier

Color was added to NTSC in 1953 without breaking the black-and-white sets already in use. The brightness signal stays exactly as before, now called luma or Y. Color rides on top of it as a fast sine wave, the color subcarrier, at 3.579545 MHz. The subcarrier's amplitude and phase carry the color, and together they're called chroma. A black-and-white set shows the subcarrier as a fine pattern of dots, too small to notice from a distance.

Reading a phase needs a reference, so each line carries one. On the back porch, right after sync, the transmitter sends about nine cycles of the subcarrier at a fixed phase, called the color burst. The receiver locks its own oscillator to the burst and holds it through the rest of the line. A receiver that finds no burst shows the picture in black and white.

The figure shows one line of color bars, a standard test signal. At the full-line scale the subcarrier's 227.5 cycles per line blur into a band, so the lower lane magnifies a 3 µs window. Move it to see the burst, and then how each bar has its own amplitude and phase.

Hue and saturation

Chroma is built from two color-difference signals, \class{c-i}{U} (blue minus luma) and \class{c-q}{V} (red minus luma), each on its own copy of the subcarrier, a quarter cycle apart:

This is the same form as the I/Q signals from the FM section: two amplitudes on quadrature carriers. The receiver separates them the same way, by multiplying the chroma by the sine and cosine of its burst-locked reference and low-pass filtering each product. With Y, U and V it can solve for red, green and blue.

Drawn as a phasor, chroma's angle is the hue and its length is the saturation. A grey pixel has no chroma at all. A plot of this plane is called a vectorscope, and its boxes mark where each standard color bar should land. The burst sits at 180°, pointing along -U.

If the receiver's reference slips relative to the burst, every hue rotates by the same angle, just as a tuning error spun the FM phasor. Set a reference phase error to see the decoded color shift. Transmission paths that shifted the subcarrier's phase caused exactly this, which is why NTSC sets had a tint control.

The NTSC standard itself uses a pair of axes named I and Q, rotated 33° from U and V. They're the same idea at a different angle; this page uses U and V to keep them apart from the radio's I and Q.

A color picture

Putting it together, a color line carries three things at once: sync and blanking for timing, luma for brightness, and chroma riding on the luma for color. A black-and-white set reads only the luma line through the middle of the band and ignores the rest.

The figure scans the earlier picture in color, with color bars in place of the grey scale. The shaded band on the waveform is the chroma: its height is the saturation, and its phase, too fast to draw here, is the hue. Burst appears on every line's back porch except during vertical sync. The top rows of the panel are a shortened vertical blanking interval, a few lines of broad sync pulses and then blank lines.

The modem's debug bus

The Wi-Fi modem can't demodulate analog FM, but the chip has a diagnostic bus, MODEM_DIAG, that can route signals from inside the modem to other peripherals. The modem samples I and Q about 80 million times a second, 10 bits each. C5VRX routes the top four bits of I and the top four of Q onto the bus, one byte per sample. The parallel I/O peripheral (PARLIO) captures every second sample, 40 million a second, straight into a ring buffer in memory.

Quantized I/Q

Four bits gives each of I and Q only 16 possible values, so every sample lands on a 16 by 16 grid. The phasor's circle becomes a ring of grid points, and the angle of each sample is the angle of the nearest point rather than the true one.

A weak signal makes a small circle that crosses only a few grid points, and near the center the angle between neighboring points jumps by large steps. That is where most of the picture's static comes from.

Shrink the amplitude to see the ring of points collapse into a few coarse steps. More bits make the grid finer.

Output codes

Everything C5VRX computes ends as an output code: a 6-bit number from 0 to 63, sent 20 million times a second. Six GPIO pins carry the six bits to a resistor ladder that turns each code into a voltage, covered at the end of this part. With 0 as the sync tip and 63 as white, blanking lands near 18 and black near 21, which leaves about 40 steps for everything from black to white.

Twenty million codes a second is enough for the whole video signal, not just the brightness. A sampled signal can represent frequencies up to half its sample rate, here 10 MHz. NTSC luma extends to about 4.2 MHz, and chroma sits around the 3.58 MHz subcarrier with sidebands reaching about the same limit, so both fit with room to spare. Each subcarrier cycle gets about 5.6 samples, enough to keep the burst's phase and every bar's hue.

Six bits is about as far as a resistor ladder goes. The ladder uses one resistor per bit, each about half the one before, and with standard resistor values each code already lands up to half a step off a straight line. A seventh bit would halve the step, and the error would be as large as the step itself.

A smooth change in brightness has to jump from one step to the next, so a gradient turns into visible bands. Reduce the bits to see the bands widen; each bit removed halves the number of steps.

Phase from a lookup table

At 40 million samples a second there's no time for atan2. Every 8-bit sample has only 256 possible values, so the angle of each one can be computed ahead of time and stored in a table. C5VRX stores it as a 5-bit phase, 32 steps of 11.25° numbered counterclockwise from the I axis.

Phase change between samples

The output runs at 20 million codes a second, half the capture rate, so C5VRX keeps every second sample and measures how far the phase moved across two sample periods, 50 ns. This is the phase-difference discriminator from the FM part, with f_s at 20 MHz.

Instead of subtracting, it uses a second table, indexed by the previous and current phase, that returns the output code directly. There are only 32 × 32 = 1024 pairs, so the rest of the work can be computed ahead of time and stored in the table: wrapping the difference, scaling it to a frequency, and turning that into a video level. The next three sections take these one at a time.

Wrapping the difference

The phases go around a circle, so a sample moving from phase 30 to phase 1 has turned 3 steps forward, not 29 back. As with the wrap in the FM part, the table treats every difference as the shortest way around, a step between −16 and +15.

That sets the same speed limit: less than half a turn in 50 ns, or ±10 MHz. A step of +20 looks exactly like a step of −12, so the table can't tell them apart.

Steps to frequency

Each step is 1/32 of a turn in 50 ns. From the discriminator formula that's f_s/32, 625 kHz, so the step count is the signal's frequency relative to the tuned frequency, in units of 625 kHz.

Brightness is carried as frequency, so the table multiplies the step by a gain: each step adds about six to the output code. With the receiver tuned to the blanking level, sync sits about three steps below it, about 1.9 MHz, and white about seven above, about 4.4 MHz.

Offset and clamp

A step of zero means the signal is at the tuned frequency, which is blanking, so the table adds an offset that puts zero steps at code 20. That's a couple of codes above the 18 calculated from the output levels.

Codes stop at 0 and 63, so steps past sync clamp to 0 and steps past white clamp to 63. A little further out, 9 to 11 steps either way, the table eases back toward blanking.

A step of 12 or more either way is too far to trust, since it could be the wrap from the other direction, so the table sends it to blanking. Noise near the center of the I/Q plane makes large random steps, and sending those to sync instead would make false sync pulses that tear the picture.

The figure shows one row of the table. Set the previous and current phase to see the wrapped step between them, and where that step lands among the output codes.

Both tables at work

The ESP32-C5's BitScrambler, a small programmable unit that rewrites a DMA stream as it passes, does both lookups for every sample without the CPU.

The figure feeds a line of grey staircase video through the real tables from C5VRX. Lower the amplitude or add noise to see the phase jitter between neighboring cells, and the output code jitter with it.

A resistor ladder

Six GPIO pins carry the six bits of each code, and each pin drives the video output through its own resistor, from 8.2 kΩ for the lowest bit to 240 Ω for the highest. Each resistor is about half the one before, so each bit adds about twice the current of the bit below it, and the voltage at the junction follows the code.

A TV input is a 75 Ω load, and together with a 200 Ω resistor to ground it sets the scale: all six bits high gives about 1 V, the full video range. Pins that are low still connect their resistors, pulling current back out of the junction. Disconnect the TV to see the voltage roughly double.

The resistors are standard values, so they're only close to doubling. Each code lands up to about 8 mV off a straight line, half the 16 mV between codes.