Receiving analog video over FM

The carrier

A carrier is a pure sine wave at one fixed frequency. On its own it carries no information, because every cycle looks like the last.

The arrow on the right is the same wave drawn as a rotating vector, called a phasor. Its length is the amplitude and it turns one full circle per cycle. The height of its tip, traced over time, draws the wave. I/Q sampling measures this arrow directly, so it will come back.

The signal

The signal is the information to send: audio for broadcast radio, or composite video for an analog FPV link. Here it is a single slow tone.

Its frequency is far below the carrier's. That separation is what lets the two be combined for transmission and pulled apart again at the receiver.

Frequency modulation

FM lets the signal steer the carrier's frequency. When the signal is high the carrier speeds up, and when it is low the carrier slows down. The amplitude never changes, so all of the information is in timing.

\Delta f is the peak deviation: how far the frequency swings at the signal's extremes. Frequency is the rate of change of phase, so the phase is the running total of the frequency:

For a single tone \class{c-message}{m(t) = \sin(2\pi f_m t)}, the integral works out to a cosine, up to a constant:

\beta is the modulation index. It is also the largest angle, in radians, that the FM signal's phase gets ahead of or behind an unmodulated carrier. Turn on the reference carrier to watch that lead and lag.

The arrow stays the same length but speeds up and slows down with the signal. Demodulation is the job of measuring that rotation speed.

Mixing with a local oscillator

The FM signal's information is in its phase, but the receiver can't read phase from a single wave with nothing to compare it to. So it makes its own sine wave near the carrier frequency, called the local oscillator (LO), and multiplies the incoming signal by it. Multiplying two signals is called mixing, and the circuit that does it is a mixer.

The product of two sine waves is two new waves, one at the difference of their frequencies and one at their sum:

With the signal near the LO frequency, the difference is small and the sum is large, around twice the carrier. The slow part is the useful one: it depends only on how the signal's phase compares with the LO's.

Set both frequencies to the same value and the slow part stops moving. It becomes a constant level, called DC (after direct current), equal to the cosine of the phase difference. It reads 1 when the two waves are in step, 0 when they are a quarter cycle apart, and −1 when they are opposite. Drag the phase slider to see it.

Low-pass filtering

To keep the slow part and throw away the fast part, the receiver uses a low-pass filter (LPF). A low-pass filter lets low frequencies through and blocks high ones.

The simplest low-pass filter is a moving average: each output is the average of the input over the last T seconds.

A fast wiggle averages to nearly zero, because its ups and downs cancel inside the window. A slow wave barely changes across the window, so its average is close to its current value.

How much of each frequency survives depends on the window length, and the panel on the right plots that gain against frequency. A longer window blocks more of the fast part, but it also shrinks the slow part and delays it, because the average lags half a window behind. The fast part cancels completely whenever the window holds a whole number of its cycles, which is where the gain curve touches zero.

Two mixers: I and Q

One mixer and a filter give the cosine of the phase difference. That isn't enough: the cosine of +45° and −45° are equal, so the output can't tell whether the signal is ahead of the LO or behind it.

The fix is a second mixer whose LO is shifted a quarter cycle ahead. The first mixer's filtered output is called I, for in-phase, and the second's is Q, for quadrature (a quarter turn apart).

I is the cosine of the phase difference and Q is its sine, so together they are the coordinates of the FM signal's phasor, drawn on a plane called the IQ plane. It is the same arrow as before, but seen from a viewpoint that rotates along with the LO:

When the LO sits exactly on the carrier, the carrier's rotation cancels out and the arrow only swings back and forth by \pm\beta. A tuning error adds a slow, steady spin on top of the swing.

Two numbers also settle the direction of rotation. With I alone, an arrow turning counterclockwise looks the same as one turning clockwise, which is the difference between a signal above the LO and one below it. Q tells them apart.

Reading the phase

The IQ plane turns demodulation into geometry. The signal is in how fast the arrow turns, so the first step is to find which way the arrow points: its angle \class{c-modulated}{\theta}, measured counterclockwise from the positive I axis.

atan2 is the arctangent extended to all four quadrants. The plain arctangent of Q/I can't tell the point (1, 1) from (−1, −1), because the ratio is the same, so atan2 also looks at the signs of I and Q. It returns an angle between −180° and +180°.

That range has a catch. When the arrow turns past 180°, the angle jumps to −180°, even though the arrow barely moved. This is called wrapping. Set a tuning error so the arrow keeps spinning, and the phase trace becomes a sawtooth of these jumps.

Frequency is the slope of the phase

Frequency is how fast phase changes, so the instantaneous frequency is the slope of \theta over time. For now, set the wrapping aside and look at the phase as one continuous curve, which is called unwrapped phase.

The phase curve is steepest where the signal is at its peaks, and flat where the signal crosses zero. With the LO exactly on the carrier, the slope is the signal scaled by \Delta f, so measuring the slope gives the signal back. That is FM demodulation.

Measuring between samples

A digital receiver doesn't have a continuous \theta(t). It has samples of I and Q taken f_s times per second, called the sample rate. So it approximates the slope with the change in angle from one sample to the next:

Here n numbers the samples and \hat f is the estimated frequency. This is called a phase-difference discriminator.

The wrap step deals with the jumps from before. If the difference comes out as +350°, the arrow really turned −10°, so wrap adds or subtracts 360° to bring every difference into the range ±180°.

That same step sets a speed limit. A turn of +200° between samples looks exactly like a turn of −160°, so the arrow must turn less than half a circle per sample. Any frequency beyond \pm f_s/2 folds back inside that range, which is called aliasing. Lower the sample rate or raise \Delta f and watch the green samples leave the orange curve at the dashed limits.

A tuning error \delta adds -\delta to every sample, so it shifts the output up or down without changing its shape. A steady shift like this is called a DC offset. One way to remove it is to measure it and subtract it: average the output over a window long enough to hold several signal cycles, where the signal itself averages to zero, and what remains is the offset. Subtracting a slow average like this is a high-pass filter, the opposite of the low-pass filter from before. Set a tuning error and turn on "Remove DC offset" to see the yellow estimate find the offset and the output recenter.

This has two limits. First, it only works when the signal averages to zero. A tone does, but composite video doesn't, because its brightness levels carry meaning, so video receivers instead line the output up against a known reference level in the signal. Second, subtracting the offset doesn't restore headroom. The offset pushes the output toward one of the \pm f_s/2 limits, so aliasing starts at a smaller \Delta f, and the folding has already happened by the time the offset is subtracted. The dashed limits move with the subtraction to show this. Fixing that means retuning the LO itself, which is called automatic frequency control (AFC).

Noise pushes each sample off the circle. Only the angle matters, so the error depends on how large the noise is compared with the arrow's length. Raise the noise to see the samples scatter. Averaging neighboring samples, which is the same moving-average low-pass filter as before, smooths them back out. The cost is detail and delay: the average lags by half its length, and it also flattens fast changes in the signal.

Composite video

The signal carried by a 5.8 GHz analog video link is the video itself. An analog camera sends its picture as one voltage that varies over time, called composite video or CVBS (color, video, blanking and sync). It is "composite" because brightness, color and timing all share that one wire. This page uses NTSC, the standard used in North America and Japan.

The picture is sent one horizontal line at a time, left to right, top to bottom. Each line lasts 63.6 µs, so there are about 15,734 lines per second. A full frame is 525 lines, about 30 frames per second.

Within each line, the voltage means different things at different times. Levels are measured in IRE units, where 0 is the blanking level and 100 is white. The whole range from −40 to 100 IRE spans 1 V.

• Horizontal sync (4.7 µs) drops to −40 IRE, the sync tip. Nothing in the picture goes that low, so the receiver can always find it. It marks the start of a new line.
• The back porch (4.7 µs) sits at blanking level after sync. It gives the receiver a known reference level, and later holds the color burst.
• Active video (52.7 µs) is the picture itself. Higher voltage is brighter, from black at 7.5 IRE to white at 100 IRE.
• The front porch (1.5 µs) returns to blanking level before the next sync, so the end of the picture can't blur into the sync edge.

Sync, the porches and anything between black and blanking all show as black on screen. The gap between blanking (0) and black (7.5) is called setup or pedestal. It is specific to North American NTSC; Japanese NTSC and PAL put black at blanking.

This whole waveform is the signal m(t) from the start of the page. An FM video transmitter swings its frequency with this voltage, so the sync tip sits at one end of the deviation and white at the other, and the demodulator's output should reproduce it.

Scanning a picture

A receiver turns the waveform back into a picture by sweeping a spot across the screen in step with it. The spot's brightness follows the voltage during active video. Each sync pulse sends it back to the left edge of the next line, and the blanking around sync covers the time that return takes.

The figure below does this with a small greyscale picture. The magenta spot on the picture and the magenta cursor on the waveform are the same moment in time. The panel on the right spans a whole line of time, so the spot spends part of each line in the blanking strips either side of the picture.

The figure simplifies a real frame. It has 60 lines instead of 525, it runs millions of times slower, and it jumps straight from the bottom line back to the top. A real signal pauses at the bottom of each picture, which the next section covers.

Vertical blanking

At the bottom of each picture the spot has to return to the top, which takes much longer than returning to the start of a line. The signal spends about 20 lines on this, called the vertical blanking interval, and nothing in those lines is drawn.

The receiver needs a way to tell this apart from an ordinary line. That is the vertical sync pulse: instead of a 4.7 µs dip, the signal stays at the sync tip for most of three lines. A receiver that smooths the signal sees a much longer stretch of sync and knows to return to the top. The sync is cut into six broad pulses by short gaps, called serrations, so horizontal timing keeps going through it.

Either side of vertical sync are three lines of equalizing pulses, narrow pulses at twice the line rate. They exist because of interlacing. A 525-line frame is sent as two fields of 262.5 lines each, the first carrying the odd lines and the second the even lines. That half line shifts the second field's lines down so they land between the first field's. Toggle the second field to see its horizontal sync start half a line later relative to vertical sync. The equalizing pulses make the interval look the same to the vertical sync detector either way, so both fields start at the same height.

After vertical sync come blank lines with ordinary horizontal sync, which give the spot time to settle before the picture resumes around line 21. Each field takes 1/60 s (59.94 Hz exactly), so the screen is refreshed 60 times a second while the full picture updates 30 times.

The color subcarrier

Color was added to NTSC in 1953 without breaking the black-and-white sets already in use. The brightness signal stays exactly as before, now called luma or Y. Color rides on top of it as a fast sine wave, the color subcarrier, at 3.579545 MHz. The subcarrier's amplitude and phase carry the color, and together they're called chroma. A black-and-white set shows the subcarrier as a fine pattern of dots, too small to notice from a distance.

Reading a phase needs a reference, so each line carries one. On the back porch, right after sync, the transmitter sends about nine cycles of the subcarrier at a fixed phase, called the color burst. The receiver locks its own oscillator to the burst and holds it through the rest of the line. A receiver that finds no burst shows the picture in black and white.

The figure shows one line of color bars, a standard test signal. At the full-line scale the subcarrier's 227.5 cycles per line blur into a band, so the lower lane magnifies a 3 µs window. Move it to see the burst, and then how each bar has its own amplitude and phase.

Hue and saturation

Chroma is built from two color-difference signals, \class{c-i}{U} (blue minus luma) and \class{c-q}{V} (red minus luma), each on its own copy of the subcarrier, a quarter cycle apart:

This is the same form as the I/Q signals from the FM section: two amplitudes on quadrature carriers. The receiver separates them the same way, by multiplying the chroma by the sine and cosine of its burst-locked reference and low-pass filtering each product. With Y, U and V it can solve for red, green and blue.

Drawn as a phasor, chroma's angle is the hue and its length is the saturation. A grey pixel has no chroma at all. A plot of this plane is called a vectorscope, and its boxes mark where each standard color bar should land. The burst sits at 180°, pointing along -U.

If the receiver's reference slips relative to the burst, every hue rotates by the same angle, just as a tuning error spun the FM phasor. Set a reference phase error to see the decoded color shift. Transmission paths that shifted the subcarrier's phase caused exactly this, which is why NTSC sets had a tint control.

The NTSC standard itself uses a pair of axes named I and Q, rotated 33° from U and V. They're the same idea at a different angle; this page uses U and V to keep them apart from the radio's I and Q.

A color picture

Putting it together, a color line carries three things at once: sync and blanking for timing, luma for brightness, and chroma riding on the luma for color. A black-and-white set reads only the luma line through the middle of the band and ignores the rest.

The figure scans the earlier picture in color, with color bars in place of the grey scale. The shaded band on the waveform is the chroma: its height is the saturation, and its phase, too fast to draw here, is the hue. Burst appears on every line's back porch except during vertical sync. The top rows of the panel are a shortened vertical blanking interval, a few lines of broad sync pulses and then blank lines.

The modem's debug bus

The Wi-Fi modem can't demodulate analog FM, but the chip has a diagnostic bus, MODEM_DIAG, that can route signals from inside the modem to other peripherals. The modem samples I and Q about 80 million times a second, 10 bits each. C5VRX routes the top four bits of I and the top four of Q onto the bus, one byte per sample. The parallel I/O peripheral (PARLIO) captures every second sample, 40 million a second, straight into a ring buffer in memory.

Quantized I/Q

Four bits gives each of I and Q only 16 possible values, so every sample lands on a 16 by 16 grid. The phasor's circle becomes a ring of grid points, and the angle of each sample is the angle of the nearest point rather than the true one.

A weak signal makes a small circle that crosses only a few grid points, and near the center the angle between neighboring points jumps by large steps. That is where most of the picture's static comes from.

Shrink the amplitude to see the ring of points collapse into a few coarse steps. More bits make the grid finer, which is what the P4 part later takes advantage of.

Keeping the circle the right size

On a 16 by 16 grid the size of the circle decides the picture. A circle that is too small lands on a few cells and the angle jumps. One that is too big piles samples onto the outer ring of cells, the rail, and the circle flattens. The radio's gain sets the size, and C5VRX has to set it itself: the vendor packet AGC hunts on continuous analog FM, so the firmware turns it off and forces one entry of the PHY's gain table, which has up to 90 entries. The controller uses entries 20 and up.

An entry is three settings: an RF stage at the front end, which sets the noise figure, a baseband code, and one of six fine steps within a baseband bank. A fine step is a small change. A baseband or RF change is a bigger one, and shows in the picture while the radio settles, so the controller prefers fine steps, then baseband, then RF.

It never reads the radio's signal strength. Every 200 µs it takes 256 samples from the newest completed receive buffer, four separate 64-byte regions, and measures them on the same 16 by 16 grid the demodulator sees. Each sample's power is about r^2 + 1 in grid steps, and the controller keeps the median (P50), the 90th and 95th percentiles, the share of samples near the center, the share on the rail, and coherence: the share of samples that are strong enough to have a reliable angle and whose phase moved no more than 45° from the one before, which a carrier does and noise does not.

Median power between 13 and 32, with P95 at most 65 and few samples on the rail or near the center, is the healthy band. That is a circle of radius about 3.5 to 5.5 steps out of 8. Move the level in the figure to see the measurements, and the action each would cause.

In the healthy band the controller makes no gain writes at all, since each write disturbs the picture. Outside it, it acts on the first window rather than waiting. It asks for the whole correction at once: it aims for a power of 27 when the signal is too strong and 17 when too weak, converts the difference to decibels, and picks the gain entry whose predicted P50 and P95 land in the band. Predictions come from the response it has learned for each entry, and for an entry it has never measured it trusts only a fine step, about 0.8 dB. A move has to pass a margin first, 0.35 dB for a fine step, 0.75 dB for baseband and 1.5 dB for RF, so a level sitting on the edge of the band does not make it jump back and forth.

After a write it waits for the gain to settle, at least 0.3 ms or three quarters of the time it learned for that kind of move, then checks that the measurements are steady before it trusts them. It uses the result to learn how that entry really changes the power, and scores any drop in coherence or rise in clipping as an artifact. An entry that keeps coming out bad, with 10% of samples on the rail, half near the center, or coherence under 30%, is skipped.

Overload does not wait. With 10% of samples on the rail, or a P95 of 95 or more, the controller drops at once: baseband first, and the RF stage if the next window is still saturated, since baseband overdrive and front-end compression look the same from the rail alone. A second task, woken by the same 200 µs timer, looks at one 64-byte window and calls the controller early if it sees the rail.

Two rules keep it from hunting. Two reversals of direction within 20 ms make it wait for 8 windows, about 1.6 ms, for the next 200 ms, though saturation is never delayed. And with no carrier at all it listens at the table's maximum gain rather than a safe middle one, because a weak carrier at the middle gain looks like no carrier.

Once the gain is at its maximum and the circle is still too small, the controller changes which bits it reads rather than adding gain. The modem's samples are 10 bits, and the C5 sends four of them: the sign and three below it. Dropping one of the bits just under the sign leaves the same angle with the circle twice as large, 6 dB more. The first of these lanes keeps bits 9, 7, 6 and 5, and the second bits 9, 6, 5 and 4, for 12 dB. Switching is a remap of the output pins, so it is instant and exact. Outside its window a lane folds, so a signal that rises past it shows up as rail samples or incoherent junk, and the controller goes back to the coarse lane at once. The finest lane is also capped where the receiver's own noise reaches about a step, because finer would only resolve noise.

As a last resort for a weak carrier at the cap of the finest lane, an automatic mode halves the radio's bandwidth from 40 MHz to 20 MHz after a second. That cuts the noise by about 3 dB and trims fine detail and color, and it goes back after a second of clear signal.

Output codes

Everything C5VRX computes ends as an output code: a 6-bit number from 0 to 63, sent 20 million times a second. Six GPIO pins carry the six bits to a resistor ladder that turns each code into a voltage, covered at the end of this part. With 0 as the sync tip and 63 as white, blanking lands near 18 and black near 21, which leaves about 40 steps for everything from black to white.

Twenty million codes a second is enough for the whole video signal, not just the brightness. A sampled signal can represent frequencies up to half its sample rate, here 10 MHz. NTSC luma extends to about 4.2 MHz, and chroma sits around the 3.58 MHz subcarrier with sidebands reaching about the same limit, so both fit with room to spare. Each subcarrier cycle gets about 5.6 samples, enough to keep the burst's phase and every bar's hue.

Six bits is about as far as a resistor ladder goes. The ladder uses one resistor per bit, each about half the one before, and with standard resistor values each code already lands up to half a step off a straight line. A seventh bit would halve the step, and the error would be as large as the step itself.

A smooth change in brightness has to jump from one step to the next, so a gradient turns into visible bands. Reduce the bits to see the bands widen; each bit removed halves the number of steps.

Phase from a lookup table

At 40 million samples a second there's no time for atan2. Every 8-bit sample has only 256 possible values, so the angle of each one can be computed ahead of time and stored in a table. C5VRX stores it as a 5-bit phase, 32 steps of 11.25° numbered counterclockwise from the I axis.

Phase change between samples

The output runs at 20 million codes a second, half the capture rate, so C5VRX keeps every second sample and measures how far the phase moved across two sample periods, 50 ns. This is the phase-difference discriminator from the FM part, with f_s at 20 MHz.

Instead of subtracting, it uses a second table, indexed by the previous and current phase, that returns the output code directly. There are only 32 × 32 = 1024 pairs, so the rest of the work can be computed ahead of time and stored in the table: wrapping the difference, scaling it to a frequency, and turning that into a video level. The next three sections take these one at a time.

Wrapping the difference

The phases go around a circle, so a sample moving from phase 30 to phase 1 has turned 3 steps forward, not 29 back. As with the wrap in the FM part, the table treats every difference as the shortest way around, a step between −16 and +15.

That sets the same speed limit: less than half a turn in 50 ns, or ±10 MHz. A step of +20 looks exactly like a step of −12, so the table can't tell them apart.

Steps to frequency

Each step is 1/32 of a turn in 50 ns. From the discriminator formula that's f_s/32, 625 kHz, so the step count is the signal's frequency relative to the tuned frequency, in units of 625 kHz.

Brightness is carried as frequency, so the table multiplies the step by a gain: each step adds about six to the output code. With the receiver tuned to the blanking level, sync sits about three steps below it, about 1.9 MHz, and white about seven above, about 4.4 MHz.

Offset and clamp

A step of zero means the signal is at the tuned frequency, which is blanking, so the table adds an offset that puts zero steps at code 20. That's a couple of codes above the 18 calculated from the output levels.

Codes stop at 0 and 63, so steps past sync clamp to 0 and steps past white clamp to 63. A little further out, 9 to 11 steps either way, the table eases back toward blanking.

A step of 12 or more either way is too far to trust, since it could be the wrap from the other direction, so the table sends it to blanking. Noise near the center of the I/Q plane makes large random steps, and sending those to sync instead would make false sync pulses that tear the picture.

The figure shows one row of the table. Set the previous and current phase to see the wrapped step between them, and where that step lands among the output codes.

Both tables at work

The ESP32-C5's BitScrambler, a small programmable unit that rewrites a DMA stream as it passes, does both lookups for every sample without the CPU.

The figure feeds a line of grey staircase video through the real tables from C5VRX. Lower the amplitude or add noise to see the phase jitter between neighboring cells, and the output code jitter with it.

A resistor ladder

Six GPIO pins carry the six bits of each code, and each pin drives the video output through its own resistor, from 8.2 kΩ for the lowest bit to 240 Ω for the highest. Each resistor is about half the one before, so each bit adds about twice the current of the bit below it, and the voltage at the output follows the code.

The voltage comes from three sums. First, a pin that is high pushes a current into the output, 3.3\,\text{V} \div R, as it would if the output were held at ground. A pin that is low pushes nothing. Second, every resistor, high or low, also connects the output to ground, so the output sees all of them in parallel, along with a 200 Ω resistor and, when connected, the TV's 75 Ω input. Third, the voltage is the total current times that parallel resistance.

Move the code and watch the sums change. A high pin lights up green and its current joins the total, and a low pin's current drops out while its resistor stays in the parallel group. With all six bits high the output is about 1 V, the full video range. Disconnect the TV and the parallel resistance rises, so the voltage roughly doubles.

The resistors are standard values, so they're only close to doubling. Each code lands up to about 8 mV off a straight line, half the 16 mV between codes.

Seven bits, seven bits and a clock

The C5 puts the top seven bits of I and the top seven of Q on 14 wires, with a fifteenth carrying a sample clock. The clock comes from the same PLL as the modem's, so the two chips cannot drift apart. The P4 captures the wires with its own PARLIO on that clock, one 16-bit word per sample, I in the low byte and Q in the high byte.

The P4's PARLIO takes 16 data lines, which could hold 8 bits of each, and the modem's samples have 8 usable bits, so the 7 comes from the wires between the two boards. Each lane is one wire, and 8 bits of I and Q plus the clock would need 17. The two boards face each other across a gap of about 8 mm, and only so many pads line up on that edge. Four of them are already control wires: the C5's serial transmit and receive, its boot pin and its reset. The C5's two USB pins had to become data pads as well, which gives up its USB port. A 17th data wire would only fit by dropping the serial link and controlling the C5 over its USB from the P4 instead. The firmware keeps the simpler serial control, which also lets the P4 flash the C5, and takes 7 bits.

The lost bit sets a floor. For a weak signal the error in the companding figure below is limited by these 7 bits, not by the table.

Seven bits gives a 128 by 128 grid instead of 16 by 16. The unused lowest bit of each byte is tied to 1, so a lane that reads v in its seven bits means 2v+1, the middle of its step, and the grid stays centered on zero.

The modem's bus changes 80 million times a second, so each sample is only valid for about 12.5 ns. Whether the clock's edge lands on a transition is settled when the C5 resets. When the decoder starts, the P4 captures a block on each edge of the clock and keeps the edge whose samples jump around less, and if neither edge reads clean it has the C5 slip its clock a tick and tries again.

The C5's PHY is set to its 802.11p mode, which narrows the receive filter to about 12 MHz and leaves two clock positions in three that read the lanes steady, against one in three without it.

Drag the amplitude down to see how a weak signal lands on each receiver's grid.

A slower sample rate

The C5 keeps every sixth sample of its 80 MS/s bus, 13.33 million a second, where its own tables use 40 million. The speed limit from the FM part is half the sample rate, here ±6.67 MHz. The picture signal has to stay inside it, since a frequency past the limit wraps to the other end.

At 360 MHz the slower rate gives a core 27 clock cycles per sample.

Phase from two small tables

The P4 also has a BitScrambler, and it does the phase lookup there, as the C5 does, but with a table of 1024 entries rather than 256. A lookup indexed by 14 bits would be far too big, so it works in two steps.

First each lane's 7 bits become a 5-bit magnitude code with a sign. The 32 codes are spaced logarithmically, a trick called companding that the next section explains. A flag marks a lane at full scale.

Second, the two magnitude codes address a table of the angle of that point in the first quadrant, 64 steps to a quarter turn. The signs say which quadrant, and in the odd quadrants the two codes swap places so the angle always counts on from the start of the quadrant. The result is a phase byte, 256 steps to a turn, and the BitScrambler needs four cycles for each sample to do it.

It writes a second byte for each sample as well: I's magnitude code and the clipped flags, which the gain control reads in a later section.

Companding

Companding is compressing a value on the way in and expanding it on the way out. Telephone systems use it to fit speech into 8 bits. Here a lane's 7 bits are compressed to a 5-bit code, and the angle table then reads each code as the middle of its range, the expanded value.

The compression is there because the angle table has to be small. Two lanes of 7 bits would index 128 × 128 = 16384 entries, while two lanes of 5-bit codes index 32 × 32 = 1024. The question is where to put the 32 steps in each lane.

A step of size \Delta moves a point on a phasor of length r by an angle of about \Delta / r. So evenly spaced steps are fine for strong samples and coarse for weak ones: a step of 4 units is about a 2° error at r = 100 but about 40° at r = 6. What matters is the step size relative to the sample, which is what log spacing keeps constant.

The P4's codes follow \text{code} = \lfloor s \log_2(1 + |v|/k) \rfloor, where k is the knee, 6 units, and s sets the 32 codes to span 160. Well below the knee the steps are about even and small, under one unit wide. Well above it each step is a fixed share of the value. A pure logarithm would need infinitely small steps at zero, and the knee stops that. Move the knee in the first figure to see the steps change.

The second figure measures the result. It runs the same phasor through the whole lookup, including the rounding of the angle to 256 steps, and plots the RMS angle error against the signal's amplitude. At 5% of full scale the evenly spaced codes give 10.6° of error and the log-spaced codes 5.0°, which is what all 7 bits of each lane would give. At 2% it is 26° against 11°. The trade goes the other way for strong signals: at half of full scale the evenly spaced codes do a little better, 1.1° against 1.4°, because the log-spaced steps are wider there. Companding spends its precision on weak signals, which are the ones that fade into noise.

Differences and wraps

One step of the phase byte is 1/256 of a turn in 75 ns, or f_s/256, about 52 kHz, against 625 kHz on the C5. The BitScrambler can't subtract one sample's phase from the previous one's, so a CPU core does it, 16 samples at a time with the P4's vector instructions. An int8 difference wraps on its own, which is the ±180° wrap from the FM part for free. The same code also sums each four differences, which the sync detector uses.

Noise wraps too. A sample that lands half a turn or more from the mean of its neighbors has wrapped past the range, and is moved to the end of the range it wrapped past, so a noise spike no longer turns white to black or sync to white.

Centering the table

The I and Q samples carry a DC offset that grows with the radio's gain, and the phase table is only right if the circle is centered on the middle of the table. On an empty channel the offset read 71, 50, 40, 24 and 20 units at gains 77, 74, 71, 68 and 65. The table is built around an offset, and a table that is 3 units out triples its error.

The P4 finds the offset from signs. A carrier circling its center spends half its time on each side, so the share of negative samples says how far the table's center sits from the circle's: the offset is r \sin(\pi (n - \tfrac12)) for a negative share n, and the radius r follows from the power about the table's center. The BitScrambler's bytes carry no raw I and Q, so the power comes from I's magnitude code alone and the signs from the phase byte.

Every 50 ms the control task reads this, smooths it over about four seconds, and rebuilds the table when the offset is two units out. It remembers the offset found at each gain, so a gain step starts from the offset it had before, or from the baseband gain's share of the step for a gain it has not seen.

Rewriting the table means stopping the BitScrambler for about 17 lines, so the rewrite waits for the last picture lines of a field and the next vertical sync sets the line count right again.

Setting the gain

The P4 sets the same gain as the C5 firmware does, one entry of the PHY's gain table. It sends gain N over the serial link to the C5, whose firmware turns off the packet AGC and forces that entry. The P4 never reads the radio's signal strength, and it has no fine, baseband and RF choices of its own. It moves one number, about a decibel a step, from 30 up to 77, the top of the C5's table. Above 77 the indices clip on noise alone.

Its measurement is the radius from the previous section, in the lanes' 2v+1 units with full scale 127. From 32 samples in each DMA node it totals the power and counts the rail. Nodes where more than a quarter of the samples clip are left out, because another transmitter's burst, such as a Wi-Fi station, clips a few nodes whole and says nothing about this carrier's level. If most nodes clip, that is the carrier itself, and the loop treats it as full clipping. A pass needs 4096 samples to count.

The loop aims for an RMS of 54, between 40 and 72, which keeps the table's rounding small: about 20 adds a third to the picture's noise and about 60 adds almost nothing. When the RMS is out of range the step is 20 \log_{10}(54 / \text{RMS}) steps of gain, at most 10 either way, and at least 2 down when it is too high. More than 0.5% of samples clipping steps down by 2, and more than 10% steps down by 8. When the table's center is outside the circle the RMS reads too high, so then it may raise the gain but not lower it.

It does not chase small errors. Each gain step costs a table reload, which costs lines, so a level a few decibels off has to stay off for 6 passes, about 0.3 s, before the gain moves. A level far off does not wait: an RMS under 27 or over 100, or heavy clipping, moves it at once. After any step the next pass is thrown away, since its samples were taken at the old gain, and a step of 10 or more means a signal arrived or left, so the offset is found afresh.

When the decoder starts, a probe captures raw samples and steps the gain by 6 until the circle is in range, a clip share over 2% or an RMS over 80 down and an RMS under 30 up, from 22 to 70 in at most 8 passes. It measures the offset from that capture too, so the first table starts close.

Finding lines and fields

The demodulated signal has no fixed zero, because a tuning error shifts all of it. The P4 uses the reference levels the video signal provides: it averages the sync tips and back porches of the lines it has locked, and scales everything else from those two levels. Until lines lock it takes them from a histogram of the field, and tries a few fixed sync depths in turn.

The sync detector sums four samples at a time, then the last 16 of those sums, about the width of an hsync, and slices that against a level halfway between sync tip and blanking with a little hysteresis. A pulse between 3.4 and 6.4 µs wide is an hsync, and one over 18 µs is a broad pulse of the vertical interval. Bright picture is clamped so it can't hide a pulse's start.

A line clock predicts where each horizontal sync should fall, 847.4 samples after the last. A pulse within 3 µs of the prediction moves the clock by an eighth of the error and its period by 1/256 of it, within 1% of nominal. Anything else is ignored, so the clock coasts through noise. After 30 lines without a hit it takes the next pulse of the right width as the new start.

The broad pulses of vertical sync reset the line count, and where they fall within the line tells the two fields apart. Dark parts of a picture can dip low enough for long enough to look like vertical sync, so a vertical sync only counts within six lines of where the count expects it, unless three fields have passed without one there.

Try losing most of the pulses and adding false ones, which the figure draws as hsync-width pulses. The clock keeps the line start steady where a receiver that restarts on the first pulse it finds wanders.

With no sync the line and field counts coast at their last rate, 262.5 lines a field, and every field is still decoded, so a weak signal shows as static and relocks without a flash, as it does on an analog monitor.

Decoding color

At 13.33 million samples a second the color subcarrier is exactly 189/704 of the sample rate. A table of 704 sine and cosine values therefore repeats forever without drifting, and it serves as the receiver's reference oscillator.

The P4 measures the burst's vector against that table on every line, and averages it over lines. Then it multiplies the chroma by the sine and cosine and sums the products in blocks of 8 samples, which is the multiply and low-pass from the hue section, about 0.6 µs a block.

Sharp brightness edges leak into the chroma band and show as colored fringes. The subcarrier has 227.5 cycles per line, so it turns half a cycle from one line to the next. Adding the chroma of two neighboring lines cancels the leak while the color adds up.

FM noise rises with frequency, which makes chroma the noisiest part of the picture, so each block is also smoothed with its neighbors in a 1 2 1 weighting. A line with an FM click, a jump of most of a cycle between two samples, takes the color of the lines above it, because the click rings in the chroma band as a dash of saturated color.

Color switches on only when a field has enough locked hsyncs and a strong enough burst, and stays on until they fall further. Otherwise the P4 draws black and white rather than colored noise.

Fields to frames

Each field carries 240 picture lines, every other row of a 720 by 480 frame. The luma of each line is a weighted sum of seven samples, wider on lines that start half a sample late, and one pixel is about one sample.

The P4 sends a full frame for every field. Each missing row takes the median of three values for its luma: the row above, the row below, and the same row from the previous field. Its chroma comes from the row below. Where the picture is still, the median usually picks the previous field's row and keeps full detail. Where it moves, it falls back to a neighboring line.

Rows of the previous field and the finished strips of the frame move between memory and the PSRAM frame by DMA, so the cores spend their time on the pixels. Each row is also pulled toward the same row two fields back, by less as the difference grows and not at all past a threshold that follows the measured noise, so noise settles without smearing motion.

The hardware JPEG encoder turns each frame into an MJPEG image, lowering its quality when a noisy frame would not fit, and the P4 sends it over high-speed USB as a standard webcam, with a serial console on the same cable.

What runs where

A sample passes through hardware first, then two CPU tasks on different cores, then hardware again. The demodulator and sync detector get a core to themselves, because at 13.33 million samples a second a 360 MHz core has 27 clock cycles for each one. The numbers follow one sample's path, and the lettered box is a control loop that runs beside it.

The control loop is the one place the two sides touch. The control task reads statistics that the decode task collects from the second byte of each sample, sends a new gain to the C5, and builds a new table. The decode task is the one that loads it, stopping the receive for about 17 lines in the last picture lines of a field.

A line crosses from core 1 to core 0 through a queue of 40 lines. The ring of demodulated samples holds about 38 lines, so a line that waits longer would be drawn from samples already overwritten, and the draw task drops it.

The same job, step by step

Both receivers turn the radio's I and Q into video, but they spend their resources differently. The C5 has fast peripherals and no CPU time to spare, so it uses small tables and stays analog. The P4 has CPU time and memory, so it uses finer numbers and decodes the video itself.