Decode data signals, starting with on-off keying

Much of what a scanner finds is not speech.  Doorbells, tyre-pressure
sensors, weather stations, remote controls, paging and packet radio all
carry something a receiver can read, and until now the answer was "OOK /
ASK data burst" and a WAV file.  Now the bits come out.

The observation the whole thing is built on is that whatever the
modulation, a data signal is the same shape once it has been sliced: a
train of alternating runs whose lengths carry the information.  On-off
keying gives that directly -- the carrier is up or it is down -- and
two-level FSK gives exactly the same thing from the discriminator, one
tone or the other.  So both reduce to a run-length train and everything
after that is shared.

What the runs mean is the line code, and it is worked out from the runs
alone rather than configured, because each code makes a different
prediction about which of the two histograms is the bimodal one: PWM
(EV1527, PT2262, and nearly every 433 MHz remote), PPM, Manchester, and
plain NRZ.  Four-level FSK is recognised as such and read as symbols
rather than sliced down the middle, which produces bits that mean
nothing; where a frame sync word appears the system is named outright.

Two protocols carry their own framing and checksums and so are read in
full.  POCSAG paging: all three rates tried because nothing in the signal
says which it is, every codeword checked and single-bit errors corrected
against the BCH code, and the address, function letter and message text
reported.  AX.25 as APRS uses it: the frame check has to come out right
before a frame is reported at all, and the sender's callsign goes onto
the map with everyone else's.

The hard half is refusing what is not data.  Noise sliced at a threshold
produces runs and runs produce bits, so three things guard against it:
the runs have to quantise to the line code's own grid; most of the bursts
in a capture have to decode the same way, because one lucky window in
eight is a coincidence and that is exactly what SSB voice produced; and,
much the strongest, the packet has to repeat, because bits that come back
identical six times did not come from noise.  A reading with none of that
behind it is reported as nothing at all rather than as a bit string with
a low number beside it that somebody will read anyway.  Across 27
recordings of speech, music, static, a bare carrier, Morse and PSK it
returns nothing 27 times.

A firm decode also outranks the content check, which is statistical: a
burst of keying demodulated as FM audio is a buzz and the speech detector
likes a buzz, but a frame whose own checksum came out right is not a
statistic.  Such a capture is kept and filed as data, not as voice.

What comes out is written to a _data.txt beside the recording, shown on
the live display and in the line-per-hit output, and takes the place of
the transcript at the top of saunterbrowse -- where it is searchable, so
"which page mentioned engine 4" is a question that can be asked.
`bandsaunter analyze` decodes a file you already have.

The simulator gained two honest transmitters to test against: a
pulse-width remote that repeats a real payload, and a pager that sends
real POCSAG batches with real BCH check bits.  Random keying exercises
the classifier but leaves a decoder nothing to get right.  The POCSAG
encoder lives next to the decoder rather than in the test helpers, so a
bug shared by both cannot hide.

Fixed along the way:

- Rich reads a square bracket as markup, and a decoded page is arbitrary
  text off the air.  "[/x]" in a message ended the live display with a
  MarkupError; so did typing "[/" at saunterbrowse's search prompt.
  Everything that did not come from this program is escaped now.

- Otsu returned the first bin of a plateau.  Two populations with nothing
  between them -- silence and full carrier, which is what on-off keying
  is -- make every threshold in the gap equally good, and taking the
  first put it hard against the lower population with the hysteresis band
  outside the data entirely, so nothing sliced at all.

- Estimating the symbol clock by counting along a cumulative grid is a
  fixed point: a unit two per cent small produces two per cent more
  symbols and reproduces itself exactly.  Rounding each run on its own
  converges instead, because every run votes independently.  The grid is
  then the right way to extract the bits, where rounding runs one at a
  time drifts.

- A clipped first repeat used to truncate every other repeat to its
  length.  The consensus is taken over the commonest length now.

761 -> 869 tests.

README.md

@@ -618,6 +618,94 @@ The band name is written into each recording's sidecar too, so it travels with
 the capture, and `saunterbrowse` will search on it — typing `/70 cm` finds
 everything in the band without having to remember 420–450 MHz.
 
+### Reading data signals
+
+A great deal of what a scanner finds is not speech. Doorbells, tyre-pressure
+sensors, weather stations, remote controls, paging, packet radio — all of it
+carries something a receiver can read, and bandsaunter reads it:
+
+```
+21:14:07        433.92 MHz  70 cm Amateur          3.2s  SNR 45.8 dB  EV1527 / PT2262-style remote (93%)
+                                                   EV1527 / PT2262-style remote  24 bits  516 baud  x12  B2 35 4E
+21:14:31      929.6125 MHz  UHF / 900 MHz Paging   4.5s  SNR 49.5 dB  POCSAG 1200 (97%)
+                                                   [1234568D] ENGINE 4 RESPOND
+                                                   [0098765A] CALL EXT 4412
+21:15:02       144.39 MHz   2 m Amateur            1.8s  SNR 31.2 dB  AX.25 / APRS (93%)
+                                                   W1AW>APRS>WIDE1-1: !4142.45N/07243.63W-Newington CT
+```
+
+**Whatever the modulation, a data signal is the same shape once it has been
+sliced**: a train of alternating runs whose *lengths* carry the information.
+On-off keying gives that directly — the carrier is up or it is down — and
+two-level FSK gives exactly the same thing from the discriminator, one tone or
+the other. So both are reduced to runs, and everything after that is shared.
+
+What the runs mean is the line code, and it is worked out from the runs alone
+rather than configured, because each code makes a different prediction about
+which of the two histograms is the bimodal one:
+
+| Code | Pulses | Gaps | Who uses it |
+|---|---|---|---|
+| **PWM** | two lengths | constant, or the period is | EV1527, PT2262 and nearly every 433 MHz remote |
+| **PPM** | constant | two lengths | the other half of the same market |
+| **Manchester** | T and 2T only | T and 2T only | anything whose receiver recovers its own clock |
+| **NRZ** | any whole number of symbols | same | what a framed protocol sits on |
+
+Four-level FSK — C4FM, as P25, DMR and NXDN send it — is recognised as such and
+read as symbols. Slicing it down the middle also produces bits, and they mean
+nothing; a capture that had been coming back as "10783 bits of NRZ at 5335
+baud" now says *4-level FSK, 5334 baud, no frame sync recognised*, which is
+both true and useful. Where a frame sync word does appear, the system is named
+outright.
+
+### Protocols that can be read in full
+
+Two carry their own framing and checksums, so a frame either passes or it does
+not — and one that passes is not a guess:
+
+**POCSAG** paging, at 512, 1200 or 2400 baud. Nothing in the signal announces
+which rate it is, so all three are tried and the one whose 32-bit sync word
+turns up is the right one. Every codeword is checked — and a single bit error
+corrected — against the BCH code the standard puts there for exactly that. The
+address, function letter and message text all come out.
+
+**AX.25 / APRS** on 1200 baud AFSK. The frame check has to come out right
+before a frame is reported at all. The sender's callsign, the digipeater path
+and the payload are shown — and the callsign goes onto the map with everyone
+else.
+
+```bash
+bandsaunter analyze capture.cf32 --rate 48000   # decode a file you already have
+bandsaunter scan --no-decode-data               # turn it off
+saunterbrowse                                   # decoded packets sit where a transcript would
+```
+
+### Believing a decode
+
+This is the hard half. A decoder that always returns *something* is worse than
+useless: noise sliced at a threshold produces runs, and runs produce bits.
+Three things guard against that.
+
+- **The runs have to fit.** A decode whose runs do not quantise to the line
+  code's own grid is thrown away.
+- **Most of the capture has to agree.** A data signal is data all the way
+  through. One lucky window in eight is a coincidence — and that is exactly
+  what SSB voice produced before this check existed.
+- **The packet has to repeat.** Much the strongest of the three. These
+  transmitters send the same thing three to ten times over, and bits that come
+  back identical every time did not come from noise.
+
+A bare reading with none of that behind it — where the run lengths merely
+happened to land on a grid — is reported as **nothing at all**, rather than as
+a bit string with a low number beside it that somebody will read anyway. Across
+27 recordings of speech, music, static, a bare carrier, Morse and PSK, the
+decoder returns nothing 27 times.
+
+And a decode that *does* have repeats or a checksum behind it outranks the
+content check. A burst of keying demodulated as FM audio is a buzz, and the
+speech detector likes a buzz — but a frame whose own checksum came out right is
+not a statistic.
+
 ### Trunked systems and their control channels
 
 Police, fire and most large business radio in the US runs on *trunked*