Decode data signals, starting with on-off keying
Much of what a scanner finds is not speech. Doorbells, tyre-pressure sensors, weather stations, remote controls, paging and packet radio all carry something a receiver can read, and until now the answer was "OOK / ASK data burst" and a WAV file. Now the bits come out. The observation the whole thing is built on is that whatever the modulation, a data signal is the same shape once it has been sliced: a train of alternating runs whose lengths carry the information. On-off keying gives that directly -- the carrier is up or it is down -- and two-level FSK gives exactly the same thing from the discriminator, one tone or the other. So both reduce to a run-length train and everything after that is shared. What the runs mean is the line code, and it is worked out from the runs alone rather than configured, because each code makes a different prediction about which of the two histograms is the bimodal one: PWM (EV1527, PT2262, and nearly every 433 MHz remote), PPM, Manchester, and plain NRZ. Four-level FSK is recognised as such and read as symbols rather than sliced down the middle, which produces bits that mean nothing; where a frame sync word appears the system is named outright. Two protocols carry their own framing and checksums and so are read in full. POCSAG paging: all three rates tried because nothing in the signal says which it is, every codeword checked and single-bit errors corrected against the BCH code, and the address, function letter and message text reported. AX.25 as APRS uses it: the frame check has to come out right before a frame is reported at all, and the sender's callsign goes onto the map with everyone else's. The hard half is refusing what is not data. Noise sliced at a threshold produces runs and runs produce bits, so three things guard against it: the runs have to quantise to the line code's own grid; most of the bursts in a capture have to decode the same way, because one lucky window in eight is a coincidence and that is exactly what SSB voice produced; and, much the strongest, the packet has to repeat, because bits that come back identical six times did not come from noise. A reading with none of that behind it is reported as nothing at all rather than as a bit string with a low number beside it that somebody will read anyway. Across 27 recordings of speech, music, static, a bare carrier, Morse and PSK it returns nothing 27 times. A firm decode also outranks the content check, which is statistical: a burst of keying demodulated as FM audio is a buzz and the speech detector likes a buzz, but a frame whose own checksum came out right is not a statistic. Such a capture is kept and filed as data, not as voice. What comes out is written to a _data.txt beside the recording, shown on the live display and in the line-per-hit output, and takes the place of the transcript at the top of saunterbrowse -- where it is searchable, so "which page mentioned engine 4" is a question that can be asked. `bandsaunter analyze` decodes a file you already have. The simulator gained two honest transmitters to test against: a pulse-width remote that repeats a real payload, and a pager that sends real POCSAG batches with real BCH check bits. Random keying exercises the classifier but leaves a decoder nothing to get right. The POCSAG encoder lives next to the decoder rather than in the test helpers, so a bug shared by both cannot hide. Fixed along the way: - Rich reads a square bracket as markup, and a decoded page is arbitrary text off the air. "[/x]" in a message ended the live display with a MarkupError; so did typing "[/" at saunterbrowse's search prompt. Everything that did not come from this program is escaped now. - Otsu returned the first bin of a plateau. Two populations with nothing between them -- silence and full carrier, which is what on-off keying is -- make every threshold in the gap equally good, and taking the first put it hard against the lower population with the hysteresis band outside the data entirely, so nothing sliced at all. - Estimating the symbol clock by counting along a cumulative grid is a fixed point: a unit two per cent small produces two per cent more symbols and reproduces itself exactly. Rounding each run on its own converges instead, because every run votes independently. The grid is then the right way to extract the bits, where rounding runs one at a time drifts. - A clipped first repeat used to truncate every other repeat to its length. The consensus is taken over the commonest length now. 761 -> 869 tests.
This commit is contained in:
The Dust Council
parent
fb2bb3344b
commit
68b05a031c
19 changed files with 3176 additions and 23 deletions
|
|
@ -1,5 +1,5 @@
|
|||
.\" Generated by packaging/make-man.py -- do not edit by hand.
|
||||
.TH BANDSAUNTER 1 "2026-08-28" "bandsaunter 2026-08-28_01" "User Commands"
|
||||
.TH BANDSAUNTER 1 "2026-08-28" "bandsaunter 2026-08-28_02" "User Commands"
|
||||
.SH NAME
|
||||
bandsaunter \- scan, record and identify radio signals with an RTL-SDR
|
||||
.SH SYNOPSIS
|
||||
|
|
@ -543,6 +543,15 @@ Setting name \fBdecode_morse\fR, default \fByes\fR.
|
|||
Turn keyed carriers into readable text, with the sending speed. Morse is still in daily use by amateurs and by beacons, and this saves you learning to read it by ear. It costs almost nothing when there is no Morse about.
|
||||
.RE
|
||||
.TP
|
||||
.B --decode-data / --no-decode-data
|
||||
Decode data signals \[em] read the packets out of anything carrying data.
|
||||
.br
|
||||
Setting name \fBdecode_data\fR, default \fByes\fR.
|
||||
.RS
|
||||
.PP
|
||||
Read what a data signal actually says. A great deal of what a scanner finds is not speech: doorbells, tyre-pressure sensors, weather stations, remote controls, paging, packet radio. Each one is sliced into its pulses, the line code worked out from the pulse lengths alone, and the bits reported -- with the packet named where its framing says what it is, and the message printed in full where the protocol carries one. The check that keeps it honest is repetition: these transmitters send the same packet several times over, and bits that come back identical every time did not come from noise. Turn it off to save a little processing on a busy band.
|
||||
.RE
|
||||
.TP
|
||||
.B --log-file
|
||||
Log file \[em] name of the run log inside the output directory.
|
||||
.br
|
||||
|
|
@ -934,6 +943,78 @@ than a directory of placeholders.
|
|||
.BR saunterbrowse (1)
|
||||
reads these back, and lists any callsigns it finds in them with the licence
|
||||
they belong to.
|
||||
.SH DECODING DATA
|
||||
A great deal of what a scanner finds is not speech. Doorbells, tyre\-pressure
|
||||
sensors, weather stations, remote controls, paging and packet radio all carry
|
||||
words or numbers that a receiver can read, and
|
||||
.B bandsaunter
|
||||
reads them.
|
||||
.PP
|
||||
Whatever the modulation, a data signal comes down to the same shape once it
|
||||
has been sliced: a train of alternating runs whose lengths carry the
|
||||
information. On\-off keying gives that directly \[em] the carrier is up or it is
|
||||
down \[em] and two\-level FSK gives the same thing from the discriminator, one
|
||||
tone or the other. So both are reduced to runs and everything after that is
|
||||
shared.
|
||||
.PP
|
||||
What the runs mean is the line code, and it is worked out from the runs alone
|
||||
rather than being configured:
|
||||
.TP
|
||||
.B PWM
|
||||
The pulse carries the bit and the gap or the period holds still. Nearly every
|
||||
cheap 433 MHz remote, and everything built on an EV1527 or PT2262.
|
||||
.TP
|
||||
.B PPM
|
||||
The pulse holds still and the gap carries the bit. The other half of the same
|
||||
market.
|
||||
.TP
|
||||
.B Manchester
|
||||
Every bit is a transition in the middle of its own period, so runs come in
|
||||
only two lengths.
|
||||
.TP
|
||||
.B NRZ
|
||||
The level is held for as many symbol periods as there are bits. What a framed
|
||||
protocol sits on top of.
|
||||
.PP
|
||||
Four\-level FSK \[em] C4FM, as P25, DMR and NXDN use it \[em] is recognised as
|
||||
such and read as symbols rather than being sliced down the middle, which would
|
||||
give bits that mean nothing. Where a frame sync word appears the system is
|
||||
named outright.
|
||||
.SH PROTOCOLS THAT CAN BE READ IN FULL
|
||||
Two carry their own framing and checksums, so a frame either passes or it does
|
||||
not, and one that passes is not a guess.
|
||||
.TP
|
||||
.B POCSAG
|
||||
Paging, at 512, 1200 or 2400 baud. The rate is not announced anywhere in the
|
||||
signal, so all three are tried and the one whose sync word appears is the
|
||||
right one. Each codeword is checked, and a single bit error is corrected,
|
||||
against the BCH code the standard puts there for the purpose. The address, the
|
||||
function letter and the message text are all reported.
|
||||
.TP
|
||||
.B "AX.25 / APRS"
|
||||
Amateur packet on 1200 baud AFSK. The frame check has to come out right before
|
||||
a frame is reported at all. The sender's callsign, the digipeater path and the
|
||||
payload are shown \[em] and the callsign goes onto the map with the rest.
|
||||
.SH BELIEVING A DECODE
|
||||
A decoder that always returns something is worse than useless: noise sliced at
|
||||
a threshold produces runs, and runs produce bits. Three things guard against
|
||||
that.
|
||||
.PP
|
||||
The runs have to quantise to the line code's own grid, and a decode whose runs
|
||||
are scattered is thrown away. Most of the bursts in a capture have to decode
|
||||
the same way, because a data signal is data all the way through and one lucky
|
||||
window among eight is a coincidence. And, much the strongest, the packet has
|
||||
to repeat \[em] these transmitters send the same thing three to ten times over,
|
||||
and bits that come back identical every time did not come from noise.
|
||||
.PP
|
||||
A bare reading with none of that behind it, where the runs merely happened to
|
||||
land on a grid, is reported as nothing at all rather than as a bit string with
|
||||
a low number beside it that somebody will read anyway.
|
||||
.PP
|
||||
A decode that does have repeats or a checksum behind it outranks the content
|
||||
check: a burst of keying demodulated as FM audio is a buzz, and the speech
|
||||
detector likes a buzz, but a frame whose own checksum came out right is not a
|
||||
statistic.
|
||||
.SH THE MAP
|
||||
A callsign heard in a transcript is looked up in the FCC's published licence
|
||||
data, which gives the licensee, the town, and coordinates. Those go into a
|
||||
|
|
@ -995,6 +1076,9 @@ Where recordings, transcripts and logs are written, unless
|
|||
.B \-\-output
|
||||
says otherwise. Chosen on first run.
|
||||
.TP
|
||||
.IR ... _data.txt
|
||||
What a data capture said, where anything was decoded.
|
||||
.TP
|
||||
.I ~/bandsaunter/callsigns.kml
|
||||
The map of stations heard, added to as scans run.
|
||||
.TP
|
||||
|
|
|
|||
|
|
@ -225,6 +225,21 @@ KML is the format Google Earth uses.
|
|||
.BR marble (1)
|
||||
and OsmAnd open it too, and it is XML, so a scan interrupted halfway through
|
||||
leaves a file that still opens.
|
||||
.SH DECODED DATA
|
||||
Where a capture carried data rather than speech, what was decoded takes the
|
||||
place of the transcript at the top of the screen: the kind of packet, and then
|
||||
the message. A pager's text, an APRS position report, or the bits and hex of a
|
||||
remote control. It is searchable with
|
||||
.B /
|
||||
like anything else, so "which page mentioned engine 4" is a question that can
|
||||
be asked here.
|
||||
.PP
|
||||
The text comes from the
|
||||
.I _data.txt
|
||||
beside the recording, or from the sidecar where there is none.
|
||||
.BR bandsaunter (1)
|
||||
describes how it is decoded and what has to be true before a decode is
|
||||
believed.
|
||||
.SH TRANSCRIPTS
|
||||
A transcript appears only where a recogniser produced one, which means the
|
||||
capture was judged to be speech and
|
||||
|
|
@ -252,6 +267,9 @@ Its measurements and identification.
|
|||
.TP
|
||||
.IR ... _transcription.txt
|
||||
What was said, where a recogniser heard speech.
|
||||
.TP
|
||||
.IR ... _data.txt
|
||||
What was decoded, where the capture carried data.
|
||||
.SH ENVIRONMENT
|
||||
.TP
|
||||
.B BANDSAUNTER_OUTPUT
|
||||
|
|
|
|||
|
|
@ -365,6 +365,78 @@ than a directory of placeholders.
|
|||
.BR saunterbrowse (1)
|
||||
reads these back, and lists any callsigns it finds in them with the licence
|
||||
they belong to.
|
||||
.SH DECODING DATA
|
||||
A great deal of what a scanner finds is not speech. Doorbells, tyre\-pressure
|
||||
sensors, weather stations, remote controls, paging and packet radio all carry
|
||||
words or numbers that a receiver can read, and
|
||||
.B bandsaunter
|
||||
reads them.
|
||||
.PP
|
||||
Whatever the modulation, a data signal comes down to the same shape once it
|
||||
has been sliced: a train of alternating runs whose lengths carry the
|
||||
information. On\-off keying gives that directly \[em] the carrier is up or it is
|
||||
down \[em] and two\-level FSK gives the same thing from the discriminator, one
|
||||
tone or the other. So both are reduced to runs and everything after that is
|
||||
shared.
|
||||
.PP
|
||||
What the runs mean is the line code, and it is worked out from the runs alone
|
||||
rather than being configured:
|
||||
.TP
|
||||
.B PWM
|
||||
The pulse carries the bit and the gap or the period holds still. Nearly every
|
||||
cheap 433 MHz remote, and everything built on an EV1527 or PT2262.
|
||||
.TP
|
||||
.B PPM
|
||||
The pulse holds still and the gap carries the bit. The other half of the same
|
||||
market.
|
||||
.TP
|
||||
.B Manchester
|
||||
Every bit is a transition in the middle of its own period, so runs come in
|
||||
only two lengths.
|
||||
.TP
|
||||
.B NRZ
|
||||
The level is held for as many symbol periods as there are bits. What a framed
|
||||
protocol sits on top of.
|
||||
.PP
|
||||
Four\-level FSK \[em] C4FM, as P25, DMR and NXDN use it \[em] is recognised as
|
||||
such and read as symbols rather than being sliced down the middle, which would
|
||||
give bits that mean nothing. Where a frame sync word appears the system is
|
||||
named outright.
|
||||
.SH PROTOCOLS THAT CAN BE READ IN FULL
|
||||
Two carry their own framing and checksums, so a frame either passes or it does
|
||||
not, and one that passes is not a guess.
|
||||
.TP
|
||||
.B POCSAG
|
||||
Paging, at 512, 1200 or 2400 baud. The rate is not announced anywhere in the
|
||||
signal, so all three are tried and the one whose sync word appears is the
|
||||
right one. Each codeword is checked, and a single bit error is corrected,
|
||||
against the BCH code the standard puts there for the purpose. The address, the
|
||||
function letter and the message text are all reported.
|
||||
.TP
|
||||
.B "AX.25 / APRS"
|
||||
Amateur packet on 1200 baud AFSK. The frame check has to come out right before
|
||||
a frame is reported at all. The sender's callsign, the digipeater path and the
|
||||
payload are shown \[em] and the callsign goes onto the map with the rest.
|
||||
.SH BELIEVING A DECODE
|
||||
A decoder that always returns something is worse than useless: noise sliced at
|
||||
a threshold produces runs, and runs produce bits. Three things guard against
|
||||
that.
|
||||
.PP
|
||||
The runs have to quantise to the line code's own grid, and a decode whose runs
|
||||
are scattered is thrown away. Most of the bursts in a capture have to decode
|
||||
the same way, because a data signal is data all the way through and one lucky
|
||||
window among eight is a coincidence. And, much the strongest, the packet has
|
||||
to repeat \[em] these transmitters send the same thing three to ten times over,
|
||||
and bits that come back identical every time did not come from noise.
|
||||
.PP
|
||||
A bare reading with none of that behind it, where the runs merely happened to
|
||||
land on a grid, is reported as nothing at all rather than as a bit string with
|
||||
a low number beside it that somebody will read anyway.
|
||||
.PP
|
||||
A decode that does have repeats or a checksum behind it outranks the content
|
||||
check: a burst of keying demodulated as FM audio is a buzz, and the speech
|
||||
detector likes a buzz, but a frame whose own checksum came out right is not a
|
||||
statistic.
|
||||
.SH THE MAP
|
||||
A callsign heard in a transcript is looked up in the FCC's published licence
|
||||
data, which gives the licensee, the town, and coordinates. Those go into a
|
||||
|
|
@ -426,6 +498,9 @@ Where recordings, transcripts and logs are written, unless
|
|||
.B \-\-output
|
||||
says otherwise. Chosen on first run.
|
||||
.TP
|
||||
.IR ... _data.txt
|
||||
What a data capture said, where anything was decoded.
|
||||
.TP
|
||||
.I ~/bandsaunter/callsigns.kml
|
||||
The map of stations heard, added to as scans run.
|
||||
.TP
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
.\" Generated by packaging/make-browse-man.py -- do not edit by hand.
|
||||
.TH SAUNTERBROWSE 1 "2026-08-28" "bandsaunter 2026-08-28_01" "User Commands"
|
||||
.TH SAUNTERBROWSE 1 "2026-08-28" "bandsaunter 2026-08-28_02" "User Commands"
|
||||
.SH NAME
|
||||
saunterbrowse \- read and listen to what a bandsaunter scan collected
|
||||
.SH SYNOPSIS
|
||||
|
|
@ -207,6 +207,21 @@ KML is the format Google Earth uses.
|
|||
.BR marble (1)
|
||||
and OsmAnd open it too, and it is XML, so a scan interrupted halfway through
|
||||
leaves a file that still opens.
|
||||
.SH DECODED DATA
|
||||
Where a capture carried data rather than speech, what was decoded takes the
|
||||
place of the transcript at the top of the screen: the kind of packet, and then
|
||||
the message. A pager's text, an APRS position report, or the bits and hex of a
|
||||
remote control. It is searchable with
|
||||
.B /
|
||||
like anything else, so "which page mentioned engine 4" is a question that can
|
||||
be asked here.
|
||||
.PP
|
||||
The text comes from the
|
||||
.I _data.txt
|
||||
beside the recording, or from the sidecar where there is none.
|
||||
.BR bandsaunter (1)
|
||||
describes how it is decoded and what has to be true before a decode is
|
||||
believed.
|
||||
.SH TRANSCRIPTS
|
||||
A transcript appears only where a recogniser produced one, which means the
|
||||
capture was judged to be speech and
|
||||
|
|
@ -234,6 +249,9 @@ Its measurements and identification.
|
|||
.TP
|
||||
.IR ... _transcription.txt
|
||||
What was said, where a recogniser heard speech.
|
||||
.TP
|
||||
.IR ... _data.txt
|
||||
What was decoded, where the capture carried data.
|
||||
.SH ENVIRONMENT
|
||||
.TP
|
||||
.B BANDSAUNTER_OUTPUT
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue