dustcouncil/bandsaunter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Decode data signals, starting with on-off keying

Browse source 
Much of what a scanner finds is not speech.  Doorbells, tyre-pressure
sensors, weather stations, remote controls, paging and packet radio all
carry something a receiver can read, and until now the answer was "OOK /
ASK data burst" and a WAV file.  Now the bits come out.

The observation the whole thing is built on is that whatever the
modulation, a data signal is the same shape once it has been sliced: a
train of alternating runs whose lengths carry the information.  On-off
keying gives that directly -- the carrier is up or it is down -- and
two-level FSK gives exactly the same thing from the discriminator, one
tone or the other.  So both reduce to a run-length train and everything
after that is shared.

What the runs mean is the line code, and it is worked out from the runs
alone rather than configured, because each code makes a different
prediction about which of the two histograms is the bimodal one: PWM
(EV1527, PT2262, and nearly every 433 MHz remote), PPM, Manchester, and
plain NRZ.  Four-level FSK is recognised as such and read as symbols
rather than sliced down the middle, which produces bits that mean
nothing; where a frame sync word appears the system is named outright.

Two protocols carry their own framing and checksums and so are read in
full.  POCSAG paging: all three rates tried because nothing in the signal
says which it is, every codeword checked and single-bit errors corrected
against the BCH code, and the address, function letter and message text
reported.  AX.25 as APRS uses it: the frame check has to come out right
before a frame is reported at all, and the sender's callsign goes onto
the map with everyone else's.

The hard half is refusing what is not data.  Noise sliced at a threshold
produces runs and runs produce bits, so three things guard against it:
the runs have to quantise to the line code's own grid; most of the bursts
in a capture have to decode the same way, because one lucky window in
eight is a coincidence and that is exactly what SSB voice produced; and,
much the strongest, the packet has to repeat, because bits that come back
identical six times did not come from noise.  A reading with none of that
behind it is reported as nothing at all rather than as a bit string with
a low number beside it that somebody will read anyway.  Across 27
recordings of speech, music, static, a bare carrier, Morse and PSK it
returns nothing 27 times.

A firm decode also outranks the content check, which is statistical: a
burst of keying demodulated as FM audio is a buzz and the speech detector
likes a buzz, but a frame whose own checksum came out right is not a
statistic.  Such a capture is kept and filed as data, not as voice.

What comes out is written to a _data.txt beside the recording, shown on
the live display and in the line-per-hit output, and takes the place of
the transcript at the top of saunterbrowse -- where it is searchable, so
"which page mentioned engine 4" is a question that can be asked.
`bandsaunter analyze` decodes a file you already have.

The simulator gained two honest transmitters to test against: a
pulse-width remote that repeats a real payload, and a pager that sends
real POCSAG batches with real BCH check bits.  Random keying exercises
the classifier but leaves a decoder nothing to get right.  The POCSAG
encoder lives next to the decoder rather than in the test helpers, so a
bug shared by both cannot hide.

Fixed along the way:

- Rich reads a square bracket as markup, and a decoded page is arbitrary
  text off the air.  "[/x]" in a message ended the live display with a
  MarkupError; so did typing "[/" at saunterbrowse's search prompt.
  Everything that did not come from this program is escaped now.

- Otsu returned the first bin of a plateau.  Two populations with nothing
  between them -- silence and full carrier, which is what on-off keying
  is -- make every threshold in the gap equally good, and taking the
  first put it hard against the lower population with the hysteresis band
  outside the data entirely, so nothing sliced at all.

- Estimating the symbol clock by counting along a cumulative grid is a
  fixed point: a unit two per cent small produces two per cent more
  symbols and reproduces itself exactly.  Rounding each run on its own
  converges instead, because every run votes independently.  The grid is
  then the right way to extract the bits, where rounding runs one at a
  time drifts.

- A clipped first repeat used to truncate every other repeat to its
  length.  The consensus is taken over the commonest length now.

761 -> 869 tests.
This commit is contained in:
The Dust Council 2026-08-28 12:55:37 -07:00
parent fb2bb3344b
commit 68b05a031c
19 changed files with 3176 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

88
README.md
View file

@ -618,6 +618,94 @@ The band name is written into each recording's sidecar too, so it travels with
the capture, and `saunterbrowse` will search on it — typing `/70 cm` finds the capture, and `saunterbrowse` will search on it — typing `/70 cm` finds
everything in the band without having to remember 420–450 MHz. everything in the band without having to remember 420–450 MHz.
### Reading data signals
A great deal of what a scanner finds is not speech. Doorbells, tyre-pressure
sensors, weather stations, remote controls, paging, packet radio — all of it
carries something a receiver can read, and bandsaunter reads it:
```
21:14:07 433.92 MHz 70 cm Amateur 3.2s SNR 45.8 dB EV1527 / PT2262-style remote (93%)
EV1527 / PT2262-style remote 24 bits 516 baud x12 B2 35 4E
21:14:31 929.6125 MHz UHF / 900 MHz Paging 4.5s SNR 49.5 dB POCSAG 1200 (97%)
[1234568D] ENGINE 4 RESPOND
[0098765A] CALL EXT 4412
21:15:02 144.39 MHz 2 m Amateur 1.8s SNR 31.2 dB AX.25 / APRS (93%)
W1AW>APRS>WIDE1-1: !4142.45N/07243.63W-Newington CT
```
**Whatever the modulation, a data signal is the same shape once it has been
sliced**: a train of alternating runs whose *lengths* carry the information.
On-off keying gives that directly — the carrier is up or it is down — and
two-level FSK gives exactly the same thing from the discriminator, one tone or
the other. So both are reduced to runs, and everything after that is shared.
What the runs mean is the line code, and it is worked out from the runs alone
rather than configured, because each code makes a different prediction about
which of the two histograms is the bimodal one:
| Code | Pulses | Gaps | Who uses it |
|---|---|---|---|
| **PWM** | two lengths | constant, or the period is | EV1527, PT2262 and nearly every 433 MHz remote |
| **PPM** | constant | two lengths | the other half of the same market |
| **Manchester** | T and 2T only | T and 2T only | anything whose receiver recovers its own clock |
| **NRZ** | any whole number of symbols | same | what a framed protocol sits on |
Four-level FSK — C4FM, as P25, DMR and NXDN send it — is recognised as such and
read as symbols. Slicing it down the middle also produces bits, and they mean
nothing; a capture that had been coming back as "10783 bits of NRZ at 5335
baud" now says *4-level FSK, 5334 baud, no frame sync recognised*, which is
both true and useful. Where a frame sync word does appear, the system is named
outright.
### Protocols that can be read in full
Two carry their own framing and checksums, so a frame either passes or it does
not — and one that passes is not a guess:
**POCSAG** paging, at 512, 1200 or 2400 baud. Nothing in the signal announces
which rate it is, so all three are tried and the one whose 32-bit sync word
turns up is the right one. Every codeword is checked — and a single bit error
corrected — against the BCH code the standard puts there for exactly that. The
address, function letter and message text all come out.
**AX.25 / APRS** on 1200 baud AFSK. The frame check has to come out right
before a frame is reported at all. The sender's callsign, the digipeater path
and the payload are shown — and the callsign goes onto the map with everyone
else.
```bash
bandsaunter analyze capture.cf32 --rate 48000 # decode a file you already have
bandsaunter scan --no-decode-data # turn it off
saunterbrowse # decoded packets sit where a transcript would
```
### Believing a decode
This is the hard half. A decoder that always returns *something* is worse than
useless: noise sliced at a threshold produces runs, and runs produce bits.
Three things guard against that.
- **The runs have to fit.** A decode whose runs do not quantise to the line
code's own grid is thrown away.
- **Most of the capture has to agree.** A data signal is data all the way
through. One lucky window in eight is a coincidence — and that is exactly
what SSB voice produced before this check existed.
- **The packet has to repeat.** Much the strongest of the three. These
transmitters send the same thing three to ten times over, and bits that come
back identical every time did not come from noise.
A bare reading with none of that behind it — where the run lengths merely
happened to land on a grid — is reported as **nothing at all**, rather than as
a bit string with a low number beside it that somebody will read anyway. Across
27 recordings of speech, music, static, a bare carrier, Morse and PSK, the
decoder returns nothing 27 times.
And a decode that *does* have repeats or a checksum behind it outranks the
content check. A burst of keying demodulated as FM audio is a buzz, and the
speech detector likes a buzz — but a frame whose own checksum came out right is
not a statistic.
### Trunked systems and their control channels ### Trunked systems and their control channels
Police, fire and most large business radio in the US runs on *trunked* Police, fire and most large business radio in the US runs on *trunked*

2
bandsaunter/__init__.py
View file

@ -9,7 +9,7 @@ and transcribing speech.
# 2026-08-21_02 is the second build made on the 21st. The revision is padded # 2026-08-21_02 is the second build made on the 21st. The revision is padded
# to two digits so versions sort as text. # to two digits so versions sort as text.
VERSION_DATE = "2026-08-28" VERSION_DATE = "2026-08-28"
VERSION_REVISION = 1 VERSION_REVISION = 2
__version__ = f"{VERSION_DATE}_{VERSION_REVISION:02d}" __version__ = f"{VERSION_DATE}_{VERSION_REVISION:02d}"

112
bandsaunter/browse.py
View file

@ -31,6 +31,7 @@ from rich.align import Align
from rich.console import Console, Group from rich.console import Console, Group
from rich.layout import Layout from rich.layout import Layout
from rich.live import Live from rich.live import Live
from rich.markup import escape
from rich.panel import Panel from rich.panel import Panel
from rich.table import Table from rich.table import Table
from rich.text import Text from rich.text import Text
@ -85,6 +86,7 @@ class Capture:
_transcript: str | None = field(default=None, init=False, repr=False) _transcript: str | None = field(default=None, init=False, repr=False)
_duration: float | None = field(default=None, init=False, repr=False) _duration: float | None = field(default=None, init=False, repr=False)
_calls: list | None = field(default=None, init=False, repr=False) _calls: list | None = field(default=None, init=False, repr=False)
_decoded: list | None = field(default=None, init=False, repr=False)
# -- lazily read sidecars --------------------------------------------- # -- lazily read sidecars ---------------------------------------------
@property @property
@ -139,6 +141,49 @@ class Capture:
pass pass
return self._duration return self._duration
@property
def decoded(self) -> list[str]:
"""What a data capture turned out to say, a line at a time.
The file beside the recording wins over the sidecar for the same
reason the transcript does: it is what the most recent run wrote.
"""
if self._decoded is None:
lines: list[str] = []
try:
text = self.path.with_name(
self.path.stem + "_data.txt").read_text()
lines = [ln for ln in text.splitlines()
if ln.strip() and not ln.startswith("#")]
except OSError:
saved = self.meta.get("data_messages") or []
lines = [str(m) for m in saved if str(m).strip()]
for key in ("data_hex", "data_bits"):
value = str(self.meta.get(key, "")).strip()
if value and not saved:
lines.append(value)
self._decoded = lines
return self._decoded
@property
def data_headline(self) -> str:
"""What kind of data it was, never what the data said."""
protocol = str(self.meta.get("data_protocol", "")).strip()
encoding = str(self.meta.get("data_encoding", "")).strip()
if not (protocol or encoding):
return "decoded data" if self.decoded else ""
bits = [protocol or f"{encoding} data"]
baud = self.meta.get("baud")
if baud:
bits.append(f"{float(baud):.0f} baud")
repeats = int(self.meta.get("data_repeats") or 0)
if repeats > 1:
bits.append(f"x{repeats}")
checks = self.meta.get("data_checks") or []
if checks:
bits.append(str(checks[0]))
return " ".join(bits)
@property @property
def callsigns(self) -> list[str]: def callsigns(self) -> list[str]:
"""Callsign-shaped runs in the transcript, found once and kept.""" """Callsign-shaped runs in the transcript, found once and kept."""
@ -499,6 +544,10 @@ class Browser:
# of a range, and remembering 420-450 MHz is not the point. # of a range, and remembering 420-450 MHz is not the point.
if any(q in band.lower() for band in cap.bands): if any(q in band.lower() for band in cap.bands):
return True return True
# And in what a data capture said: a pager message is as searchable
# as a spoken one, and the reason for searching is the same.
if any(q in line.lower() for line in cap.decoded):
return True
return q in cap.transcript.lower() return q in cap.transcript.lower()
@property @property
@ -567,7 +616,10 @@ class Browser:
# they get room of their own rather than eating into it: a taller # they get room of their own rather than eating into it: a taller
# ceiling when there are any, and never fewer than enough to show them. # ceiling when there are any, and never fewer than enough to show them.
ceiling = max(6, min(20 if calls else 16, screen // (2 if calls else 3))) ceiling = max(6, min(20 if calls else 16, screen // (2 if calls else 3)))
needed = len(self._transcript_lines()) + len(calls) + 4 cap = self.current
body = len(self._transcript_lines()) or (
len(cap.decoded) + 2 if cap is not None and cap.decoded else 0)
needed = body + len(calls) + 4
if calls: if calls:
needed += 1 # the blank line above needed += 1 # the blank line above
floor = min(ceiling, len(calls) + 6) floor = min(ceiling, len(calls) + 6)
@ -614,6 +666,22 @@ class Browser:
height = self._transcript_height() height = self._transcript_height()
if cap is None: if cap is None:
return Panel("", border_style="bright_black", height=height) return Panel("", border_style="bright_black", height=height)
if not cap.transcript and cap.decoded:
# A data capture has no words, but it does have content, and it
# belongs in the same place a transcript would be: at the top,
# where the reader is already looking.
body = Text()
headline = cap.data_headline
if headline:
body.append(headline + "\n\n", style="bold cyan")
room = max(1, height - 4 - (2 if headline else 0))
shown = cap.decoded[:room]
body.append("\n".join(shown), style="bold white")
if len(cap.decoded) > len(shown):
body.append(f"\n… {len(cap.decoded) - len(shown)} more line(s)"
" — press t to read it all", style="yellow")
return Panel(body, title="decoded", title_align="left",
border_style="cyan", padding=(1, 3), height=height)
if not cap.transcript: if not cap.transcript:
inner = Align.center(Text(self._why_no_transcript(cap), inner = Align.center(Text(self._why_no_transcript(cap),
style="bright_black", justify="center"), style="bright_black", justify="center"),
@ -696,6 +764,11 @@ class Browser:
if cap.meta.get("morse_wpm"): if cap.meta.get("morse_wpm"):
line.append(f" {float(cap.meta['morse_wpm']):.0f} WPM", line.append(f" {float(cap.meta['morse_wpm']):.0f} WPM",
style="yellow") style="yellow")
checks = cap.meta.get("data_checks") or []
if checks:
# A checksum that came out right is the strongest thing anyone
# can say about a decode, so it is said on the front line.
line.append(f" {checks[0]} ✓", style="green")
second = Text() second = Text()
bands = cap.bands bands = cap.bands
@ -743,7 +816,9 @@ class Browser:
base = "on grey19 " if here else "" base = "on grey19 " if here else ""
cat = CATEGORY_STYLE.get(cap.category, "white") cat = CATEGORY_STYLE.get(cap.category, "white")
when = cap.when when = cap.when
summary = cap.transcript.replace("\n", " ") or cap.classification summary = (cap.transcript.replace("\n", " ")
or (cap.decoded[0] if cap.decoded else "")
or cap.classification)
t.add_row( t.add_row(
Text(mark, style=base + ("bold red" if playing Text(mark, style=base + ("bold red" if playing
else "bold cyan")), else "bold cyan")),
@ -770,13 +845,17 @@ class Browser:
border_style="blue", padding=(0, 1)) border_style="blue", padding=(0, 1))
def _footer(self) -> Text: def _footer(self) -> Text:
# Both of these carry text that did not come from this program -- what
# the user typed, a filename, a player's error -- and rich reads a
# square bracket as markup. Typing "[/" at the search prompt used to
# end the session with a MarkupError.
if self.searching: if self.searching:
return Text.from_markup( return Text.from_markup(
f"[bold]search:[/bold] {self.query}[blink]_[/blink]" f"[bold]search:[/bold] {escape(self.query)}[blink]_[/blink]"
" [bright_black]enter to accept, esc to clear" " [bright_black]enter to accept, esc to clear"
"[/bright_black]") "[/bright_black]")
if self.message: if self.message:
return Text.from_markup(f"[yellow]{self.message}[/yellow]") return Text.from_markup(f"[yellow]{escape(self.message)}[/yellow]")
if self.player.active and self.player.playing is not None: if self.player.active and self.player.playing is not None:
total = self.player.playing.duration total = self.player.playing.duration
done = self.player.elapsed done = self.player.elapsed
@ -837,6 +916,13 @@ class Browser:
calls = self._callsign_lines(pad=4) calls = self._callsign_lines(pad=4)
room = max(3, screen - 4 - (len(calls) + 1 if calls else 0)) room = max(3, screen - 4 - (len(calls) + 1 if calls else 0))
lines = self._transcript_lines(pad=4) lines = self._transcript_lines(pad=4)
title_word = "transcript"
if not lines and cap is not None and cap.decoded:
# A long paging capture is as worth reading in full as a long
# net, and there is nowhere else to read it.
lines = ([cap.data_headline, ""] if cap.data_headline else []) \
+ list(cap.decoded)
title_word = "decoded"
self.read_top = max(0, min(self.read_top, max(0, len(lines) - room))) self.read_top = max(0, min(self.read_top, max(0, len(lines) - room)))
shown = lines[self.read_top:self.read_top + room] shown = lines[self.read_top:self.read_top + room]
body = Text("\n".join(shown), style="white") body = Text("\n".join(shown), style="white")
@ -851,7 +937,7 @@ class Browser:
body.append("\n" + line) body.append("\n" + line)
where = (f"{self.read_top + 1}-{self.read_top + len(shown)}" where = (f"{self.read_top + 1}-{self.read_top + len(shown)}"
f" of {len(lines)}" if len(lines) > room else "") f" of {len(lines)}" if len(lines) > room else "")
title = "transcript" title = title_word
if cap is not None and cap.frequency: if cap is not None and cap.frequency:
title += f" · {fmt_hz(cap.frequency)}" title += f" · {fmt_hz(cap.frequency)}"
when = cap.when when = cap.when
@ -1062,6 +1148,20 @@ def build_parser() -> argparse.ArgumentParser:
return p return p
def _first_line(cap: Capture) -> str:
"""The most informative thing about a capture, in one line."""
if cap.transcript:
return cap.transcript.splitlines()[0]
if cap.decoded:
# A pager message speaks for itself; a packet of hex does not, so
# that one is introduced by what kind of packet it is.
first = cap.decoded[0]
headline = cap.data_headline
return first if " " in first.strip(" 0123456789ABCDEF") or not headline \
else f"{headline} {first}"
return cap.classification
def _write_kml(console: Console, browser: "Browser", book: CallsignBook, def _write_kml(console: Console, browser: "Browser", book: CallsignBook,
where: str) -> int: where: str) -> int:
"""Build a map from the transcripts already on disk. """Build a map from the transcripts already on disk.
@ -1171,7 +1271,7 @@ def main(argv: list[str] | None = None) -> int:
line = (f"{fmt_hz(cap.frequency):>14} {shorten_band(cap.band, 20):<20} " line = (f"{fmt_hz(cap.frequency):>14} {shorten_band(cap.band, 20):<20} "
f"{when} " f"{when} "
f"{_dur(cap.duration):>7} {cap.category:<8} " f"{_dur(cap.duration):>7} {cap.category:<8} "
f"{cap.transcript.splitlines()[0] if cap.transcript else cap.classification}") f"{_first_line(cap)}")
console.print(line, highlight=False, soft_wrap=True) console.print(line, highlight=False, soft_wrap=True)
return 0 return 0

18
bandsaunter/cli.py
View file

@ -828,6 +828,24 @@ def cmd_analyze(args) -> int:
console.print(Panel(Text.from_markup("\n".join(body)), console.print(Panel(Text.from_markup("\n".join(body)),
title="identification", border_style="green")) title="identification", border_style="green"))
# Whatever it is, try to read it: the whole point of pointing this at a
# file is to find out what is in it.
from .decode import decode_data
got = decode_data(iq, rate, family=cls.family,
baud_hint=cls.features.baud if cls.features else 0.0)
if got.ok:
# Printed as plain text, not markup: a decoded packet is arbitrary
# bytes from the air, and square brackets in it are common.
lines = got.report()
body = Text(lines[0], style="bold")
for line in lines[1:]:
body.append("\n" + line)
body.append(f"\n{got.confidence * 100:.0f}% confident",
style="not bold grey62")
console.print(Panel(body, title="decoded data", border_style="cyan"))
elif cls.family in ("ook", "fsk", "psk", "digital", "control"):
console.print(f"[yellow]nothing decoded: {got.note}[/yellow]")
f = cls.features f = cls.features
if f: if f:
t = Table(box=None, header_style="bold") t = Table(box=None, header_style="bold")

1
bandsaunter/config.py
View file

@ -108,6 +108,7 @@ class ScanConfig:
iq_format: str = "cf32" # cf32 or cs16 iq_format: str = "cf32" # cf32 or cs16
classify: bool = True classify: bool = True
decode_morse: bool = True decode_morse: bool = True
decode_data: bool = True # read packets out of data signals
# -- one file per frequency ------------------------------------------ # -- one file per frequency ------------------------------------------
combine_by_frequency: bool = False combine_by_frequency: bool = False

1226
bandsaunter/decode.py Normal file
View file

File diff suppressed because it is too large Load diff

453
bandsaunter/protocols.py Normal file
View file

@ -0,0 +1,453 @@
"""Data protocols that carry their own framing, and so can be read outright.
The generic decoder in :mod:`bandsaunter.decode` recovers bits and has to
argue about whether they are real. The protocols here do not have that
problem: each has a sync word to find and a checksum to verify, so a frame
either passes or it does not, and one that passes is not a guess.
Two are implemented, because between them they cover most of what an ordinary
receiver actually hears carrying words rather than measurements:
``POCSAG``
Paging. Still in daily use by hospitals, fire services and industrial
plant long after the consumer pagers went away, and it carries plain
text. 512, 1200 or 2400 baud two-level FSK.
``AX.25 / APRS``
Amateur packet. 1200 baud AFSK on VHF, and the payload is position
reports, weather and messages -- with the sender's callsign in the
header, which the map already knows what to do with.
"""
from __future__ import annotations
import math
import numpy as np
from .decode import DataDecode, PulseTrain, nrz_bits, slice_fsk
__all__ = ["decode_pocsag", "decode_ax25", "PROTOCOLS", "AX25Frame",
"POCSAG_BAUDS", "SYNC_WORD", "MAX_ADDRESS", "pocsag_bits",
"pocsag_codeword"]
# ---------------------------------------------------------------------------
# Shared
# ---------------------------------------------------------------------------
def _find(bits: str, pattern: str, start: int = 0) -> int:
return bits.find(pattern, start)
def _invert(bits: str) -> str:
return bits.translate(str.maketrans("01", "10"))
# ---------------------------------------------------------------------------
# POCSAG
# ---------------------------------------------------------------------------
# The frame synchronisation codeword, sent before each batch of sixteen.
SYNC_WORD = 0x7CD215D8
SYNC_BITS = f"{SYNC_WORD:032b}"
# The three rates in the standard. Which one a transmitter uses is not
# announced anywhere in the signal, so all three are tried and the one whose
# sync word appears is the right one -- a 32-bit pattern does not turn up in
# the wrong reading by accident.
POCSAG_BAUDS: tuple[float, ...] = (1200.0, 512.0, 2400.0)
# BCH(31,21) with an even parity bit, which is what protects each codeword.
_BCH_POLY = 0b11101101001 # x^10 + x^9 + x^8 + x^6 + x^5 + x^3 + 1
_IDLE = 0x7A89C197
# The 20 data bits of a message codeword are packed end to end and then read
# out either as 7-bit ASCII or as 4-bit digits, depending on the pager.
_NUMERIC = "0123456789*U -)("
def _bch_syndrome(word: int) -> int:
"""Zero when the codeword's BCH check passes."""
remainder = word >> 1 # drop the parity bit
for shift in range(30, 9, -1):
if remainder & (1 << shift):
remainder ^= _BCH_POLY << (shift - 10)
return remainder & 0x3FF
def _parity_ok(word: int) -> bool:
return bin(word).count("1") % 2 == 0
def _correct(word: int) -> tuple[int, bool]:
"""Check a codeword, correcting a single bit error if there is one.
Worth doing rather than discarding: one bad bit in thirty-two is exactly
what a fading paging signal delivers, and the BCH code was put there to
survive it.
"""
if _bch_syndrome(word) == 0 and _parity_ok(word):
return word, True
for bit in range(32):
trial = word ^ (1 << bit)
if _bch_syndrome(trial) == 0 and _parity_ok(trial):
return trial, True
return word, False
def _pocsag_text(payload: str) -> tuple[str, str]:
"""Read a run of message bits as text and as digits.
Both, because nothing in the message says which it is: an alphanumeric
pager sends 7-bit ASCII least significant bit first, a numeric one sends
4-bit digits, and the only way to tell is to look at what comes out.
"""
letters = []
for i in range(0, len(payload) - 6, 7):
chunk = payload[i:i + 7]
code = int(chunk[::-1], 2) # LSB first on the air
letters.append(chr(code) if 32 <= code < 127 else
("\n" if code in (10, 13) else "."))
digits = []
for i in range(0, len(payload) - 3, 4):
code = int(payload[i:i + 4][::-1], 2)
digits.append(_NUMERIC[code])
return "".join(letters).rstrip(". \n"), "".join(digits).rstrip(" ")
def _readable(text: str) -> float:
"""What share of a string is characters a message would really contain."""
if not text:
return 0.0
good = sum(1 for c in text if c.isalnum() or c in " .,:;/-+()@#'\"!?\n")
return good / len(text)
def _pocsag_batches(bits: str) -> list[tuple[int, str, str]]:
"""Every address and message in a bit stream.
Returns ``(address, function, text)``. Frames are read from each sync
word independently, so a stream that loses lock partway through still
yields everything before and after it.
"""
out: list[tuple[int, str, str]] = []
pos = 0
pending_address: int | None = None
pending_function = 0
pending_bits: list[str] = []
def flush():
if pending_address is None:
return
payload = "".join(pending_bits)
text, digits = _pocsag_text(payload)
# Whichever reading looks more like a message someone would send.
chosen = text if _readable(text) >= 0.75 and len(text) >= 2 else digits
# Function bits 00..11 are the pager's four addresses, written A to
# D by everyone who documents them.
out.append((pending_address, "ABCD"[pending_function & 3],
chosen.strip()))
while True:
at = _find(bits, SYNC_BITS, pos)
if at < 0:
break
pos = at + 32
for frame in range(8):
for half in range(2):
start = pos + (frame * 2 + half) * 32
if start + 32 > len(bits):
pos = len(bits)
break
word = int(bits[start:start + 32], 2)
if word == _IDLE:
flush()
pending_address, pending_bits = None, []
continue
word, valid = _correct(word)
if not valid:
continue
if word & 0x80000000:
# A message codeword: twenty bits of payload.
if pending_address is not None:
pending_bits.append(f"{word:032b}"[1:21])
continue
flush()
# An address codeword. The low three bits of the address are
# not sent: they are which of the eight frames it arrived in.
pending_address = ((word >> 13) & 0x3FFFF) << 3 | frame
pending_function = (word >> 11) & 0x3
pending_bits = []
else:
continue
break
pos += 16 * 32
flush()
return out
def decode_pocsag(x: np.ndarray, sample_rate: float,
baud_hint: float = 0.0) -> DataDecode | None:
"""Read POCSAG paging out of a two-level FSK signal."""
train = slice_fsk(x, sample_rate)
if train is None or len(train) < 16:
return None
order = list(POCSAG_BAUDS)
if baud_hint:
order.sort(key=lambda b: abs(b - baud_hint))
for baud in order:
raw = nrz_bits(train, baud)
if len(raw) < 64:
continue
for bits in (raw, _invert(raw)):
if SYNC_BITS not in bits:
continue
pages = _pocsag_batches(bits)
if not pages:
continue
# A capture usually spans several batches and a transmitter
# repeats its queue, so the same page arrives more than once.
# Reported once, in the order it first appeared.
messages: list[str] = []
for address, function, text in pages:
line = f"[{address:07d}{function}]"
line = f"{line} {text}" if text else line
if line not in messages:
messages.append(line)
syncs = bits.count(SYNC_BITS)
return DataDecode(
ok=True, protocol=f"POCSAG {baud:.0f}", encoding="NRZ",
baud=baud, bits=bits[:256], repeats=syncs,
agreement=1.0 if syncs > 1 else 0.0,
checks=["BCH(31,21)"], messages=messages,
confidence=min(0.97, 0.72 + 0.05 * len(pages)
+ 0.05 * min(3, syncs)))
return None
# ---------------------------------------------------------------------------
# AX.25 over Bell 202 AFSK -- APRS and amateur packet
# ---------------------------------------------------------------------------
MARK_HZ = 1200.0
SPACE_HZ = 2200.0
AFSK_BAUD = 1200.0
_FLAG = "01111110"
class AX25Frame:
"""One AX.25 frame whose frame-check sequence was correct."""
def __init__(self, raw: bytes):
self.raw = raw
self.source = ""
self.destination = ""
self.path: list[str] = []
self.info = ""
self._parse()
def _parse(self) -> None:
addresses = []
i = 0
while i + 7 <= len(self.raw) and len(addresses) < 10:
field = self.raw[i:i + 7]
call = "".join(chr(b >> 1) for b in field[:6]).strip()
ssid = (field[6] >> 1) & 0x0F
addresses.append(f"{call}-{ssid}" if ssid else call)
i += 7
if field[6] & 0x01: # the end-of-address bit
break
if len(addresses) >= 2:
self.destination, self.source = addresses[0], addresses[1]
self.path = addresses[2:]
# Skip the control and protocol-identifier bytes.
body = self.raw[i + 2:] if len(self.raw) > i + 2 else b""
self.info = body.decode("ascii", "replace").rstrip("\r\n")
def describe(self) -> str:
route = ">".join([self.source or "?", self.destination or "?"]
+ self.path)
return f"{route}: {self.info}" if self.info else route
def _fcs(data: bytes) -> int:
"""The AX.25 frame check: CRC-16/X.25, reflected, inverted at the end."""
crc = 0xFFFF
for byte in data:
crc ^= byte
for _ in range(8):
crc = (crc >> 1) ^ 0x8408 if crc & 1 else crc >> 1
return crc ^ 0xFFFF
def _afsk_symbols(x: np.ndarray, sample_rate: float) -> np.ndarray | None:
"""Turn Bell 202 audio into a two-level signal, mark against space.
A correlator rather than a discriminator: the two tones are close enough
together, and radio audio distorted enough, that measuring which of the
two a bit-length window contains more of works where measuring the
instantaneous frequency does not.
"""
x = np.asarray(x)
if np.iscomplexobj(x):
# FM first: AFSK is audio, and on VHF it arrives inside an FM carrier.
phase = np.unwrap(np.angle(x))
audio = np.diff(phase)
else:
audio = x.astype(np.float64)
if audio.size < int(sample_rate / 100.0):
return None
audio = audio - float(audio.mean())
window = int(round(sample_rate / AFSK_BAUD))
if window < 4:
return None
n = np.arange(window)
out = []
for tone in (MARK_HZ, SPACE_HZ):
arg = 2.0 * math.pi * tone * n / sample_rate
i = np.convolve(audio, np.cos(arg)[::-1], mode="same")
q = np.convolve(audio, np.sin(arg)[::-1], mode="same")
out.append(np.hypot(i, q))
return out[0] - out[1]
def _hdlc_frames(bits: str) -> list[bytes]:
"""Split a bit stream at HDLC flags and undo the bit stuffing."""
frames: list[bytes] = []
pos = bits.find(_FLAG)
if pos < 0:
return frames
while True:
# Flags repeat back to back between frames; skip past all of them.
while bits.startswith(_FLAG, pos):
pos += 8
end = bits.find(_FLAG, pos)
if end < 0:
break
body = bits[pos:end]
pos = end
if len(body) < 8 * 17: # shorter than an empty AX.25 frame
continue
# Undo stuffing: a zero inserted after every five ones.
out, ones = [], 0
for bit in body:
if ones == 5:
ones = 0
if bit == "0":
continue # the stuffed bit
out.append(bit)
ones = ones + 1 if bit == "1" else 0
clean = "".join(out)
whole = len(clean) - len(clean) % 8
# Least significant bit first on the air.
frames.append(bytes(int(clean[i:i + 8][::-1], 2)
for i in range(0, whole, 8)))
return frames
def decode_ax25(x: np.ndarray, sample_rate: float,
baud_hint: float = 0.0) -> DataDecode | None:
"""Read AX.25 packet, as APRS uses it, from 1200 baud AFSK."""
soft = _afsk_symbols(x, sample_rate)
if soft is None:
return None
levels = soft > 0
edges = np.flatnonzero(np.diff(levels.astype(np.int8))) + 1
if edges.size < 16:
return None
starts = np.concatenate(([0], edges))
ends = np.concatenate((edges, [levels.size]))
train = PulseTrain(levels[starts], (ends - starts).astype(np.int64),
float(sample_rate), "fsk")
raw = nrz_bits(train, AFSK_BAUD)
if len(raw) < 200:
return None
frames: list[AX25Frame] = []
for stream in (raw, _invert(raw)):
# NRZI: the data is in whether the level changed, not what it is.
decoded = ["1" if a == b else "0"
for a, b in zip(stream, stream[1:])]
for frame in _hdlc_frames("".join(decoded)):
if len(frame) < 17 or _fcs(frame[:-2]) != \
(frame[-1] << 8 | frame[-2]):
continue
frames.append(AX25Frame(frame[:-2]))
if frames:
break
if not frames:
return None
return DataDecode(
ok=True, protocol="AX.25 / APRS", encoding="NRZI",
baud=AFSK_BAUD, bits=raw[:256], repeats=len(frames),
agreement=1.0, checks=["FCS (CRC-16/X.25)"],
messages=[f.describe() for f in frames],
confidence=min(0.98, 0.85 + 0.04 * len(frames)))
# Tried in order; each returns None when the signal is not its own.
PROTOCOLS = (decode_pocsag, decode_ax25)
# ---------------------------------------------------------------------------
# Encoding POCSAG, for the simulator and the tests
# ---------------------------------------------------------------------------
#
# Kept next to the decoder rather than in the test helpers so the two cannot
# drift apart: a bug shared by an encoder and its decoder is invisible, and
# the way to avoid one is to have exactly one copy of the polynomial, the
# sync word and the idle word for both to use.
def pocsag_codeword(payload: int) -> int:
"""Add the BCH check bits and the parity bit to 21 bits of payload."""
word = (payload & 0x1FFFFF) << 10
remainder = word
for shift in range(30, 9, -1):
if remainder & (1 << shift):
remainder ^= _BCH_POLY << (shift - 10)
word = (word | (remainder & 0x3FF)) << 1
return word | (1 if bin(word).count("1") % 2 else 0)
# Eighteen bits go out on the air and three more are implied by which frame
# the codeword arrived in, so this is the whole address space.
MAX_ADDRESS = (1 << 21) - 1
def pocsag_bits(pages, preamble: int = 600) -> str:
"""A complete POCSAG transmission: preamble, then batches of sixteen.
Raises on an address that will not fit rather than truncating it: a
stream built around a silently mangled address decodes to a different
pager, which is worse than not building one.
"""
for address, _, _ in pages:
if not 0 <= address <= MAX_ADDRESS:
raise ValueError(
f"POCSAG address {address} is outside 0-{MAX_ADDRESS}")
out = ["10" * (preamble // 2)]
batch: list[int] = []
def flush() -> None:
if not batch:
return
while len(batch) < 16:
batch.append(_IDLE)
out.append(SYNC_BITS + "".join(f"{w:032b}" for w in batch[:16]))
batch.clear()
for address, function, text in pages:
frame = address & 0x7
while len(batch) < frame * 2:
batch.append(_IDLE)
batch.append(pocsag_codeword(((address >> 3) << 2) | (function & 3)))
payload = "".join(f"{ord(c):07b}"[::-1] for c in text)
payload += "0" * (-len(payload) % 20)
for i in range(0, len(payload), 20):
batch.append(pocsag_codeword((1 << 20) | int(payload[i:i + 20], 2)))
if len(batch) >= 16:
flush()
flush()
return "".join(out)

43
bandsaunter/recorder.py
View file

@ -68,6 +68,19 @@ class HitRecord:
ctcss_hz: float = 0.0 ctcss_hz: float = 0.0
baud: float = 0.0 baud: float = 0.0
# What a data signal turned out to say. Kept apart from the transcript
# because it is not speech and nothing that reads transcripts should
# have to know about it.
data_protocol: str = "" # POCSAG, AX.25, EV1527-style, ...
data_encoding: str = "" # PWM / PPM / Manchester / NRZ / 4-level FSK
data_bits: str = ""
data_hex: str = ""
data_repeats: int = 0
data_checks: list[str] = field(default_factory=list)
data_messages: list[str] = field(default_factory=list)
data_confidence: float = 0.0
data_path: str = ""
category: str = "" # voice / cw / digital / carrier / noise category: str = "" # voice / cw / digital / carrier / noise
signal_score: float = 0.0 signal_score: float = 0.0
voice_score: float = 0.0 voice_score: float = 0.0
@ -85,6 +98,34 @@ class HitRecord:
kept: bool = True kept: bool = True
features: dict = field(default_factory=dict) features: dict = field(default_factory=dict)
def data_headline(self) -> str:
"""What kind of data it was, never what the data said."""
if not self.data_encoding:
return ""
bits = [self.data_protocol or f"{self.data_encoding} data"]
if self.data_messages:
n = len(self.data_messages)
bits.append(f"{n} message{'' if n == 1 else 's'}")
elif self.data_bits:
bits.append(f"{len(self.data_bits)} bits")
if self.baud:
bits.append(f"{self.baud:.0f} baud")
if self.data_repeats > 1:
bits.append(f"x{self.data_repeats}")
if self.data_checks:
bits.append(self.data_checks[0])
return " ".join(bits)
def data_summary(self) -> str:
"""One line for a display, or "" if nothing was decoded.
The message where there is one, because that is what somebody wants
to read; the description of the packet where there is not.
"""
if self.data_messages:
return self.data_messages[0]
return self.data_headline()
def to_dict(self) -> dict: def to_dict(self) -> dict:
return asdict(self) return asdict(self)
@ -95,6 +136,8 @@ class HitRecord:
self.classification or "unclassified"] self.classification or "unclassified"]
if self.morse_text: if self.morse_text:
bits.append(f'CW "{self.morse_text.strip()[:40]}"') bits.append(f'CW "{self.morse_text.strip()[:40]}"')
elif self.data_messages:
bits.append(self.data_messages[0][:48])
elif self.ctcss_hz: elif self.ctcss_hz:
bits.append(f"CTCSS {self.ctcss_hz:.1f}") bits.append(f"CTCSS {self.ctcss_hz:.1f}")
return " ".join(bits) return " ".join(bits)

114
bandsaunter/scanner.py
View file

@ -24,6 +24,7 @@ from .bandplan import fmt_hz, label_for, presets_covering
from .callsign import CallsignBook, find_callsigns from .callsign import CallsignBook, find_callsigns
from .classify import classify, ssb_alignment from .classify import classify, ssb_alignment
from .config import ScanConfig, remember_lockouts from .config import ScanConfig, remember_lockouts
from .decode import decode_data
from .demod import make_demodulator from .demod import make_demodulator
from .device import RtlSdrDevice, RtlSdrError from .device import RtlSdrDevice, RtlSdrError
from .kml import KmlLog from .kml import KmlLog
@ -879,7 +880,11 @@ class Scanner:
self._note_control(det.frequency, verdict) self._note_control(det.frequency, verdict)
self._reject(rec, hit) self._reject(rec, hit)
return hit return hit
if cfg.require_signal and verdict is not None and not verdict.accept: if cfg.require_signal and verdict is not None and not verdict.accept \
and not hit.data_messages:
# A decoded packet outranks the content check. That test works
# from statistics -- how noise-like, how speech-like -- and a
# frame whose own checksum came out right is not a statistic.
hit.kept = False hit.kept = False
hit.stop_reason = f"no signal content: {verdict.reason}" hit.stop_reason = f"no signal content: {verdict.reason}"
self._reject(rec, hit) self._reject(rec, hit)
@ -891,6 +896,7 @@ class Scanner:
hit.filename = rec.stem hit.filename = rec.stem
hit.audio_path = str(rec.audio_path) if cfg.save_audio else "" hit.audio_path = str(rec.audio_path) if cfg.save_audio else ""
hit.iq_path = str(rec.iq_path) if cfg.save_iq else "" hit.iq_path = str(rec.iq_path) if cfg.save_iq else ""
self._write_data_file(rec, hit)
# Add this transmission to the running file for its frequency, after # Add this transmission to the running file for its frequency, after
# a spoken timestamp, so one channel plays back as one recording. # a spoken timestamp, so one channel plays back as one recording.
@ -1060,6 +1066,93 @@ class Scanner:
meta_path=Path(hit.meta_path) if hit.meta_path else None, meta_path=Path(hit.meta_path) if hit.meta_path else None,
recording=rec.audio_path.name) recording=rec.audio_path.name)
# Families whose signals carry bits. Voice and Morse are excluded not to
# save the work -- it is a fraction of a second -- but because a decoder
# run over speech will eventually find a window that fits, and a scanner
# that occasionally reports a doorbell code from a conversation is worse
# than one that never looks.
DATA_FAMILIES = ("ook", "fsk", "psk", "digital", "control", "data")
def _decode_payload(self, rec: Recording, hit: HitRecord, demod,
cls) -> None:
"""Read the bits out of a data capture and attach them to the hit."""
if not self.cfg.decode_data or cls.family not in self.DATA_FAMILIES:
return
iq = rec.classification_iq(limit=0)
if iq.size < 2048:
return
try:
got = decode_data(iq, demod.if_rate, family=cls.family,
baud_hint=hit.baud)
except Exception as exc:
self._error(exc)
return
if not got.ok:
return
hit.data_protocol = got.protocol
hit.data_encoding = got.encoding
hit.data_bits = got.bits[:512]
hit.data_hex = got.hex[:512]
hit.data_repeats = got.repeats
hit.data_checks = list(got.checks)
hit.data_messages = list(got.messages)
hit.data_confidence = round(got.confidence, 3)
if got.baud:
hit.baud = got.baud
if got.protocol:
# A named protocol beats the modulation label: "2-FSK" is true
# and useless where "POCSAG 1200" says what it is.
hit.classification = got.protocol
hit.confidence = max(hit.confidence, got.confidence)
hit.reasons.insert(0, f"decoded: {got.summary()}")
self._announce_data(hit, got)
def _write_data_file(self, rec: Recording, hit: HitRecord) -> None:
"""Put what was decoded beside the recording, as text.
A separate file from the transcript: this is not speech, and the
browser, the callsign search and anything else that reads transcripts
should not have to sort one kind from the other. Written only once
the capture has been renamed and kept, so it cannot be left orphaned
beside a recording that was thrown away.
"""
if not (hit.data_messages or hit.data_bits):
return
path = rec.dir / f"{rec.stem}_data.txt"
lines = [f"# {fmt_hz(hit.frequency)} {hit.started_iso}",
f"# {hit.data_headline()}"]
lines += hit.data_messages
if hit.data_bits and not hit.data_messages:
lines += [hit.data_hex, hit.data_bits]
try:
path.write_text("\n".join(lines) + "\n")
hit.data_path = str(path)
except OSError as exc:
self._error(exc)
def _announce_data(self, hit: HitRecord, got) -> None:
"""Say on the display what was just read off the air."""
self._status(f"decoded {fmt_hz(hit.frequency)}: {got.summary()}")
# APRS carries callsigns, and the map already knows what to do with
# one. Nothing else in a data packet is a person.
book = self.callsigns
if book is None or "AX.25" not in got.protocol:
return
changed = False
for message in got.messages:
call = message.split(">", 1)[0].split("-", 1)[0].strip()
if not call or not call.isalnum():
continue
entry = book.get(call)
self.heard[entry.call] = self.heard.get(entry.call, 0) + 1
if self.kml is not None:
changed |= self.kml.add(entry, hit.frequency, hit.started_at,
hit.band, hit.filename)
if changed and self.kml is not None:
# Saved as we go, not only at the end: a scan left running
# overnight and stopped with a signal should still have its map.
self.kml.save()
def _on_transcript(self, path, result, job) -> None: def _on_transcript(self, path, result, job) -> None:
"""Pull callsigns out of a finished transcript and map them. """Pull callsigns out of a finished transcript and map them.
@ -1141,6 +1234,10 @@ class Scanner:
if k != "extras" and not k.startswith("_") if k != "extras" and not k.startswith("_")
} }
# After the features, so the decoder is handed the classifier's own
# symbol-rate estimate rather than a zero.
self._decode_payload(rec, hit, demod, cls)
if morse is not None and morse.is_morse: if morse is not None and morse.is_morse:
hit.morse_text = morse.text hit.morse_text = morse.text
hit.morse_wpm = round(morse.wpm, 1) hit.morse_wpm = round(morse.wpm, 1)
@ -1152,7 +1249,13 @@ class Scanner:
hit.reasons.append( hit.reasons.append(
"keyed carrier but the Morse timing did not resolve") "keyed carrier but the Morse timing did not resolve")
# A decode that has repeats or a checksum behind it outranks the
# audio content check below. A burst of on-off keying demodulated as
# FM audio is a buzz, and the speech detector likes a buzz; but bits
# that arrived identically twelve times are not a conversation.
decoded_firmly = hit.data_confidence >= 0.7
if verdict is not None and verdict.category == "voice" and \ if verdict is not None and verdict.category == "voice" and \
not decoded_firmly and \
hit.family not in ("nfm", "wfm", "am", "ssb"): hit.family not in ("nfm", "wfm", "am", "ssb"):
# The content check found speech, so the modulation label was # The content check found speech, so the modulation label was
# wrong. Speech on a quiet FM channel is easy to mistake for # wrong. Speech on a quiet FM channel is easy to mistake for
@ -1174,6 +1277,15 @@ class Scanner:
hit.classification = label hit.classification = label
hit.family = fam hit.family = fam
if decoded_firmly and verdict is not None and \
verdict.category != "digital":
# It carries data: that is what "content" means for a signal
# nobody speaks on, and the packet is better evidence of it than
# any statistic the content check could compute.
verdict.category = "digital"
verdict.accept = True
verdict.reason = f"decoded: {hit.data_summary()}"
if verdict is not None: if verdict is not None:
hit.category = verdict.category hit.category = verdict.category
hit.signal_score = round(verdict.score, 3) hit.signal_score = round(verdict.score, 3)

20
bandsaunter/settings.py
View file

@ -390,6 +390,14 @@ _TABLE: tuple[Setting, ...] = (
unit="s", minimum=0.0, flags=("--transcribe-min",), metavar="SEC"), unit="s", minimum=0.0, flags=("--transcribe-min",), metavar="SEC"),
# -- callsigns --------------------------------------------------------- # -- callsigns ---------------------------------------------------------
S("decode_data", "Decode data signals", "Output", "bool",
"read the packets out of anything carrying data",
"On-off keyed remotes and sensors, two-level FSK, POCSAG paging and "
"APRS packet are all read down to their bits, and named where the "
"framing gives them away. Costs a fraction of a second per capture "
"and only runs on captures the classifier called data.",
flags=("--decode-data",), off_flags=("--no-decode-data",)),
S("callsign_lookup", "Look callsigns up", "Callsigns", "bool", S("callsign_lookup", "Look callsigns up", "Callsigns", "bool",
"ask the licence database who a callsign belongs to", "ask the licence database who a callsign belongs to",
"Callsigns heard in a transcript are looked up in the FCC's published " "Callsigns heard in a transcript are looked up in the FCC's published "
@ -763,6 +771,18 @@ _GUIDANCE: dict[str, str] = {
"Do not bother transcribing captures shorter than this. Very short " "Do not bother transcribing captures shorter than this. Very short "
"clips rarely contain a whole word and mostly produce noise or " "clips rarely contain a whole word and mostly produce noise or "
"nothing, while still costing the processing.", "nothing, while still costing the processing.",
"decode_data":
"Read what a data signal actually says. A great deal of what a "
"scanner finds is not speech: doorbells, tyre-pressure sensors, "
"weather stations, remote controls, paging, packet radio. Each one "
"is sliced into its pulses, the line code worked out from the "
"pulse lengths alone, and the bits reported -- with the packet "
"named where its framing says what it is, and the message printed "
"in full where the protocol carries one. The check that keeps it "
"honest is repetition: these transmitters send the same packet "
"several times over, and bits that come back identical every time "
"did not come from noise. Turn it off to save a little processing "
"on a busy band.",
"callsign_lookup": "callsign_lookup":
"When someone gives their callsign, look it up and say who they " "When someone gives their callsign, look it up and say who they "
"are. The data is the FCC's own published licence register, which " "are. The data is the FCC's own published licence register, which "

59
bandsaunter/simulator.py
View file

@ -107,7 +107,8 @@ class VirtualTransmitter:
"""One synthetic signal on the air.""" """One synthetic signal on the air."""
frequency: float frequency: float
mode: str = "nfm" # nfm wfm am usb cw fsk2 fsk4 psk carrier ook mode: str = "nfm" # nfm wfm am usb cw fsk2 fsk4 psk
# carrier ook packet pocsag
power: float = 0.35 # linear amplitude power: float = 0.35 # linear amplitude
bandwidth: float = 12_500.0 bandwidth: float = 12_500.0
label: str = "" label: str = ""
@ -121,6 +122,12 @@ class VirtualTransmitter:
wpm: float = 18.0 wpm: float = 18.0
baud: float = 4800.0 baud: float = 4800.0
deviation: float = 2_500.0 deviation: float = 2_500.0
# For the packet modes: the payload, and how many times a transmitter
# repeats it. Cheap remotes send everything three to ten times over,
# which is what makes a decode checkable.
payload: str = "101100100011010101001110"
repeats: int = 5
pages: tuple = () # (address, function, text) for POCSAG
pitch_hz: float = 120.0 # synthetic talker's voice pitch pitch_hz: float = 120.0 # synthetic talker's voice pitch
_phase: float = field(default=0.0, init=False, repr=False) _phase: float = field(default=0.0, init=False, repr=False)
@ -175,6 +182,19 @@ class VirtualTransmitter:
elif m == "ook": elif m == "ook":
env = (self._symbols(t, fs, 2, self.baud) > 0).astype(np.float64) env = (self._symbols(t, fs, 2, self.baud) > 0).astype(np.float64)
out = env * np.exp(1j * self._advance(np.zeros(n), fs)) out = env * np.exp(1j * self._advance(np.zeros(n), fs))
elif m == "pocsag":
# A real paging batch, not random keying: preamble, sync word,
# addressed message codewords with their BCH check bits. A
# decoder has to have something correct to find.
lv = self._pocsag_levels(t, fs)
out = np.exp(1j * self._advance(lv * self.deviation, fs))
elif m == "packet":
# A real remote, not a random bit stream: a pulse-width coded
# payload sent several times over with a silence between the
# repeats. Random keying exercises the classifier but there is
# nothing in it for a decoder to get right.
env = self._packet_envelope(t, fs)
out = env * np.exp(1j * self._advance(np.zeros(n), fs))
elif m in ("fsk2", "fsk4"): elif m in ("fsk2", "fsk4"):
levels = 2 if m == "fsk2" else 4 levels = 2 if m == "fsk2" else 4
lv = self._shaped_levels(t, fs, levels, self.baud) lv = self._shaped_levels(t, fs, levels, self.baud)
@ -188,6 +208,33 @@ class VirtualTransmitter:
return (self.power * out).astype(np.complex64) return (self.power * out).astype(np.complex64)
def _pocsag_levels(self, t: np.ndarray, fs: float) -> np.ndarray:
"""The POCSAG bit stream as +/-1, read out against absolute time."""
from .protocols import pocsag_bits
pages = tuple(self.pages) or ((1234568, 3, "TEST PAGE"),)
bits = pocsag_bits(pages)
table = np.where(np.frombuffer(bits.encode(), dtype=np.uint8) ==
ord("1"), 1.0, -1.0)
unit = 1.0 / max(1.0, self.baud)
pos = np.floor((t % (table.size * unit)) / unit).astype(np.int64)
return table[np.clip(pos, 0, table.size - 1)]
def _packet_envelope(self, t: np.ndarray, fs: float) -> np.ndarray:
"""Pulse-width keying of ``payload``, repeated, on absolute time.
Read out against absolute time like everything else here, so a block
boundary falls in the middle of a packet without disturbing it.
"""
unit = 1.0 / max(1.0, self.baud) # the short pulse
frame = []
for bit in self.payload:
frame += [1] * (3 if bit == "1" else 1)
frame += [0] * (1 if bit == "1" else 3)
gap = [0] * (24 * 4) # silence between repeats
cycle = np.array((frame + gap) * max(1, self.repeats), dtype=np.float64)
pos = np.floor((t % (cycle.size * unit)) / unit).astype(np.int64)
return cycle[np.clip(pos, 0, cycle.size - 1)]
def _advance(self, inst_freq: np.ndarray, fs: float) -> np.ndarray: def _advance(self, inst_freq: np.ndarray, fs: float) -> np.ndarray:
"""Integrate an instantaneous-frequency series, keeping phase continuous.""" """Integrate an instantaneous-frequency series, keeping phase continuous."""
ph = self._phase + np.cumsum(2.0 * np.pi * inst_freq / fs) ph = self._phase + np.cumsum(2.0 * np.pi * inst_freq / fs)
@ -326,11 +373,13 @@ def default_transmitters() -> list[VirtualTransmitter]:
V(97_500_000, "wfm", 0.45, 180_000, "FM broadcast"), V(97_500_000, "wfm", 0.45, 180_000, "FM broadcast"),
V(460_025_000, "fsk4", 0.30, 12_500, "P25-style digital voice", V(460_025_000, "fsk4", 0.30, 12_500, "P25-style digital voice",
baud=4800, deviation=1_800, period_seconds=13, on_seconds=4), baud=4800, deviation=1_800, period_seconds=13, on_seconds=4),
V(929_612_500, "fsk2", 0.28, 12_500, "POCSAG pager", baud=1200, V(929_612_500, "pocsag", 0.28, 12_500, "POCSAG pager", baud=1200,
deviation=2_400, period_seconds=11, on_seconds=2, phase_offset=5), deviation=4_500, period_seconds=11, on_seconds=2.5, phase_offset=5,
pages=((1234568, 3, "ENGINE 4 RESPOND"), (98765, 0, "CALL EXT 4412"))),
V(446_000_000, "carrier", 0.25, 1_000, "unmodulated carrier"), V(446_000_000, "carrier", 0.25, 1_000, "unmodulated carrier"),
V(433_920_000, "ook", 0.30, 40_000, "ISM remote", baud=2000, V(433_920_000, "packet", 0.30, 40_000, "ISM remote", baud=2000,
period_seconds=9, on_seconds=1.2), period_seconds=9, on_seconds=1.2,
payload="101100100011010101001110", repeats=5),
# Always on, because that is the whole character of a control channel # Always on, because that is the whole character of a control channel
# and the reason it needs recognising rather than recording. # and the reason it needs recognising rather than recording.
V(856_562_500, "fsk2", 0.42, 12_500, "SMARTNET control channel", V(856_562_500, "fsk2", 0.42, 12_500, "SMARTNET control channel",

23
bandsaunter/tui.py
View file

@ -559,7 +559,26 @@ It is recognised by its symbol rate and its refusal to pause, named on screen,
and skipped within a second or two. Turn 'Skip trunk control channels' off and skipped within a second or two. Turn 'Skip trunk control channels' off
only if you are collecting them for a decoder. If digital voice calls are only if you are collecting them for a decoder. If digital voice calls are
being skipped by mistake, raise 'Control channel patience'."""), being skipped by mistake, raise 'Control channel patience'."""),
"9": ("Callsigns and the map", """ "9": ("Reading data signals", """
Much of what a scanner finds is not speech: doorbells, tyre-pressure sensors,
weather stations, remote controls, paging, packet radio. All of it is read.
Whatever the modulation, a data signal comes down to the same shape once it has
been sliced — a train of runs whose lengths carry the information — and which
line code it is (PWM, PPM, Manchester, NRZ) is worked out from the runs alone
rather than configured. Four-level FSK, as P25 and DMR use it, is recognised
as such and read as symbols; where a frame sync word appears the system is
named.
POCSAG paging and APRS packet carry their own checksums, so those are read in
full: the address and message text of a page, the callsign and position of an
APRS beacon. A decoded callsign goes onto the map with the rest.
What keeps it honest is repetition. Noise sliced at a threshold produces runs,
and runs produce bits, so a reading with nothing behind it is reported as
nothing at all. These transmitters send the same packet three to ten times
over, and bits that come back identical every time did not come from noise."""),
"10": ("Callsigns and the map", """
Anyone who identifies themselves in a transcript is picked out and looked up in Anyone who identifies themselves in a transcript is picked out and looked up in
the FCC's published licence register: the name, the town, the class of licence. the FCC's published licence register: the name, the town, the class of licence.
The callsign is the only thing sent, and each one is asked about once and then The callsign is the only thing sent, and each one is asked about once and then
@ -575,7 +594,7 @@ can reach.
Turn 'Look callsigns up' off to keep the scan entirely offline; callsigns are Turn 'Look callsigns up' off to keep the scan entirely offline; callsigns are
still found, and named by country from their prefix. Clear 'Map file' to stop still found, and named by country from their prefix. Clear 'Map file' to stop
writing the map. US licence records are public, and include addresses."""), writing the map. US licence records are public, and include addresses."""),
"10": ("Keys during a scan", """ "11": ("Keys during a scan", """
q stop the scan q stop the scan
p pause and resume p pause and resume
s skip the signal being recorded and carry on sweeping s skip the signal being recorded and carry on sweeping

34
bandsaunter/ui.py
View file

@ -12,6 +12,7 @@ from dataclasses import dataclass
import numpy as np import numpy as np
from rich.console import Console, Group from rich.console import Console, Group
from rich.markup import escape
from rich.panel import Panel from rich.panel import Panel
from rich.table import Table from rich.table import Table
from rich.text import Text from rich.text import Text
@ -175,7 +176,13 @@ class ScanDisplay:
self._dirty = True self._dirty = True
def on_status(self, msg: str): def on_status(self, msg: str):
self.messages.appendleft(msg) # Escaped on the way in. A status line can carry text straight off
# the air -- a decoded pager message, a callsign, a Morse decode --
# and rich reads square brackets as markup: "[/x]" in a message is
# not a style, it is an exception in the middle of the display.
# Errors keep their markup, which is written here rather than by
# whatever raised.
self.messages.appendleft(escape(msg))
self._dirty = True self._dirty = True
def on_error(self, exc: Exception): def on_error(self, exc: Exception):
@ -258,7 +265,7 @@ class ScanDisplay:
# and abandoned; saying "REC" while that happens is a lie. # and abandoned; saying "REC" while that happens is a lie.
return Panel( return Panel(
Text.from_markup( Text.from_markup(
f"[bold black on yellow] {r.note} [/bold black on yellow] " f"[bold black on yellow] {escape(r.note)} [/bold black on yellow] "
f"{fmt_hz(r.frequency)}{where} [{r.mode}] " f"{fmt_hz(r.frequency)}{where} [{r.mode}] "
f"SNR {r.snr:5.1f} dB [yellow]skipping[/yellow]"), f"SNR {r.snr:5.1f} dB [yellow]skipping[/yellow]"),
border_style="yellow", padding=(0, 1)) border_style="yellow", padding=(0, 1))
@ -350,7 +357,14 @@ class ScanDisplay:
for h in shown: for h in shown:
extra = "" extra = ""
if h.morse_text: if h.morse_text:
extra = f' [yellow]"{h.morse_text.strip()[:32]}"[/yellow]' extra = f' [yellow]"{escape(h.morse_text.strip()[:32])}"[/yellow]'
elif h.data_messages:
# What was decoded is the most interesting thing about a data
# capture, and far more so than its baud rate.
extra = (" [bright_cyan]"
+ escape(h.data_messages[0][:40]) + "[/bright_cyan]")
elif h.data_encoding:
extra = f" [cyan]{escape(h.data_headline()[:40])}[/cyan]"
elif h.ctcss_hz: elif h.ctcss_hz:
extra = f" [grey62]CTCSS {h.ctcss_hz:.1f}[/grey62]" extra = f" [grey62]CTCSS {h.ctcss_hz:.1f}[/grey62]"
elif h.baud: elif h.baud:
@ -429,10 +443,18 @@ def print_hit(console: Console, hit: HitRecord) -> None:
indent = 26 + _PLAIN_BAND + 2 indent = 26 + _PLAIN_BAND + 2
if hit.morse_text: if hit.morse_text:
console.print(f'{"":>{indent}}[yellow]Morse @ {hit.morse_wpm:.0f} WPM: ' console.print(f'{"":>{indent}}[yellow]Morse @ {hit.morse_wpm:.0f} WPM: '
f'"{hit.morse_text.strip()}"[/yellow]', highlight=False) f'"{escape(hit.morse_text.strip())}"[/yellow]',
if hit.reasons:
console.print(f'{"":>{indent}}[grey54]{hit.reasons[0]}[/grey54]',
highlight=False) highlight=False)
for line in hit.data_messages[:4]:
console.print(f'{"":>{indent}}[bright_cyan]{escape(line)}'
f'[/bright_cyan]', highlight=False)
if hit.data_hex and not hit.data_messages:
console.print(f'{"":>{indent}}[cyan]{escape(hit.data_headline())}'
f'[/cyan] [grey54]{hit.data_hex[:48]}[/grey54]',
highlight=False)
if hit.reasons:
console.print(f'{"":>{indent}}[grey54]{escape(hit.reasons[0])}'
f'[/grey54]', highlight=False)
def print_band_table(console: Console, presets, title: str = "band plan") -> None: def print_band_table(console: Console, presets, title: str = "band plan") -> None:

86
packaging/bandsaunter.1
View file

@ -1,5 +1,5 @@
.\" Generated by packaging/make-man.py -- do not edit by hand. .\" Generated by packaging/make-man.py -- do not edit by hand.
.TH BANDSAUNTER 1 "2026-08-28" "bandsaunter 2026-08-28_01" "User Commands" .TH BANDSAUNTER 1 "2026-08-28" "bandsaunter 2026-08-28_02" "User Commands"
.SH NAME .SH NAME
bandsaunter \- scan, record and identify radio signals with an RTL-SDR bandsaunter \- scan, record and identify radio signals with an RTL-SDR
.SH SYNOPSIS .SH SYNOPSIS
@ -543,6 +543,15 @@ Setting name \fBdecode_morse\fR, default \fByes\fR.
Turn keyed carriers into readable text, with the sending speed. Morse is still in daily use by amateurs and by beacons, and this saves you learning to read it by ear. It costs almost nothing when there is no Morse about. Turn keyed carriers into readable text, with the sending speed. Morse is still in daily use by amateurs and by beacons, and this saves you learning to read it by ear. It costs almost nothing when there is no Morse about.
.RE .RE
.TP .TP
.B --decode-data / --no-decode-data
Decode data signals \[em] read the packets out of anything carrying data.
.br
Setting name \fBdecode_data\fR, default \fByes\fR.
.RS
.PP
Read what a data signal actually says. A great deal of what a scanner finds is not speech: doorbells, tyre-pressure sensors, weather stations, remote controls, paging, packet radio. Each one is sliced into its pulses, the line code worked out from the pulse lengths alone, and the bits reported -- with the packet named where its framing says what it is, and the message printed in full where the protocol carries one. The check that keeps it honest is repetition: these transmitters send the same packet several times over, and bits that come back identical every time did not come from noise. Turn it off to save a little processing on a busy band.
.RE
.TP
.B --log-file .B --log-file
Log file \[em] name of the run log inside the output directory. Log file \[em] name of the run log inside the output directory.
.br .br
@ -934,6 +943,78 @@ than a directory of placeholders.
.BR saunterbrowse (1) .BR saunterbrowse (1)
reads these back, and lists any callsigns it finds in them with the licence reads these back, and lists any callsigns it finds in them with the licence
they belong to. they belong to.
.SH DECODING DATA
A great deal of what a scanner finds is not speech. Doorbells, tyre\-pressure
sensors, weather stations, remote controls, paging and packet radio all carry
words or numbers that a receiver can read, and
.B bandsaunter
reads them.
.PP
Whatever the modulation, a data signal comes down to the same shape once it
has been sliced: a train of alternating runs whose lengths carry the
information. On\-off keying gives that directly \[em] the carrier is up or it is
down \[em] and two\-level FSK gives the same thing from the discriminator, one
tone or the other. So both are reduced to runs and everything after that is
shared.
.PP
What the runs mean is the line code, and it is worked out from the runs alone
rather than being configured:
.TP
.B PWM
The pulse carries the bit and the gap or the period holds still. Nearly every
cheap 433 MHz remote, and everything built on an EV1527 or PT2262.
.TP
.B PPM
The pulse holds still and the gap carries the bit. The other half of the same
market.
.TP
.B Manchester
Every bit is a transition in the middle of its own period, so runs come in
only two lengths.
.TP
.B NRZ
The level is held for as many symbol periods as there are bits. What a framed
protocol sits on top of.
.PP
Four\-level FSK \[em] C4FM, as P25, DMR and NXDN use it \[em] is recognised as
such and read as symbols rather than being sliced down the middle, which would
give bits that mean nothing. Where a frame sync word appears the system is
named outright.
.SH PROTOCOLS THAT CAN BE READ IN FULL
Two carry their own framing and checksums, so a frame either passes or it does
not, and one that passes is not a guess.
.TP
.B POCSAG
Paging, at 512, 1200 or 2400 baud. The rate is not announced anywhere in the
signal, so all three are tried and the one whose sync word appears is the
right one. Each codeword is checked, and a single bit error is corrected,
against the BCH code the standard puts there for the purpose. The address, the
function letter and the message text are all reported.
.TP
.B "AX.25 / APRS"
Amateur packet on 1200 baud AFSK. The frame check has to come out right before
a frame is reported at all. The sender's callsign, the digipeater path and the
payload are shown \[em] and the callsign goes onto the map with the rest.
.SH BELIEVING A DECODE
A decoder that always returns something is worse than useless: noise sliced at
a threshold produces runs, and runs produce bits. Three things guard against
that.
.PP
The runs have to quantise to the line code's own grid, and a decode whose runs
are scattered is thrown away. Most of the bursts in a capture have to decode
the same way, because a data signal is data all the way through and one lucky
window among eight is a coincidence. And, much the strongest, the packet has
to repeat \[em] these transmitters send the same thing three to ten times over,
and bits that come back identical every time did not come from noise.
.PP
A bare reading with none of that behind it, where the runs merely happened to
land on a grid, is reported as nothing at all rather than as a bit string with
a low number beside it that somebody will read anyway.
.PP
A decode that does have repeats or a checksum behind it outranks the content
check: a burst of keying demodulated as FM audio is a buzz, and the speech
detector likes a buzz, but a frame whose own checksum came out right is not a
statistic.
.SH THE MAP .SH THE MAP
A callsign heard in a transcript is looked up in the FCC's published licence A callsign heard in a transcript is looked up in the FCC's published licence
data, which gives the licensee, the town, and coordinates. Those go into a data, which gives the licensee, the town, and coordinates. Those go into a
@ -995,6 +1076,9 @@ Where recordings, transcripts and logs are written, unless
.B \-\-output .B \-\-output
says otherwise. Chosen on first run. says otherwise. Chosen on first run.
.TP .TP
.IR ... _data.txt
What a data capture said, where anything was decoded.
.TP
.I ~/bandsaunter/callsigns.kml .I ~/bandsaunter/callsigns.kml
The map of stations heard, added to as scans run. The map of stations heard, added to as scans run.
.TP .TP

18
packaging/make-browse-man.py
View file

@ -225,6 +225,21 @@ KML is the format Google Earth uses.
.BR marble (1) .BR marble (1)
and OsmAnd open it too, and it is XML, so a scan interrupted halfway through and OsmAnd open it too, and it is XML, so a scan interrupted halfway through
leaves a file that still opens. leaves a file that still opens.
.SH DECODED DATA
Where a capture carried data rather than speech, what was decoded takes the
place of the transcript at the top of the screen: the kind of packet, and then
the message. A pager's text, an APRS position report, or the bits and hex of a
remote control. It is searchable with
.B /
like anything else, so "which page mentioned engine 4" is a question that can
be asked here.
.PP
The text comes from the
.I _data.txt
beside the recording, or from the sidecar where there is none.
.BR bandsaunter (1)
describes how it is decoded and what has to be true before a decode is
believed.
.SH TRANSCRIPTS .SH TRANSCRIPTS
A transcript appears only where a recogniser produced one, which means the A transcript appears only where a recogniser produced one, which means the
capture was judged to be speech and capture was judged to be speech and
@ -252,6 +267,9 @@ Its measurements and identification.
.TP .TP
.IR ... _transcription.txt .IR ... _transcription.txt
What was said, where a recogniser heard speech. What was said, where a recogniser heard speech.
.TP
.IR ... _data.txt
What was decoded, where the capture carried data.
.SH ENVIRONMENT .SH ENVIRONMENT
.TP .TP
.B BANDSAUNTER_OUTPUT .B BANDSAUNTER_OUTPUT

75
packaging/make-man.py
View file

@ -365,6 +365,78 @@ than a directory of placeholders.
.BR saunterbrowse (1) .BR saunterbrowse (1)
reads these back, and lists any callsigns it finds in them with the licence reads these back, and lists any callsigns it finds in them with the licence
they belong to. they belong to.
.SH DECODING DATA
A great deal of what a scanner finds is not speech. Doorbells, tyre\-pressure
sensors, weather stations, remote controls, paging and packet radio all carry
words or numbers that a receiver can read, and
.B bandsaunter
reads them.
.PP
Whatever the modulation, a data signal comes down to the same shape once it
has been sliced: a train of alternating runs whose lengths carry the
information. On\-off keying gives that directly \[em] the carrier is up or it is
down \[em] and two\-level FSK gives the same thing from the discriminator, one
tone or the other. So both are reduced to runs and everything after that is
shared.
.PP
What the runs mean is the line code, and it is worked out from the runs alone
rather than being configured:
.TP
.B PWM
The pulse carries the bit and the gap or the period holds still. Nearly every
cheap 433 MHz remote, and everything built on an EV1527 or PT2262.
.TP
.B PPM
The pulse holds still and the gap carries the bit. The other half of the same
market.
.TP
.B Manchester
Every bit is a transition in the middle of its own period, so runs come in
only two lengths.
.TP
.B NRZ
The level is held for as many symbol periods as there are bits. What a framed
protocol sits on top of.
.PP
Four\-level FSK \[em] C4FM, as P25, DMR and NXDN use it \[em] is recognised as
such and read as symbols rather than being sliced down the middle, which would
give bits that mean nothing. Where a frame sync word appears the system is
named outright.
.SH PROTOCOLS THAT CAN BE READ IN FULL
Two carry their own framing and checksums, so a frame either passes or it does
not, and one that passes is not a guess.
.TP
.B POCSAG
Paging, at 512, 1200 or 2400 baud. The rate is not announced anywhere in the
signal, so all three are tried and the one whose sync word appears is the
right one. Each codeword is checked, and a single bit error is corrected,
against the BCH code the standard puts there for the purpose. The address, the
function letter and the message text are all reported.
.TP
.B "AX.25 / APRS"
Amateur packet on 1200 baud AFSK. The frame check has to come out right before
a frame is reported at all. The sender's callsign, the digipeater path and the
payload are shown \[em] and the callsign goes onto the map with the rest.
.SH BELIEVING A DECODE
A decoder that always returns something is worse than useless: noise sliced at
a threshold produces runs, and runs produce bits. Three things guard against
that.
.PP
The runs have to quantise to the line code's own grid, and a decode whose runs
are scattered is thrown away. Most of the bursts in a capture have to decode
the same way, because a data signal is data all the way through and one lucky
window among eight is a coincidence. And, much the strongest, the packet has
to repeat \[em] these transmitters send the same thing three to ten times over,
and bits that come back identical every time did not come from noise.
.PP
A bare reading with none of that behind it, where the runs merely happened to
land on a grid, is reported as nothing at all rather than as a bit string with
a low number beside it that somebody will read anyway.
.PP
A decode that does have repeats or a checksum behind it outranks the content
check: a burst of keying demodulated as FM audio is a buzz, and the speech
detector likes a buzz, but a frame whose own checksum came out right is not a
statistic.
.SH THE MAP .SH THE MAP
A callsign heard in a transcript is looked up in the FCC's published licence A callsign heard in a transcript is looked up in the FCC's published licence
data, which gives the licensee, the town, and coordinates. Those go into a data, which gives the licensee, the town, and coordinates. Those go into a
@ -426,6 +498,9 @@ Where recordings, transcripts and logs are written, unless
.B \-\-output .B \-\-output
says otherwise. Chosen on first run. says otherwise. Chosen on first run.
.TP .TP
.IR ... _data.txt
What a data capture said, where anything was decoded.
.TP
.I ~/bandsaunter/callsigns.kml .I ~/bandsaunter/callsigns.kml
The map of stations heard, added to as scans run. The map of stations heard, added to as scans run.
.TP .TP

20
packaging/saunterbrowse.1
View file

@ -1,5 +1,5 @@
.\" Generated by packaging/make-browse-man.py -- do not edit by hand. .\" Generated by packaging/make-browse-man.py -- do not edit by hand.
.TH SAUNTERBROWSE 1 "2026-08-28" "bandsaunter 2026-08-28_01" "User Commands" .TH SAUNTERBROWSE 1 "2026-08-28" "bandsaunter 2026-08-28_02" "User Commands"
.SH NAME .SH NAME
saunterbrowse \- read and listen to what a bandsaunter scan collected saunterbrowse \- read and listen to what a bandsaunter scan collected
.SH SYNOPSIS .SH SYNOPSIS
@ -207,6 +207,21 @@ KML is the format Google Earth uses.
.BR marble (1) .BR marble (1)
and OsmAnd open it too, and it is XML, so a scan interrupted halfway through and OsmAnd open it too, and it is XML, so a scan interrupted halfway through
leaves a file that still opens. leaves a file that still opens.
.SH DECODED DATA
Where a capture carried data rather than speech, what was decoded takes the
place of the transcript at the top of the screen: the kind of packet, and then
the message. A pager's text, an APRS position report, or the bits and hex of a
remote control. It is searchable with
.B /
like anything else, so "which page mentioned engine 4" is a question that can
be asked here.
.PP
The text comes from the
.I _data.txt
beside the recording, or from the sidecar where there is none.
.BR bandsaunter (1)
describes how it is decoded and what has to be true before a decode is
believed.
.SH TRANSCRIPTS .SH TRANSCRIPTS
A transcript appears only where a recogniser produced one, which means the A transcript appears only where a recogniser produced one, which means the
capture was judged to be speech and capture was judged to be speech and
@ -234,6 +249,9 @@ Its measurements and identification.
.TP .TP
.IR ... _transcription.txt .IR ... _transcription.txt
What was said, where a recogniser heard speech. What was said, where a recogniser heard speech.
.TP
.IR ... _data.txt
What was decoded, where the capture carried data.
.SH ENVIRONMENT .SH ENVIRONMENT
.TP .TP
.B BANDSAUNTER_OUTPUT .B BANDSAUNTER_OUTPUT

151
tests/signals.py
View file

@ -74,3 +74,154 @@ def make(kind, n=64000, fs=FS, snr_db=30.0, seed=3):
else: else:
raise ValueError(kind) raise ValueError(kind)
return _noise(np.asarray(x, dtype=np.complex128), snr_db, rng) return _noise(np.asarray(x, dtype=np.complex128), snr_db, rng)
# ---------------------------------------------------------------------------
# Packets. What the decoder is for: signals that carry something, rather
# than random keying that only exercises the classifier.
# ---------------------------------------------------------------------------
def _rf(env, fs, snr_db=30.0, seed=1):
"""An envelope on a carrier, with noise."""
x = np.asarray(env, dtype=np.float64).astype(np.complex128)
rng = np.random.default_rng(seed)
p = float(np.mean(np.abs(x) ** 2)) or 1e-9
n = np.sqrt(p / (2 * 10 ** (snr_db / 10.0)))
return (x + n * (rng.standard_normal(x.size)
+ 1j * rng.standard_normal(x.size))).astype(np.complex64)
def ook_pwm(bits, fs=50_000.0, alpha=350e-6, repeats=4, gap=10e-3,
sync=True, snr_db=30.0, seed=1, fixed_gap=None):
"""Pulse-width keying, as an EV1527 or PT2262 remote sends it.
With ``fixed_gap`` the gap is constant and only the pulse varies; without
it the two are complementary and the bit period is constant, which is
what the common parts actually do.
"""
a = int(round(alpha * fs))
env = []
for _ in range(repeats):
if sync:
env += [1] * a + [0] * (31 * a) # the sync pulse and its gap
for b in bits:
if fixed_gap is None:
high, low = (3 * a, a) if b == "1" else (a, 3 * a)
else:
high = 3 * a if b == "1" else a
low = int(round(fixed_gap * fs))
env += [1] * high + [0] * low
env += [0] * int(gap * fs)
return _rf(env, fs, snr_db, seed)
def ook_ppm(bits, fs=50_000.0, alpha=400e-6, repeats=4, gap=10e-3,
snr_db=30.0, seed=2):
"""Pulse-distance keying: the pulse is constant, the gap carries the bit."""
a = int(round(alpha * fs))
env = []
for _ in range(repeats):
for b in bits:
env += [1] * a + [0] * (3 * a if b == "1" else a)
env += [0] * int(gap * fs)
return _rf(env, fs, snr_db, seed)
def ook_manchester(bits, fs=50_000.0, baud=2000.0, repeats=3, gap=10e-3,
snr_db=30.0, seed=3, preamble="10101010"):
half = int(round(fs / baud / 2))
env = []
for _ in range(repeats):
for b in preamble + bits:
first, second = (1, 0) if b == "1" else (0, 1) # IEEE 802.3
env += [first] * half + [second] * half
env += [0] * int(gap * fs)
return _rf(env, fs, snr_db, seed)
def fsk_nrz(bits, fs=48_000.0, baud=1200.0, dev=4500.0, snr_db=30.0, seed=4):
"""Two-level FSK holding each bit for one symbol period."""
sp = fs / baud
n = int(round(len(bits) * sp))
idx = np.clip((np.arange(n) / sp).astype(int), 0, len(bits) - 1)
level = np.array([1.0 if c == "1" else -1.0 for c in bits])[idx]
rng = np.random.default_rng(seed)
x = np.exp(1j * np.cumsum(2 * np.pi * level * dev / fs))
nz = np.sqrt(1.0 / (2 * 10 ** (snr_db / 10.0)))
return (x + nz * (rng.standard_normal(n)
+ 1j * rng.standard_normal(n))).astype(np.complex64)
_C4FM = {"01": 3, "00": 1, "10": -1, "11": -3}
def c4fm(bits, fs=48_000.0, baud=4800.0, dev=1800.0, snr_db=30.0, seed=8):
"""Four-level FSK, mapped the way P25 and DMR map it."""
bits = bits[:len(bits) - len(bits) % 2]
level = np.array([_C4FM[bits[i:i + 2]] / 3.0
for i in range(0, len(bits), 2)])
sp = fs / baud
n = int(level.size * sp)
idx = np.clip((np.arange(n) / sp).astype(int), 0, level.size - 1)
rng = np.random.default_rng(seed)
x = np.exp(1j * np.cumsum(2 * np.pi * level[idx] * dev / fs))
nz = np.sqrt(1.0 / (2 * 10 ** (snr_db / 10.0)))
return (x + nz * (rng.standard_normal(n)
+ 1j * rng.standard_normal(n))).astype(np.complex64)
def ax25_address(call, ssid, last=False):
call = (call.upper() + " ")[:6]
return bytes((ord(c) << 1) & 0xFE for c in call) + \
bytes([0x60 | ((ssid & 0x0F) << 1) | (1 if last else 0)])
def ax25_frame(source, dest, info, path=()):
"""A complete AX.25 frame, with its frame-check sequence."""
body = ax25_address(*dest) + ax25_address(*source, last=not path)
for i, hop in enumerate(path):
body += ax25_address(*hop, last=(i == len(path) - 1))
body += bytes([0x03, 0xF0]) + info.encode()
crc = 0xFFFF
for byte in body:
crc ^= byte
for _ in range(8):
crc = (crc >> 1) ^ 0x8408 if crc & 1 else crc >> 1
crc ^= 0xFFFF
return body + bytes([crc & 0xFF, (crc >> 8) & 0xFF])
def aprs_afsk(frames, fs=48_000.0, baud=1200.0, snr_db=30.0, seed=6,
mark=1200.0, space=2200.0, dev=3000.0, flags=8):
"""AX.25 over Bell 202 AFSK inside an FM carrier, as APRS is sent."""
stream = ""
for frame in frames:
body = "01111110" * flags
ones = 0
for byte in frame:
for k in range(8): # least significant first
bit = (byte >> k) & 1
body += "1" if bit else "0"
if bit:
ones += 1
if ones == 5: # stuff a zero after five
body += "0"
ones = 0
else:
ones = 0
stream += body + "01111110" * flags
level, nrzi = 1, []
for bit in stream:
if bit == "0":
level ^= 1 # NRZI: a zero is a change
nrzi.append(level)
sp = fs / baud
n = int(len(nrzi) * sp)
idx = np.clip((np.arange(n) / sp).astype(int), 0, len(nrzi) - 1)
tone = np.where(np.array(nrzi)[idx] == 1, mark, space)
audio = np.sin(np.cumsum(2 * np.pi * tone / fs))
rng = np.random.default_rng(seed)
x = np.exp(1j * np.cumsum(2 * np.pi * dev * audio / fs))
nz = np.sqrt(1.0 / (2 * 10 ** (snr_db / 10.0)))
return (x + nz * (rng.standard_normal(n)
+ 1j * rng.standard_normal(n))).astype(np.complex64)

656
tests/test_decode.py Normal file
View file

@ -0,0 +1,656 @@
"""Reading the data out of a data signal.
Two things have to be true of a decoder and they pull against each other: it
has to read a real packet correctly, and it has to refuse a signal that is not
a packet. The second is the harder one -- noise sliced at a threshold makes
runs, and runs make bits -- so about half of what is here is signals that must
come back with nothing.
"""
import sys
from pathlib import Path
import numpy as np
import pytest
sys.path.insert(0, str(Path(__file__).parent))
import signals # noqa: E402
from bandsaunter import decode as D # noqa: E402
from bandsaunter.decode import (bits_to_hex, check_crc, decode_data,
decode_four_level, level_count, nrz_bits,
slice_fsk, slice_ook, split_bursts)
from bandsaunter.protocols import (POCSAG_BAUDS, SYNC_BITS, decode_ax25,
decode_pocsag, pocsag_bits,
pocsag_codeword)
PAYLOAD = "101100100011010101001110" # 24 bits, the usual remote
def _random_bits(n, seed=7):
return "".join(np.random.default_rng(seed).choice(list("01"), n))
def _contains(truth: str, got: str, window: int = 120) -> bool:
"""True if a long stretch of the truth is in the decode, either polarity.
Either polarity because nothing in an unframed stream says which level is
a one; a protocol's own sync word settles it, and these have none.
"""
flipped = got.translate(str.maketrans("01", "10"))
middle = truth[len(truth) // 4:len(truth) // 4 + window]
return middle in got or middle in flipped
# ---------------------------------------------------------------------------
# Slicing
# ---------------------------------------------------------------------------
def test_on_off_keying_slices_into_runs():
train = slice_ook(signals.ook_pwm(PAYLOAD), 50_000.0)
assert train is not None
assert len(train) > 50
assert train.contrast_db > 10.0
def test_a_signal_that_is_never_keyed_has_no_runs():
"""A steady carrier is not on-off keyed, whatever a threshold would do."""
assert slice_ook(signals.make("carrier", n=32000), 32_000.0) is None
def test_two_level_fsk_slices_the_same_way_as_keying():
"""The point of the design: after slicing, FSK and OOK are one problem."""
train = slice_fsk(signals.fsk_nrz(_random_bits(400)), 48_000.0,
baud_hint=1200.0)
assert train is not None
assert train.source == "fsk"
assert len(train) > 100
def test_a_glitch_shorter_than_a_symbol_is_absorbed():
"""One sample the wrong side of the threshold must not become two runs."""
levels = np.array([True, False, True, False, True], dtype=bool)
lengths = np.array([40, 1, 39, 80, 40], dtype=np.int64)
out_levels, out_lengths = D._despeckle(levels, lengths, minimum=4)
assert out_lengths.tolist() == [80, 80, 40]
assert out_levels.tolist() == [True, False, True]
def test_bursts_are_cut_at_the_silence_between_repeats():
train = slice_ook(signals.ook_pwm(PAYLOAD, repeats=4), 50_000.0)
bursts = split_bursts(train)
assert len(bursts) == 4
for burst in bursts:
# Trimmed to start and end on a pulse: the silence either side
# belongs to the gap between packets, not to the packet.
assert burst.levels[0] and burst.levels[-1]
def test_a_continuous_stream_is_never_cut_into_bursts():
"""Long runs of one tone are data, not the silence between packets."""
train = slice_fsk(signals.fsk_nrz(_random_bits(600)), 48_000.0,
baud_hint=1200.0)
assert len(split_bursts(train)) == 1
# ---------------------------------------------------------------------------
# The line codes
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("build,encoding,exact", [
(lambda: signals.ook_pwm(PAYLOAD), "PWM", True),
(lambda: signals.ook_pwm(PAYLOAD, fixed_gap=350e-6), "PWM", True),
(lambda: signals.ook_ppm(PAYLOAD), "PPM", False),
(lambda: signals.ook_manchester(PAYLOAD), "Manchester", True),
])
def test_each_line_code_is_recognised_and_read(build, encoding, exact):
got = decode_data(build(), 50_000.0, family="ook")
assert got.ok, got.note
assert got.encoding == encoding
if exact:
assert PAYLOAD in got.bits
else:
# A gap-length code loses its final bit: that gap ran into the
# silence before the next repeat and is no longer separable from it.
assert PAYLOAD[:-1] in got.bits
def test_a_pulse_width_code_keeps_its_last_bit():
got = decode_data(signals.ook_pwm(PAYLOAD), 50_000.0, family="ook")
assert got.bits == PAYLOAD
def test_the_symbol_rate_is_measured_not_guessed():
# 350 us units, four to a bit: 714 bits per second.
got = decode_data(signals.ook_pwm(PAYLOAD), 50_000.0, family="ook")
assert got.baud == pytest.approx(714, rel=0.05)
def test_manchester_reports_the_data_rate_not_the_cell_rate():
"""Two cells go out for every bit; the link is not twice as fast."""
got = decode_data(signals.ook_manchester(PAYLOAD, baud=2000.0),
50_000.0, family="ook")
assert got.baud == pytest.approx(2000, rel=0.06)
@pytest.mark.parametrize("baud,fs", [(512.0, 32_000.0), (1200.0, 48_000.0),
(2400.0, 48_000.0), (4800.0, 96_000.0)])
def test_plain_nrz_comes_back_bit_for_bit(baud, fs):
truth = _random_bits(400, seed=int(baud) % 97)
got = decode_data(signals.fsk_nrz(truth, fs=fs, baud=baud), fs,
family="fsk", baud_hint=baud)
assert got.ok, got.note
assert got.baud == pytest.approx(baud, rel=0.02)
assert _contains(truth, got.bits), "the bits drifted"
def test_a_clock_a_shade_out_does_not_drift_across_a_long_frame():
"""Six hundred bits is where a quarter of a per cent of error shows up."""
truth = _random_bits(600, seed=13)
got = decode_data(signals.fsk_nrz(truth, fs=50_000.0, baud=1200.0),
50_000.0, family="fsk")
assert got.n_bits == pytest.approx(600, abs=2)
assert _contains(truth, got.bits, window=400)
# ---------------------------------------------------------------------------
# Repeats, which are what make a decode believable
# ---------------------------------------------------------------------------
def test_repeats_are_counted_and_have_to_agree():
got = decode_data(signals.ook_pwm(PAYLOAD, repeats=6), 50_000.0,
family="ook")
assert got.repeats >= 4
assert got.agreement > 0.95
assert got.confidence > 0.85
def test_one_lonely_packet_is_believed_less_than_six():
once = decode_data(signals.ook_pwm(PAYLOAD, repeats=1), 50_000.0,
family="ook")
often = decode_data(signals.ook_pwm(PAYLOAD, repeats=6), 50_000.0,
family="ook")
assert often.confidence > once.confidence
def test_a_packet_repeated_with_no_gap_is_still_found():
"""Some transmitters run their repeats together with nothing between."""
bits, repeats = D._repeat_within(PAYLOAD * 4)
assert bits == PAYLOAD and repeats == 4
def test_repeats_that_disagree_are_voted_on():
consensus, certainty = D._agreement(["10110010", "10110010", "10110011"])
assert consensus == "10110010"
assert 0.5 < certainty < 1.0
# ---------------------------------------------------------------------------
# Refusing what is not data
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("kind", ["usb", "lsb", "nfm", "wfm", "am", "noise",
"carrier", "cw", "psk4"])
@pytest.mark.parametrize("seed", [1, 3, 5])
def test_signals_that_are_not_data_decode_to_nothing(kind, seed):
"""The hard half: runs exist in anything, and bits follow from runs."""
x = signals.make(kind, n=64000, fs=32_000.0, snr_db=25.0, seed=seed)
got = decode_data(x, 32_000.0)
assert not got.ok, f"{kind}: invented {got.summary()}"
assert got.note
def test_a_lucky_window_in_speech_is_not_a_packet():
"""One burst in eight fitting a grid is a coincidence, not a signal."""
got = decode_data(signals.make("usb", n=64000, fs=32_000.0), 32_000.0)
assert not got.ok
assert "bursts" in got.note or "frames" in got.note or "nothing" in got.note
def test_a_bare_grid_fit_is_refused_without_framing():
out = D.DataDecode(ok=True, encoding="NRZ", bits="1" * 40, repeats=1)
assert D._refuse(out, fit=0.6, share=1.0)
assert not D._refuse(out, fit=0.95, share=1.0)
def test_repetition_carries_a_decode_that_fit_alone_would_not():
out = D.DataDecode(ok=True, encoding="NRZ", bits="1" * 40, repeats=5,
agreement=1.0)
assert not D._refuse(out, fit=0.2, share=0.1)
# ---------------------------------------------------------------------------
# Four levels
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("kind,levels", [("fsk2", 2), ("fsk4", 4),
("nfm", 1), ("carrier", 1),
("noise", 1)])
def test_the_number_of_levels_is_counted_correctly(kind, levels):
x = signals.make(kind, n=64000, fs=32_000.0, snr_db=25.0)
assert level_count(D._frequency_of(x, 32_000.0)) == levels
def test_a_four_level_signal_is_never_read_as_two():
"""Slicing C4FM down the middle gives bits, and they mean nothing."""
x = signals.make("fsk4", n=64000, fs=32_000.0, snr_db=25.0)
got = decode_data(x, 32_000.0)
assert got.encoding == "4-level FSK"
assert "no frame sync" in got.summary()
assert got.confidence < 0.6
def test_a_frame_sync_word_names_the_system_that_sent_it():
sync = "".join(f"{int(c, 16):04b}" for c in "5575F5FF77FF")
payload = "".join(sync + _random_bits(300, seed=i) for i in range(6))
got = decode_four_level(signals.c4fm(payload), 48_000.0, 4800.0)
assert got is not None and got.ok
assert got.protocol == "P25 Phase 1"
assert got.baud == pytest.approx(4800, rel=0.02)
assert got.confidence > 0.85
def test_an_on_off_keyed_burst_never_takes_the_four_level_path():
"""Discriminator noise in the silences would count as extra levels."""
assert decode_four_level(signals.ook_pwm(PAYLOAD), 50_000.0) is None
# ---------------------------------------------------------------------------
# Checks and rendering
# ---------------------------------------------------------------------------
def test_a_checksum_that_comes_out_right_is_reported():
body = bytes([0x12, 0x34, 0x56])
bits = "".join(f"{b:08b}" for b in body + bytes([sum(body) & 0xFF]))
assert "checksum-8" in check_crc(bits)
def test_a_crc16_that_comes_out_right_is_reported():
from bandsaunter.decode import _crc16_ccitt
body = bytes([0xDE, 0xAD, 0xBE, 0xEF])
crc = _crc16_ccitt(body)
bits = "".join(f"{b:08b}" for b in body + bytes([crc >> 8, crc & 0xFF]))
assert "CRC-16/CCITT" in check_crc(bits)
def test_a_packet_with_no_valid_check_claims_none():
assert check_crc("0" * 32) == [] or "checksum" in check_crc("0" * 32)[0]
@pytest.mark.parametrize("bits,expected", [
("10110010", "B2"), ("1011001000110101", "B2 35"), ("", ""),
("1011", "B0"), # a partial byte, padded on the right
])
def test_bits_render_as_hex(bits, expected):
assert bits_to_hex(bits) == expected
def test_the_summary_says_what_matters_first():
got = decode_data(signals.ook_pwm(PAYLOAD), 50_000.0, family="ook")
assert "EV1527" in got.summary()
assert "24 bits" in got.summary()
def test_a_twenty_four_bit_pulse_width_packet_is_named():
got = decode_data(signals.ook_pwm(PAYLOAD), 50_000.0, family="ook")
assert got.protocol == "EV1527 / PT2262-style remote"
# ---------------------------------------------------------------------------
# POCSAG
# ---------------------------------------------------------------------------
def test_a_pocsag_codeword_checks_out():
word = pocsag_codeword(0x0ABCD)
assert D and word & 1 in (0, 1)
from bandsaunter.protocols import _bch_syndrome, _parity_ok
assert _bch_syndrome(word) == 0 and _parity_ok(word)
def test_a_single_bit_error_in_a_codeword_is_corrected():
from bandsaunter.protocols import _correct
word = pocsag_codeword(0x15555)
broken = word ^ (1 << 17)
fixed, ok = _correct(broken)
assert ok and fixed == word
def test_a_pocsag_transmission_reads_back_as_the_pages_that_went_in():
pages = [(1234568, 3, "ENGINE 4 RESPOND"), (98765, 0, "CALL EXT 4412")]
bits = pocsag_bits(pages)
assert SYNC_BITS in bits
got = decode_pocsag(signals.fsk_nrz(bits, fs=48_000.0, baud=1200.0),
48_000.0)
assert got is not None and got.ok
assert got.protocol == "POCSAG 1200"
assert got.messages[0] == "[1234568D] ENGINE 4 RESPOND"
assert got.messages[1] == "[0098765A] CALL EXT 4412"
assert got.checks == ["BCH(31,21)"]
@pytest.mark.parametrize("baud", POCSAG_BAUDS)
def test_every_pocsag_rate_is_found_without_being_told(baud):
"""Nothing in the signal announces the rate, so all three are tried."""
bits = pocsag_bits([(2097151, 0, "HELLO")]) # the top address
fs = max(32_000.0, baud * 20)
got = decode_pocsag(signals.fsk_nrz(bits, fs=fs, baud=baud), fs)
assert got is not None and got.ok
assert got.protocol == f"POCSAG {baud:.0f}"
assert "HELLO" in got.messages[0]
def test_pocsag_survives_being_received_upside_down():
"""Which tone is a one is a property of the receiver, not the standard."""
bits = pocsag_bits([(1234568, 3, "INVERTED")])
flipped = bits.translate(str.maketrans("01", "10"))
got = decode_pocsag(signals.fsk_nrz(flipped, fs=48_000.0, baud=1200.0),
48_000.0)
assert got is not None and "INVERTED" in got.messages[0]
def test_the_same_page_arriving_twice_is_reported_once():
bits = pocsag_bits([(1234568, 3, "ONCE ONLY")]) * 3
got = decode_pocsag(signals.fsk_nrz(bits, fs=48_000.0, baud=1200.0),
48_000.0)
assert got is not None
assert len(got.messages) == 1
def test_a_numeric_page_is_read_as_digits():
bits = pocsag_bits([(1234568, 0, "5551234")])
got = decode_pocsag(signals.fsk_nrz(bits, fs=48_000.0, baud=1200.0),
48_000.0)
assert got is not None and got.ok
assert "5551234" in got.messages[0]
def test_an_address_that_will_not_fit_is_refused_rather_than_truncated():
"""A silently mangled address is a page delivered to somebody else."""
from bandsaunter.protocols import MAX_ADDRESS
with pytest.raises(ValueError):
pocsag_bits([(MAX_ADDRESS + 1, 0, "NOPE")])
def test_something_that_is_not_pocsag_is_refused():
assert decode_pocsag(signals.fsk_nrz(_random_bits(600), fs=48_000.0),
48_000.0) is None
# ---------------------------------------------------------------------------
# AX.25 and APRS
# ---------------------------------------------------------------------------
def test_an_aprs_frame_reads_back_with_its_callsign_and_payload():
frame = signals.ax25_frame(("W1AW", 0), ("APRS", 0),
"!4142.45N/07243.63W-Newington")
got = decode_ax25(signals.aprs_afsk([frame]), 48_000.0)
assert got is not None and got.ok
assert got.protocol == "AX.25 / APRS"
assert got.messages[0].startswith("W1AW>APRS")
assert "Newington" in got.messages[0]
assert got.checks == ["FCS (CRC-16/X.25)"]
def test_a_digipeater_path_is_kept():
frame = signals.ax25_frame(("KU0W", 9), ("APZ001", 0), ">testing",
path=[("WIDE1", 1), ("WIDE2", 2)])
got = decode_ax25(signals.aprs_afsk([frame]), 48_000.0)
assert got is not None
assert "KU0W-9" in got.messages[0]
assert "WIDE1-1" in got.messages[0] and "WIDE2-2" in got.messages[0]
def test_several_frames_in_one_capture_all_come_back():
frames = [signals.ax25_frame(("W1AW", 0), ("APRS", 0), "first"),
signals.ax25_frame(("KU0W", 0), ("APRS", 0), "second")]
got = decode_ax25(signals.aprs_afsk(frames), 48_000.0)
assert got is not None and len(got.messages) == 2
def test_a_frame_whose_checksum_is_wrong_is_thrown_away():
"""The frame check is the whole reason to believe an AX.25 decode."""
frame = bytearray(signals.ax25_frame(("W1AW", 0), ("APRS", 0), "corrupt"))
frame[-1] ^= 0xFF
got = decode_ax25(signals.aprs_afsk([bytes(frame)]), 48_000.0)
assert got is None
def test_noise_is_not_a_packet_frame():
assert decode_ax25(signals.make("noise", n=64000, fs=48_000.0),
48_000.0) is None
# ---------------------------------------------------------------------------
# Robustness
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("snr_db", [30.0, 20.0, 12.0])
def test_a_remote_still_decodes_as_the_signal_weakens(snr_db):
got = decode_data(signals.ook_pwm(PAYLOAD, snr_db=snr_db, repeats=6),
50_000.0, family="ook")
assert got.ok, f"{snr_db} dB: {got.note}"
assert PAYLOAD in got.bits
@pytest.mark.parametrize("snr_db", [30.0, 18.0])
def test_paging_still_decodes_as_the_signal_weakens(snr_db):
bits = pocsag_bits([(1234568, 3, "WEAK SIGNAL")])
got = decode_pocsag(signals.fsk_nrz(bits, fs=48_000.0, baud=1200.0,
snr_db=snr_db), 48_000.0)
assert got is not None and "WEAK" in got.messages[0]
def test_a_capture_too_short_to_hold_a_packet_says_so():
got = decode_data(np.zeros(100, dtype=np.complex64), 48_000.0)
assert not got.ok and "short" in got.note
def test_nrz_bits_refuses_a_rate_it_cannot_resolve():
"""Fewer than two samples a symbol is not a sampling problem to solve."""
train = slice_ook(signals.ook_pwm(PAYLOAD), 50_000.0)
assert nrz_bits(train, 40_000.0) == ""
assert nrz_bits(train, 0.0) == ""
# ---------------------------------------------------------------------------
# A scan that finds one
# ---------------------------------------------------------------------------
def _scan(tmp_path, transmitters, **over):
from bandsaunter.config import ScanConfig
from bandsaunter.ranges import parse_range_list
from bandsaunter.scanner import Scanner, ScannerCallbacks
from bandsaunter.simulator import SimulatedDevice
cfg = ScanConfig(ranges=parse_range_list(over.pop("ranges", "433.9M-433.95M")),
output_dir=str(tmp_path), record_seconds=3.0,
hang_seconds=1.0, threshold_db=12, dwell_seconds=0.05,
max_cycles=2, revisit_seconds=0.2)
on_status = over.pop("_on_status", None)
for key, value in over.items():
setattr(cfg, key, value)
hits = []
scanner = Scanner(cfg, device=SimulatedDevice(
transmitters=transmitters).open(),
callbacks=ScannerCallbacks(on_record_end=hits.append,
on_status=on_status))
scanner.prepare()
scanner.run()
return scanner, [h for h in hits if h.kept]
def _remote():
from bandsaunter.simulator import VirtualTransmitter as V
return [V(433_920_000, "packet", 0.45, 40_000, "remote", baud=2000,
payload=PAYLOAD, repeats=6)]
def _pager():
from bandsaunter.simulator import VirtualTransmitter as V
return [V(929_612_500, "pocsag", 0.45, 12_500, "pager", baud=1200,
deviation=4_500,
pages=((1234568, 3, "ENGINE 4 RESPOND"),))]
def test_a_scan_reads_the_packet_off_a_remote(tmp_path):
_, hits = _scan(tmp_path, _remote())
assert hits, "the remote was never captured"
assert all(h.data_encoding == "PWM" for h in hits)
read = [h for h in hits if h.data_bits == PAYLOAD]
assert read, [h.data_bits for h in hits]
assert read[0].data_repeats > 1
assert read[0].classification == "EV1527 / PT2262-style remote"
def test_what_was_decoded_is_written_beside_the_recording(tmp_path):
_, hits = _scan(tmp_path, _remote())
assert hits
written = list(tmp_path.glob("*_data.txt"))
assert written, "nothing was written"
bodies = [path.read_text() for path in written]
assert any(PAYLOAD in body for body in bodies), bodies
assert any("EV1527" in body for body in bodies)
# and every record points at the file it wrote
for hit in hits:
assert Path(hit.data_path).exists()
def test_a_scan_reads_a_page_and_prints_the_message(tmp_path):
_, hits = _scan(tmp_path, _pager(), ranges="929.55M-929.7M",
record_seconds=4.0)
assert hits, "the pager was never captured"
hit = hits[0]
assert hit.data_protocol == "POCSAG 1200"
assert "ENGINE 4 RESPOND" in hit.data_messages[0]
assert hit.classification == "POCSAG 1200"
assert "BCH(31,21)" in hit.data_checks
def test_the_display_is_told_what_was_decoded(tmp_path):
said = []
_scan(tmp_path, _remote(), _on_status=said.append)
assert any("decoded" in m and PAYLOAD[:8] not in m for m in said), said
def test_decoding_can_be_turned_off(tmp_path):
_, hits = _scan(tmp_path, _remote(), decode_data=False)
assert hits
assert not hits[0].data_encoding
assert not list(tmp_path.glob("*_data.txt"))
def test_no_data_file_is_left_beside_a_recording_that_was_thrown_away(tmp_path):
"""The capture is renamed after it is kept, and orphans confuse a reader."""
_, hits = _scan(tmp_path, _remote())
for path in tmp_path.glob("*_data.txt"):
stem = path.name[:-len("_data.txt")]
assert (tmp_path / f"{stem}.wav").exists(), f"orphan: {path.name}"
def test_a_decoded_packet_is_kept_even_when_the_content_check_says_no(tmp_path):
"""A frame whose own checksum came out right is not a statistic."""
_, hits = _scan(tmp_path, _pager(), ranges="929.55M-929.7M",
record_seconds=4.0, accept=["voice"])
assert hits, "a decoded page was discarded as contentless"
assert hits[0].category == "digital"
def test_the_browser_shows_what_was_decoded(tmp_path):
from rich.console import Console
from bandsaunter.browse import Browser, Player
_scan(tmp_path, _pager(), ranges="929.55M-929.7M", record_seconds=4.0)
console = Console(width=100, height=30, force_terminal=True)
browser = Browser(tmp_path, console=console, player=Player([]))
assert browser.captures
cap = browser.captures[0]
assert cap.decoded and "ENGINE 4 RESPOND" in cap.decoded[0]
assert "POCSAG" in cap.data_headline
with console.capture() as frame:
console.print(browser.render())
text = frame.get()
assert "ENGINE 4 RESPOND" in text
assert "decoded" in text
def test_a_pager_message_can_be_searched_for(tmp_path):
from rich.console import Console
from bandsaunter.browse import Browser, Player
_scan(tmp_path, _pager(), ranges="929.55M-929.7M", record_seconds=4.0)
browser = Browser(tmp_path,
console=Console(width=100, height=30,
force_terminal=True),
player=Player([]))
browser.query = "engine 4"
browser.apply()
assert browser.view, "searching what a data capture said found nothing"
# ---------------------------------------------------------------------------
# Text that came off the air
# ---------------------------------------------------------------------------
HOSTILE = "[1234568D] ALERT [/red] see [bold] the thing"
def test_a_decoded_message_cannot_break_the_live_display():
"""Rich reads square brackets as markup, and a page is arbitrary text."""
import os
import tempfile
import time
from rich.console import Console
from bandsaunter.config import ScanConfig
from bandsaunter.ranges import parse_range_list
from bandsaunter.recorder import HitRecord
from bandsaunter.scanner import Scanner
from bandsaunter.simulator import SimulatedDevice
from bandsaunter.ui import ScanDisplay
console = Console(width=120, height=30, record=True,
file=open(os.devnull, "w"))
cfg = ScanConfig(ranges=parse_range_list("144M-148M"),
output_dir=tempfile.mkdtemp())
scanner = Scanner(cfg, device=SimulatedDevice().open())
scanner.prepare()
display = ScanDisplay(scanner, console=console)
hit = HitRecord(frequency=929.6e6, started_at=time.time(), duration=4.5,
snr_db=49.5, classification="POCSAG 1200")
hit.data_messages = [HOSTILE]
hit.data_protocol = "POCSAG 1200"
display.hits.appendleft(hit)
display.on_status(f"decoded 929.6 MHz: {HOSTILE}")
console.print(display._hits_table())
console.print(display._footer())
text = console.export_text()
assert "ALERT" in text
def test_a_decoded_message_cannot_break_the_line_per_hit_output():
import os
from rich.console import Console
from bandsaunter.recorder import HitRecord
from bandsaunter.ui import print_hit
console = Console(width=140, record=True, file=open(os.devnull, "w"))
hit = HitRecord(frequency=929.6e6, started_at=0, duration=4.5,
snr_db=49.5, classification="POCSAG 1200")
hit.kept = True
hit.data_messages = [HOSTILE]
print_hit(console, hit)
assert "ALERT" in console.export_text()
@pytest.mark.parametrize("typed", ["[/x]", "[bold", "]]]", "[/]"])
def test_what_is_typed_at_the_search_prompt_cannot_break_the_browser(typed,
tmp_path):
"""Typing "[/" used to end the session with a MarkupError."""
from rich.console import Console
from bandsaunter.browse import Browser, Player
browser = Browser(tmp_path,
console=Console(width=100, height=30,
force_terminal=True),
player=Player([]))
browser.searching = True
browser.query = typed
browser._footer()
browser.searching = False
browser.message = f"nothing matches {typed}"
browser._footer()