diff --git a/README.md b/README.md
index 8f3ce0d..e227d4c 100644
--- a/README.md
+++ b/README.md
@@ -2630,6 +2630,93 @@ dongles ship with; the stock telescopic one extended properly does well.
`--packets` shows each frame as it arrives, which is the thing to watch while
moving an aerial about.
+## FT8
+
+Fifteen seconds of everybody at once.
+
+```
+bandsaunter ft8 --band 20m --grid IO91
+```
+
+Every station on the band transmits in the same quarter-minute slots, on the
+same dial frequency, fifty hertz wide each, stacked across three kilohertz of
+audio. So one receiver parked on one frequency hears the whole band's worth
+of stations simultaneously — and hears most of them **well below the noise**.
+
+```
+FT8 20m — 14.074 MHz 12 slots 187 decodes 63 stations 15.6 a slot 3:00 from IO91
+
+last slot — 16 decoded
+snr dt hz message
+-19 -0.5 309 G4CUS SP4FCA RRR
+ -6 0.1 528 VK3EVE SQ3MZM RR73
+ 18 0.1 1110 9A9TT IK4LZH -10
+ 6 0.0 2535 CQ IZ3XJM JN55
+
+heard so far — 63 stations
+station grid away n best hz last
+IZ3XJM JN55 1180 km 4 6 2535 2s
+IK4LZH JN54 1140 km 7 18 1110 2s
+```
+
+Half of what is transmitted is error-correcting code, and that is the whole
+trick: it is what buys a mode that decodes twenty-odd decibels under what an
+operator can hear. A receiver that took the loudest tone of each symbol and
+hoped would decode almost nothing.
+
+### What it needs
+
+**The clock has to be right**, to a second or two. The slots are
+quarter-minutes of UTC and every station on earth agrees about which one it
+is. A receiver a second out still decodes; one a slot out hears every
+transmission split across two captures and decodes none of them. This is the
+one failure that looks exactly like a dead band, so the display and the
+report both say so when nothing arrives.
+
+**Almost all the activity is on shortwave**, which a plain dongle cannot
+reach. The default here is therefore the two-metre channel at 144.174 MHz,
+which it can. All thirteen channels are in the list and the shortwave ones
+work perfectly well through an upconverter or a receiver in direct-sampling
+mode — the menu says which is which rather than leaving you to find out by
+listening to silence.
+
+### What comes out
+
+`--grid IO91` is what turns decodes into geography: every station calling CQ
+says where it is, so with your own square filled in the report gives each one
+a distance and a bearing, and names the furthest heard. On shortwave that is
+the entire interest of the thing.
+
+Three tables afterwards: **stations heard** with grid, distance, count and
+best and worst report; **calling CQ**, which is who is available; and **who
+was working whom**, which is the band as a social event rather than a list.
+
+`--adif` writes the log again in the form every amateur logging program
+imports — marked as *heard*, not worked. Nothing here transmits, so nothing
+here is a contact, and an ADIF that let a logging program treat these as
+worked would put claims into somebody's log they cannot make.
+
+### How well it works
+
+Checked against eleven off-air recordings with published decodes, which is
+the only honest way to test a decoder — an encoder tested against its own
+decoder agrees with it about anything they are both wrong about, a lesson
+this project learned expensively on 433 MHz.
+
+| | |
+|---|---|
+| messages decoded | 97 of 150 (65%), **no false decodes** |
+| timing | +0.00 s, spread 0.04 |
+| frequency | +0.1 Hz, spread 1.1 |
+| signal report | −0.4 dB, spread 4.8 |
+| weakest decoded | −24 dB on this program's own scale |
+
+The 35% not decoded are the weakest signals in each slot. A mature decoder
+subtracts each signal it decodes and looks again in what is left, and does
+ordered-statistics decoding when belief propagation fails; neither is built
+here. What *is* here decodes nothing that other receivers did not also hear,
+which is the property that matters in a log.
+
## Meters on 900 MHz
A scan of 902–928 MHz that turns up a burst gets it named rather than reported
@@ -3439,3 +3526,21 @@ by the Free Software Foundation. It is distributed in the hope that it will be
useful, but with no warranty whatsoever — not even the implied warranty of
merchantability or fitness for a particular purpose. The full text is in
[LICENSE](LICENSE), and at .
+
+### Borrowed material
+
+One file is not original work. `bandsaunter/ft8tables.py` holds the two fixed
+tables that *define* the FT8 error-correcting code — the generator and the
+sparse parity-check matrix — taken from
+[ft8_lib](https://github.com/kgoba/ft8_lib), MIT licensed, copyright © 2018
+Kārlis Goba, which took them in turn from WSJT-X. They are reproduced under
+the MIT terms, which permit it, and that file carries the attribution they
+ask for.
+
+They are there because they cannot be derived. Everything else about FT8 in
+this program is worked out from first principles — the tones, the sync, the
+parity arithmetic, the belief propagation, the way a callsign is squeezed
+into twenty-eight bits — but those two tables are not derived from anything.
+They are the code itself, chosen once by its designers and published, and a
+receiver that guessed at them would be speaking a different protocol. No
+decoding logic was taken.
diff --git a/bandsaunter/__init__.py b/bandsaunter/__init__.py
index 612d1e2..5896340 100755
--- a/bandsaunter/__init__.py
+++ b/bandsaunter/__init__.py
@@ -9,7 +9,7 @@ and transcribing speech.
# 2026-08-21_02 is the second build made on the 21st. The revision is padded
# to two digits so versions sort as text.
VERSION_DATE = "2026-09-21"
-VERSION_REVISION = 3
+VERSION_REVISION = 4
__version__ = f"{VERSION_DATE}_{VERSION_REVISION:02d}"
diff --git a/bandsaunter/cli.py b/bandsaunter/cli.py
index 2f72f5a..79cd337 100755
--- a/bandsaunter/cli.py
+++ b/bandsaunter/cli.py
@@ -26,6 +26,7 @@ from .device import RtlSdrError, list_devices, set_driver_messages
from .librtlsdr import load_error
from . import settings as st
from .ax25 import APRS_CHANNELS as _APRS_CHANNELS
+from .ft8 import BANDS as _FT8_BANDS
from .ranges import (RangeError, ScanRange, build_plan, parse_range_list)
from .scanner import Scanner, ScannerCallbacks
from .tui import TUIAbort, first_run_setup, run_tui, settings_menu
@@ -639,6 +640,74 @@ examples:
a.add_argument("--freq", type=float, default=0.0,
help="centre frequency in Hz, for band-aware naming")
a.add_argument("--morse", action="store_true", help="force a CW decode")
+
+ # -- ft8 ------------------------------------------------------------------
+ f8 = sub.add_parser("ft8",
+ help="listen to FT8: fifteen-second slots, forty "
+ "stations at once, most of them under the noise")
+ f8.add_argument("--device", type=int, default=None, help="which receiver")
+ f8.add_argument("--gain", default=None, help="tuner gain in dB, or auto")
+ f8.add_argument("--rate", type=float, default=None,
+ help="sample rate in Hz; 96 kS/s is the least that holds "
+ "three kilohertz of audio")
+ f8.add_argument("--band", default=None,
+ choices=[key for key, _hz, _n in _FT8_BANDS],
+ help="which FT8 channel to listen on; sets the frequency")
+ f8.add_argument("--frequency", "--freq", dest="frequency", type=float,
+ default=None, metavar="HZ",
+ help="the dial frequency, for an upconverter or a "
+ "channel the band list does not have")
+ f8.add_argument("--simulate", dest="simulate", action="store_true",
+ default=None, help="invent a band instead of using a "
+ "receiver")
+ f8.add_argument("--no-simulate", dest="simulate", action="store_false",
+ help="use the receiver (the default)")
+ f8.add_argument("--seconds", type=float, default=None,
+ help="stop after this long (default: until interrupted)")
+ f8.add_argument("--slots", type=int, default=None, metavar="N",
+ help="stop after this many fifteen-second slots")
+ f8.add_argument("--log", dest="log", action="store_true", default=None,
+ help="write a log of every decode (the default)")
+ f8.add_argument("--no-log", dest="log", action="store_false",
+ help="do not write a log")
+ f8.add_argument("--decodes-seen", dest="decodes_seen",
+ action="store_true", default=None,
+ help="print a line per decode instead of a live table")
+ f8.add_argument("--no-decodes-seen", dest="decodes_seen",
+ action="store_false", help="show the live table")
+ f8.add_argument("--hold", type=float, default=None, metavar="SECONDS",
+ help="how long a station stays on the display")
+ f8.add_argument("--lowest", type=float, default=None, metavar="HZ",
+ help="the lowest audio frequency to search")
+ f8.add_argument("--highest", type=float, default=None, metavar="HZ",
+ help="the highest audio frequency to search")
+ f8.add_argument("--most", type=int, default=None, metavar="N",
+ help="how many candidate transmissions to try per slot")
+ f8.add_argument("--rounds", type=int, default=None, metavar="N",
+ help="how many error-correction passes before giving up")
+ f8.add_argument("--grid", default=None, metavar="SQUARE",
+ help="your own grid square, so distances can be worked "
+ "out (four characters, like IO91)")
+ f8.add_argument("--units", default=None, choices=("metric", "imperial"),
+ help="which units to show distances in")
+ f8.add_argument("--calls-only", dest="calls_only", action="store_true",
+ default=None, help="leave out free text and telemetry")
+ f8.add_argument("--no-calls-only", dest="calls_only",
+ action="store_false", help="show everything decoded")
+ f8.add_argument("--report", dest="report", action="store_true",
+ default=None, help="print the tables at the end")
+ f8.add_argument("--no-report", dest="report", action="store_false",
+ help="do not print the tables")
+ f8.add_argument("--csv", dest="csv", action="store_true", default=None,
+ help="also write a spreadsheet of every decode")
+ f8.add_argument("--no-csv", dest="csv", action="store_false",
+ help="do not write a spreadsheet")
+ f8.add_argument("--adif", dest="adif", action="store_true", default=None,
+ help="also write an ADIF of stations heard, for a "
+ "logging program")
+ f8.add_argument("--no-adif", dest="adif", action="store_false",
+ help="do not write an ADIF")
+
return p
@@ -2151,6 +2220,7 @@ def main(argv=None) -> int:
"flights": cmd_flights, "weather": cmd_weather,
"readings": cmd_readings, "sensors": cmd_sensors,
"aprs": cmd_aprs, "packets": cmd_packets,
+ "ft8": cmd_ft8,
}
try:
return handlers[args.command](args)
@@ -2161,5 +2231,42 @@ def main(argv=None) -> int:
return 0
+def cmd_ft8(args) -> int:
+ """Park on an FT8 channel and decode every slot.
+
+ Fifteen seconds of everybody at once: one dial frequency carries the
+ whole band's worth of stations, fifty hertz apart across three
+ kilohertz of audio, and most of them arrive below the noise.
+ """
+ from . import ft8
+
+ cfg, _ = load_default()
+ options = ft8.load_options()
+ for flag, key in (("device", "device"), ("gain", "gain"),
+ ("rate", "rate"), ("band", "band"),
+ ("frequency", "frequency"), ("simulate", "simulate"),
+ ("seconds", "seconds"), ("slots", "slots"),
+ ("log", "log"), ("decodes_seen", "decodes_seen"),
+ ("hold", "hold"), ("lowest", "lowest"),
+ ("highest", "highest"), ("most", "most"),
+ ("rounds", "rounds"), ("grid", "grid"),
+ ("units", "units"), ("calls_only", "calls_only"),
+ ("report", "report"), ("csv", "csv"),
+ ("adif", "adif")):
+ value = getattr(args, flag, None)
+ if value is not None:
+ setattr(options, key, value)
+ # The band picks the frequency unless the frequency was given outright,
+ # which is the one order that lets both flags mean what they say.
+ if getattr(args, "band", None) and getattr(args, "frequency", None) is None:
+ ft8.use_band(options, options.band)
+ errs = options.validate()
+ if errs:
+ for e in errs:
+ console.print(f"[red]{e}[/red]")
+ return 2
+ ft8.listen(console, options, cfg.output_dir)
+ return 0
+
if __name__ == "__main__":
sys.exit(main())
diff --git a/bandsaunter/ft8.py b/bandsaunter/ft8.py
new file mode 100644
index 0000000..a52fabe
--- /dev/null
+++ b/bandsaunter/ft8.py
@@ -0,0 +1,919 @@
+"""Listening to FT8, from either front end.
+
+Fifteen seconds of everybody at once. Every station on the band transmits
+in the same quarter-minute slots, on the same dial frequency, fifty hertz
+wide each, stacked across three kilohertz of audio -- so one receiver
+parked on one frequency hears the whole band's worth of stations at the
+same time, and hears most of them well below the noise.
+
+That is why this is a section rather than something the scanner stops on.
+A sweep catches whatever was keyed up as it went past; FT8 is forty
+stations transmitting simultaneously for twelve and a half seconds out of
+every fifteen, and the interesting thing is all of them.
+
+The band plan is mostly shortwave, which a plain receiver of this kind
+cannot reach without help -- so the default here is the two-metre channel
+at 144.174 MHz, which it can. The shortwave channels are all in the list
+and work perfectly well through an upconverter or a receiver in direct
+sampling mode; they are simply not what you get by not choosing.
+
+Nothing here transmits. This listens, decodes and writes down.
+"""
+
+from __future__ import annotations
+
+import math
+import time
+from dataclasses import asdict, dataclass, field
+from datetime import datetime, timezone
+from pathlib import Path
+
+import numpy as np
+import yaml
+
+from . import ft8code as code
+from . import ft8wave as wave
+from .settings import Setting, format_value
+
+__all__ = ["Ft8Options", "OPTIONS", "OPTION_GROUPS", "defaults", "in_group",
+ "by_key", "format_option", "describe", "summarise",
+ "Station", "Band", "Heard", "listen", "open_device", "open_log",
+ "pump", "finish", "report", "load_options", "save_options",
+ "options_path", "logs_in", "BANDS", "band_named", "band_text",
+ "use_band", "slot_start", "next_slot", "grid_at", "grid_away",
+ "SLOT", "audio_from"]
+
+
+# ---------------------------------------------------------------------------
+# Where FT8 lives
+# ---------------------------------------------------------------------------
+
+# name, hertz, what it needs. The dial frequency, in the usual convention:
+# signals sit from about 200 to 3000 Hz above it.
+BANDS: tuple[tuple[str, float, str], ...] = (
+ ("160m", 1_840_000.0, "shortwave"),
+ ("80m", 3_573_000.0, "shortwave"),
+ ("60m", 5_357_000.0, "shortwave"),
+ ("40m", 7_074_000.0, "shortwave"),
+ ("30m", 10_136_000.0, "shortwave"),
+ ("20m", 14_074_000.0, "shortwave"),
+ ("17m", 18_100_000.0, "shortwave"),
+ ("15m", 21_074_000.0, "shortwave"),
+ ("12m", 24_915_000.0, "shortwave"),
+ ("10m", 28_074_000.0, "shortwave"),
+ ("6m", 50_313_000.0, "direct"),
+ ("2m", 144_174_000.0, "direct"),
+ ("70cm", 432_174_000.0, "direct"),
+)
+
+SLOT = code.SLOT_S # fifteen seconds, and everybody keeps to it
+
+
+def band_named(name: str) -> float:
+ """The dial frequency for a band, or the two-metre one."""
+ wanted = (name or "").strip().lower()
+ for key, hz, _needs in BANDS:
+ if wanted == key:
+ return hz
+ return 144_174_000.0
+
+
+def band_needs(name: str) -> str:
+ wanted = (name or "").strip().lower()
+ for key, _hz, needs in BANDS:
+ if wanted == key:
+ return needs
+ return "direct"
+
+
+def use_band(options: "Ft8Options", name: str) -> None:
+ """Point the receiver at a band, frequency and all."""
+ options.band = (name or "").strip().lower()
+ options.frequency = band_named(options.band)
+
+
+def band_text(options: "Ft8Options") -> str:
+ where = f"{options.frequency / 1e6:g} MHz"
+ needs = band_needs(options.band)
+ if needs == "shortwave":
+ return f"{options.band} — {where}, needs an upconverter"
+ return f"{options.band} — {where}"
+
+
+# ---------------------------------------------------------------------------
+# Slots
+# ---------------------------------------------------------------------------
+
+def slot_start(when: float | None = None) -> float:
+ """The top of the fifteen-second slot that ``when`` falls in.
+
+ Against UTC, not against the local clock and not against when this
+ program happened to start: the whole protocol is built on every station
+ on earth agreeing about which quarter-minute it is. A receiver whose
+ clock is a second out still decodes -- the search covers a few seconds
+ either way -- but one that is a slot out hears every transmission
+ split across two captures and decodes none of them.
+ """
+ when = time.time() if when is None else when
+ return math.floor(when / SLOT) * SLOT
+
+
+def next_slot(when: float | None = None) -> float:
+ return slot_start(when) + SLOT
+
+
+def slot_name(at: float) -> str:
+ """The slot, as the six digits every FT8 program labels it with."""
+ return datetime.fromtimestamp(at, timezone.utc).strftime("%H%M%S")
+
+
+# ---------------------------------------------------------------------------
+# Grid squares
+# ---------------------------------------------------------------------------
+
+def grid_at(grid: str):
+ """The middle of a Maidenhead square, as latitude and longitude.
+
+ Four characters is about seventy miles by a hundred, which is all the
+ position FT8 carries and quite enough to say which continent somebody
+ is on. Six characters is finer and turns up in free text rather than
+ in the position field.
+ """
+ grid = (grid or "").strip().upper()
+ if len(grid) < 4 or not grid[:2].isalpha() or not grid[2:4].isdigit():
+ return None
+ if not ("A" <= grid[0] <= "R" and "A" <= grid[1] <= "R"):
+ return None
+ lon = (ord(grid[0]) - 65) * 20.0 + int(grid[2]) * 2.0
+ lat = (ord(grid[1]) - 65) * 10.0 + int(grid[3]) * 1.0
+ if len(grid) >= 6 and grid[4].isalpha() and grid[5].isalpha():
+ lon += (ord(grid[4]) - 65) * (2.0 / 24.0) + (1.0 / 24.0)
+ lat += (ord(grid[5]) - 65) * (1.0 / 24.0) + (0.5 / 24.0)
+ else:
+ lon += 1.0
+ lat += 0.5
+ return lat - 90.0, lon - 180.0
+
+
+def grid_away(here: str, there: str):
+ """How far apart two grid squares are, in kilometres, and on what bearing.
+
+ Great-circle, from the middles of the squares -- so it is good to the
+ size of a square and no better, which for FT8 is the honest answer and
+ is why nothing here prints a distance to the nearest kilometre.
+ """
+ a, b = grid_at(here), grid_at(there)
+ if a is None or b is None:
+ return None
+ lat1, lon1 = math.radians(a[0]), math.radians(a[1])
+ lat2, lon2 = math.radians(b[0]), math.radians(b[1])
+ d = 2 * math.asin(math.sqrt(
+ math.sin((lat2 - lat1) / 2) ** 2
+ + math.cos(lat1) * math.cos(lat2) * math.sin((lon2 - lon1) / 2) ** 2))
+ y = math.sin(lon2 - lon1) * math.cos(lat2)
+ x = (math.cos(lat1) * math.sin(lat2)
+ - math.sin(lat1) * math.cos(lat2) * math.cos(lon2 - lon1))
+ return 6371.0088 * d, (math.degrees(math.atan2(y, x)) + 360.0) % 360.0
+
+
+# ---------------------------------------------------------------------------
+# The options
+# ---------------------------------------------------------------------------
+
+@dataclass
+class Ft8Options:
+ """Everything the FT8 mode can be told, in one place."""
+
+ # -- receiver -------------------------------------------------------
+ device: int = 0
+ gain: str = "auto"
+ rate: float = 240_000.0
+ band: str = "2m"
+ frequency: float = 144_174_000.0
+ simulate: bool = False
+
+ # -- listening ------------------------------------------------------
+ seconds: float = 0.0
+ slots: int = 0 # or stop after this many slots
+ log: bool = True
+ decodes_seen: bool = False # a line per decode rather than a table
+ hold: float = 3600.0
+
+ # -- decoding -------------------------------------------------------
+ lowest: float = 200.0
+ highest: float = 3000.0
+ most: int = 300 # candidates examined per slot
+ rounds: int = 30 # how hard to try to repair one
+
+ # -- showing --------------------------------------------------------
+ grid: str = "" # where the aerial is, as a grid square
+ units: str = "metric"
+ calls_only: bool = False # leave out free text and telemetry
+
+ # -- afterwards -----------------------------------------------------
+ report: bool = True
+ csv: bool = False
+ adif: bool = False
+
+ @property
+ def imperial(self) -> bool:
+ return str(self.units).lower().startswith("imp")
+
+ def validate(self) -> list[str]:
+ out = []
+ if self.rate < 96_000:
+ out.append("the sample rate must be at least 96 kS/s to hold "
+ "three kilohertz of audio with room to filter it")
+ if self.seconds < 0 or self.slots < 0:
+ out.append("times cannot be negative")
+ if self.hold <= 0:
+ out.append("a station must stay on the display for some time")
+ if not 1e5 < self.frequency < 2e9:
+ out.append("that is not a frequency a receiver can be tuned to")
+ if self.highest <= self.lowest:
+ out.append("the top of the search must be above the bottom")
+ if self.highest - self.lowest < 100.0:
+ out.append("a search narrower than 100 Hz would miss almost "
+ "everything: one signal is 50 Hz wide and they are "
+ "spread across three kilohertz")
+ if self.most < 1:
+ out.append("at least one candidate has to be looked at")
+ if self.rounds < 1:
+ out.append("the error correction needs at least one pass")
+ if self.grid and grid_at(self.grid) is None:
+ out.append(f"{self.grid!r} is not a grid square — four "
+ "characters like IO91 or FN31")
+ return out
+
+ def to_dict(self) -> dict:
+ return asdict(self)
+
+
+def defaults() -> Ft8Options:
+ return Ft8Options()
+
+
+O = Setting
+_BANDS = tuple(key for key, _hz, _needs in BANDS)
+
+OPTIONS: tuple[Setting, ...] = (
+ # -- receiver -------------------------------------------------------
+ O("device", "Receiver", "Receiver", "int",
+ "which receiver to use, when more than one is plugged in",
+ "The index shown by `bandsaunter devices`. Zero unless you have "
+ "several dongles.",
+ minimum=0, flags=("--device",), example="0"),
+ O("gain", "Gain", "Receiver", "gain",
+ "tuner gain in dB, or automatic",
+ "FT8 is forty signals at once spanning fifty decibels of strength, "
+ "and what matters is that none of them clips rather than that any of "
+ "them is loud. Automatic copes. A fixed gain makes the signal "
+ "reports comparable between one evening and the next, which is the "
+ "whole point of them if you are judging an aerial or a band opening.",
+ flags=("--gain",), example="auto"),
+ O("rate", "Sample rate", "Receiver", "float",
+ "how fast to sample; 96 kS/s is the least that holds the channel",
+ "Only three kilohertz of it is used, so this is about having room to "
+ "filter cleanly rather than about bandwidth. The default costs "
+ "little and decimates to the twelve kilohertz the decoding wants by "
+ "a whole number.",
+ unit="Hz", minimum=96_000.0, flags=("--rate",), example="240000"),
+ O("band", "Band", "Receiver", "choice",
+ "which FT8 channel to listen on",
+ "Every band has one agreed dial frequency and everybody uses it. "
+ "Two metres is the default because a plain receiver of this kind can "
+ "reach it; the shortwave bands, which is where almost all the "
+ "activity is, need an upconverter or a receiver in direct sampling "
+ "mode. Choosing a band sets the frequency with it.",
+ choices=_BANDS, flags=("--band",), example="20m"),
+ O("frequency", "Dial frequency", "Receiver", "float",
+ "the dial frequency, if you want one the band list does not have",
+ "Signals sit from about two hundred to three thousand hertz above "
+ "this, which is the upper-sideband convention the whole mode is "
+ "built on. Setting a band sets this; setting this directly leaves "
+ "the band label alone, so use it for an upconverter's offset or a "
+ "channel that is not in the list.",
+ unit="Hz", minimum=1e5, maximum=2e9, flags=("--frequency", "--freq"),
+ example="14074000"),
+ O("simulate", "Simulate", "Receiver", "bool",
+ "invent a band instead of using a receiver",
+ "A made-up band of stations, on a real clock, for trying the display "
+ "and the report without an aerial. What it cannot tell you is "
+ "whether your receiver hears anything.",
+ flags=("--simulate",), off_flags=("--no-simulate",)),
+
+ # -- listening ------------------------------------------------------
+ O("seconds", "Listen for", "Listening", "float",
+ "how long to listen; zero means until stopped",
+ "Rounded up to a whole slot, because half a transmission decodes to "
+ "nothing at all.",
+ unit="s", minimum=0.0, flags=("--seconds",), example="600"),
+ O("slots", "Or this many slots", "Listening", "int",
+ "stop after this many fifteen-second slots; zero means no limit",
+ "The same thing as the time limit, counted the way the band counts "
+ "it. Whichever of the two is reached first stops the run.",
+ minimum=0, flags=("--slots",), example="40"),
+ O("log", "Write a log", "Listening", "bool",
+ "keep every decode in a file",
+ "One line per decode, with the slot, the frequency, how late it was "
+ "and how strongly it came in. The report afterwards is built from "
+ "this, and so is anything you want to do with it later.",
+ flags=("--log",), off_flags=("--no-log",)),
+ O("decodes_seen", "Print each decode", "Listening", "bool",
+ "a line per decode instead of a table that refreshes",
+ "What to use when the output is going into a pipe or a file. The "
+ "table is easier to watch; the lines are easier to grep.",
+ flags=("--decodes-seen",), off_flags=("--no-decodes-seen",)),
+ O("hold", "Keep on display for", "Listening", "float",
+ "how long a station stays on the table after its last decode",
+ "A station transmits every other slot at most, and many call once "
+ "and go away. Long enough that the table is a picture of the "
+ "evening rather than of the last thirty seconds.",
+ unit="s", minimum=1.0, flags=("--hold",), example="3600"),
+
+ # -- decoding -------------------------------------------------------
+ O("lowest", "Search from", "Decoding", "float",
+ "the lowest audio frequency to look for signals at",
+ "Below a couple of hundred hertz there is nothing but the "
+ "receiver's own rumble, and searching it costs time for nothing.",
+ unit="Hz", minimum=0.0, flags=("--lowest",), example="200"),
+ O("highest", "Search to", "Decoding", "float",
+ "the highest audio frequency to look for signals at",
+ "Three kilohertz is where a normal sideband receiver stops. Going "
+ "higher finds nothing unless your receiver is wider than that, and "
+ "costs time in proportion.",
+ unit="Hz", minimum=100.0, flags=("--highest",), example="3000"),
+ O("most", "Candidates per slot", "Decoding", "int",
+ "how many possible transmissions to try to decode each slot",
+ "The sync search ranks every place a transmission could be and this "
+ "is how far down the list to go. A busy shortwave band puts forty "
+ "real signals in a slot and several hundred plausible-looking "
+ "places; more than a few hundred has been measured to add nothing "
+ "but time.",
+ minimum=1, flags=("--most",), example="300"),
+ O("rounds", "Repair passes", "Decoding", "int",
+ "how many passes of error correction to make before giving up",
+ "The code either converges in twenty or so passes or it does not "
+ "converge at all -- raising this has been measured to find nothing "
+ "further, and it is here so that can be checked rather than taken "
+ "on trust.",
+ minimum=1, flags=("--rounds",), example="30"),
+
+ # -- showing --------------------------------------------------------
+ O("grid", "Aerial at", "Showing", "text",
+ "your own grid square, so distances can be worked out",
+ "Four characters, like IO91 or FN31. Every FT8 station that calls "
+ "CQ says where it is this way, so with your own square filled in "
+ "the report can say how far each of them is and on what bearing — "
+ "which on shortwave is the whole interest of the thing. Left blank "
+ "the decodes are still recorded; there is simply nothing to measure "
+ "them from.",
+ flags=("--grid",), example="IO91", metavar="SQUARE"),
+ O("units", "Show readings in", "Showing", "choice",
+ "metric or imperial, for the display and the export",
+ "Distances only. Signal reports are in decibels either way, that "
+ "being what the mode measures in and what gets sent back on the "
+ "air.",
+ choices=("metric", "imperial"), flags=("--units",), example="metric"),
+ O("calls_only", "Callsigns only", "Showing", "bool",
+ "leave out free text and telemetry",
+ "Most of what passes on FT8 is two callsigns and a report. The rest "
+ "is free text -- thirteen characters, no callsign structure, so "
+ "nothing can be said about who sent it -- and telemetry, which is "
+ "eighteen hex digits. Turn this on to see only the exchanges.",
+ flags=("--calls-only",), off_flags=("--no-calls-only",)),
+
+ # -- afterwards -----------------------------------------------------
+ O("report", "Report at the end", "Afterwards", "bool",
+ "print the tables when the listening stops",
+ "Who was heard, how far off, and what was exchanged.",
+ flags=("--report",), off_flags=("--no-report",)),
+ O("csv", "Also write a CSV", "Afterwards", "bool",
+ "a spreadsheet of every decode beside the log",
+ "The same rows as the log, in the form a spreadsheet opens without "
+ "being told anything.",
+ flags=("--csv",), off_flags=("--no-csv",)),
+ O("adif", "Also write an ADIF", "Afterwards", "bool",
+ "the log again, in the form logging programs read",
+ "ADIF is what every amateur logging program imports. What is "
+ "written is what was heard rather than a contact made -- nothing "
+ "here transmits, so nothing here is a QSO — so the records are "
+ "marked as received-only and are for feeding a propagation map or a "
+ "spotting tool rather than for claiming anything.",
+ flags=("--adif",), off_flags=("--no-adif",)),
+)
+
+OPTION_GROUPS = ("Receiver", "Listening", "Decoding", "Showing", "Afterwards")
+
+
+def in_group(group: str) -> tuple[Setting, ...]:
+ return tuple(o for o in OPTIONS if o.group == group)
+
+
+def by_key(key: str):
+ for option in OPTIONS:
+ if option.key == key:
+ return option
+ return None
+
+
+# Options where nothing set means something, rather than nothing.
+_ZERO_MEANS = {
+ "seconds": "until stopped",
+ "slots": "no limit",
+ "grid": "not set, so no distances",
+}
+
+
+def format_option(option: Setting, value) -> str:
+ """Render an option the way the menu and the manual should show it."""
+ if not value and option.key in _ZERO_MEANS:
+ return _ZERO_MEANS[option.key]
+ return format_value(option, value)
+
+
+def describe(options: Ft8Options) -> str:
+ """The one-line summary the menu shows beside Listen."""
+ bits = [band_text(options)]
+ if options.seconds:
+ bits.append(f"{options.seconds:g} s")
+ elif options.slots:
+ bits.append(f"{options.slots} slots")
+ if options.grid:
+ bits.append(f"from {options.grid.upper()}")
+ if options.simulate:
+ bits.append("simulated")
+ return ", ".join(bits)
+
+
+def summarise(options: Ft8Options) -> str:
+ return describe(options)
+
+
+def options_path(directory=None) -> Path:
+ from .config import DEFAULT_CONFIG_DIR
+
+ return Path(directory or DEFAULT_CONFIG_DIR) / "ft8.yaml"
+
+
+def load_options(directory=None) -> Ft8Options:
+ """The saved options, or the defaults. A broken file is not an error."""
+ options = Ft8Options()
+ try:
+ body = yaml.safe_load(options_path(directory).read_text()) or {}
+ except (OSError, ValueError, yaml.YAMLError):
+ return options
+ if not isinstance(body, dict):
+ return options
+ known = set(options.__dict__)
+ for key, value in body.items():
+ if key in known and value is not None:
+ try:
+ setattr(options, key, type(getattr(options, key))(value))
+ except (TypeError, ValueError):
+ pass
+ return options
+
+
+def save_options(options: Ft8Options, directory=None) -> Path:
+ path = options_path(directory)
+ path.parent.mkdir(parents=True, exist_ok=True)
+ with open(path, "w") as fh:
+ yaml.safe_dump(options.to_dict(), fh, sort_keys=False,
+ default_flow_style=False)
+ return path
+
+
+def logs_in(directory) -> list[Path]:
+ try:
+ return sorted(Path(directory).glob("ft8-*.txt"))
+ except OSError:
+ return []
+
+
+# ---------------------------------------------------------------------------
+# Who was heard
+# ---------------------------------------------------------------------------
+
+@dataclass
+class Station:
+ """One callsign, and everything heard from it."""
+
+ call: str = ""
+ grid: str = ""
+ first: float = 0.0
+ last: float = 0.0
+ decodes: int = 0
+ best_snr: float = -99.0
+ worst_snr: float = 99.0
+ hertz: float = 0.0
+ calling: int = 0 # how many times it called CQ
+ worked: set = field(default_factory=set) # who it was talking to
+ said: list = field(default_factory=list) # the lines, newest last
+
+ def away(self, here: str):
+ """How far off and on what bearing, if both squares are known."""
+ if not here or not self.grid:
+ return None
+ return grid_away(here, self.grid)
+
+
+@dataclass
+class Band:
+ """Every station heard, and every exchange between them.
+
+ Kept apart the way the APRS section keeps messages apart from stations,
+ and for the same reason: a decode is about a moment and a station is
+ about an evening, and a table that tried to be both would be neither.
+ """
+
+ stations: dict = field(default_factory=dict)
+ decodes: list = field(default_factory=list)
+ book: code.CallBook = field(default_factory=code.CallBook)
+ slots: int = 0
+
+ def add(self, found) -> None:
+ self.decodes.append(found)
+ # The caller is the second callsign in a standard exchange -- the
+ # first is who is being answered. A CQ has no first, so the one
+ # station named is the one transmitting.
+ sender = found.calls[1] if len(found.calls) > 1 else (
+ found.calls[0] if found.calls else "")
+ if not sender or sender == "<...>":
+ return
+ station = self.stations.get(sender)
+ if station is None:
+ station = Station(call=sender, first=found.at)
+ self.stations[sender] = station
+ station.last = found.at
+ station.decodes += 1
+ station.hertz = found.hertz
+ station.best_snr = max(station.best_snr, found.snr_db)
+ station.worst_snr = min(station.worst_snr, found.snr_db)
+ if found.grid:
+ station.grid = found.grid
+ if found.calling:
+ station.calling += 1
+ elif len(found.calls) > 1 and found.calls[0] != "<...>":
+ station.worked.add(found.calls[0])
+ station.said.append(found.text)
+ if len(station.said) > 32:
+ del station.said[0]
+
+ def all(self) -> list:
+ return list(self.stations.values())
+
+
+@dataclass
+class Heard:
+ """What the front ends share: the running totals of one run."""
+
+ band: Band = field(default_factory=Band)
+ decodes: int = 0
+ slots: int = 0
+ started: float = 0.0
+ latest: list = field(default_factory=list)
+
+
+# ---------------------------------------------------------------------------
+# The receiver
+# ---------------------------------------------------------------------------
+
+AUDIO_RATE = 12_000 # what the decoding wants, and a whole division
+
+
+def open_device(console, options: Ft8Options):
+ """The receiver, or a made-up band, or None if neither can be had."""
+ from rich.panel import Panel
+ from rich.text import Text
+
+ from .device import RtlSdrDevice, RtlSdrError
+
+ if options.simulate:
+ console.print("[yellow]simulated: these stations are not there."
+ "[/yellow]")
+ from .ft8sim import SimulatedBand
+
+ return SimulatedBand(rate=options.rate, realtime=True)
+ try:
+ device = RtlSdrDevice(index=options.device,
+ sample_rate=int(options.rate),
+ gain=options.gain,
+ agc=options.gain == "auto",
+ # Which is what makes the shortwave bands
+ # reachable at all on a dongle that supports
+ # it: below 24 MHz the tuner is bypassed and
+ # the sampler is fed straight off the aerial.
+ direct_sampling="auto")
+ device.open()
+ except RtlSdrError as exc:
+ console.print(Panel(Text(str(exc)),
+ title="[red]cannot open the receiver",
+ border_style="red"))
+ return None
+ return device
+
+
+def audio_from(samples, options: Ft8Options, demod=None):
+ """Upper-sideband audio at twelve kilohertz, from the receiver's samples.
+
+ Upper sideband because that is the convention the whole mode is built
+ on: the dial frequency is the bottom edge and every signal sits above
+ it, which is why two stations three hundred hertz apart are two
+ different signals rather than the same one heard twice.
+ """
+ from .demod import make_demodulator
+
+ if demod is None:
+ demod = make_demodulator("usb", sample_rate=int(options.rate),
+ bandwidth=3200.0, audio_rate=AUDIO_RATE)
+ audio, _iq = demod.step(samples)
+ return audio, demod
+
+
+def pump(device, options: Ft8Options, heard: Heard, log, started: float,
+ on_slot=None, on_decode=None, stopping=None, clock=time.time) -> int:
+ """Read the receiver, a slot at a time, until it stops or is told to.
+
+ The one loop both front ends are driven from, so that what is written
+ to the log cannot depend on which one you happened to be looking at.
+
+ Audio is collected continuously and cut on the slot boundaries rather
+ than being captured slot by slot. A receiver that started reading when
+ the slot began would miss however long it took to start reading, and
+ everything on the band begins in the first half-second.
+ """
+ demod = None
+ audio = np.zeros(0, dtype=np.float32)
+ audio_at = 0.0 # wall time of audio[0]
+ want = next_slot(clock()) # the first whole slot we can get
+ total = 0
+ device.tune(options.frequency)
+ block = int(options.rate) # a second of samples at a time
+
+ while stopping is None or not stopping():
+ at = clock()
+ samples = device.read_samples(block)
+ if samples is None or samples.size == 0:
+ break
+ fresh, demod = audio_from(samples, options, demod)
+ if fresh.size:
+ if audio.size == 0:
+ # The clock was read before asking for the samples, and the
+ # read returns once they have arrived -- so they cover the
+ # block that *starts* here. Dating them a block earlier
+ # instead slides every slice a second late, which cuts the
+ # first half-second off every transmission: three symbols,
+ # part of the opening Costas array, and most of the
+ # decodes with it.
+ audio_at = at
+ audio = np.concatenate([audio, fresh])
+
+ # Every whole slot now sitting in the buffer.
+ while audio.size and audio_at <= want and \
+ audio_at + audio.size / AUDIO_RATE >= want + SLOT:
+ first = int(round((want - audio_at) * AUDIO_RATE))
+ piece = audio[first:first + int(round(SLOT * AUDIO_RATE))]
+ found = wave.listen_to(piece, AUDIO_RATE, book=heard.band.book,
+ most=options.most, rounds=options.rounds,
+ at=want)
+ found = [d for d in found
+ if options.lowest <= d.hertz <= options.highest]
+ if options.calls_only:
+ found = [d for d in found if d.kind in ("standard",
+ "non-standard")]
+ heard.slots += 1
+ heard.band.slots += 1
+ heard.latest = found
+ for d in found:
+ heard.band.add(d)
+ heard.decodes += 1
+ total += 1
+ if log is not None:
+ log.append(d)
+ if on_decode is not None:
+ on_decode(d)
+ if on_slot is not None:
+ on_slot(want, found)
+ want += SLOT
+ # Keep a little before the next slot: a transmission can start
+ # before the slot it belongs to.
+ keep = int(round(max(0.0, want - 3.0 - audio_at) * AUDIO_RATE))
+ if keep > 0:
+ audio = audio[keep:]
+ audio_at += keep / AUDIO_RATE
+
+ if options.seconds and clock() - started >= options.seconds:
+ break
+ if options.slots and heard.slots >= options.slots:
+ break
+ return total
+
+
+def open_log(console, options: Ft8Options, output_dir, started: float):
+ from . import ft8log
+
+ if not options.log:
+ return None
+ log = ft8log.open_log(output_dir, started, options.frequency,
+ options.band)
+ if log is None:
+ console.print("[yellow]could not open a log; carrying on without "
+ "one[/yellow]")
+ return log
+
+
+def listen(console, options: Ft8Options, output_dir: str,
+ heard: Heard | None = None, stopping=None) -> int:
+ """Listen on one channel and write down everything decoded."""
+ from .ui import Ft8Display
+
+ started = time.time()
+ heard = heard if heard is not None else Heard()
+ heard.started = started
+ # The receiver first. Announcing that it is listening and then failing
+ # to open a dongle reads as though the listening went wrong, when what
+ # went wrong happened before any of it started.
+ device = open_device(console, options)
+ if device is None:
+ return 0
+ log = open_log(console, options, output_dir, started)
+ _say_where(console, options)
+ console.print(f"[grey62]listening on {band_text(options)} at "
+ f"{options.rate / 1e6:g} MS/s — control-C to stop"
+ "[/grey62]")
+ if not options.decodes_seen:
+ console.print("[grey62]the first decodes appear when the slot after "
+ "next ends, which is up to thirty seconds away"
+ "[/grey62]")
+ display = None
+ live = None
+ if not options.decodes_seen and getattr(console, "is_terminal", False):
+ from rich.live import Live
+
+ display = Ft8Display(console, hold=options.hold,
+ imperial=options.imperial, grid=options.grid,
+ band=band_text(options))
+ display.started = started
+ live = Live(display.render(), console=console, refresh_per_second=2,
+ screen=False, transient=False, vertical_overflow="crop")
+ live.start()
+
+ def show_slot(at, found):
+ if display is not None:
+ display.update(heard, at)
+ live.update(display.render())
+
+ def show_decode(d):
+ if options.decodes_seen:
+ console.print(f"{slot_name(d.at)} {d.snr_db:>4.0f} "
+ f"{d.offset:>5.1f} {d.hertz:>7.1f} ~ {d.text}")
+
+ try:
+ pump(device, options, heard, log, started, on_slot=show_slot,
+ on_decode=show_decode, stopping=stopping)
+ except KeyboardInterrupt:
+ console.print("\n[grey62]stopped[/grey62]")
+ finally:
+ if live is not None:
+ live.stop()
+ try:
+ device.close()
+ except Exception:
+ pass
+ return finish(console, options, output_dir, heard, log)
+
+
+def _say_where(console, options: Ft8Options) -> None:
+ """Whether distances can be worked out, and what it costs if not."""
+ if options.grid:
+ where = grid_at(options.grid)
+ console.print(f"[grey62]aerial at {options.grid.upper()} "
+ f"({where[0]:.2f},{where[1]:.2f}) — distances and "
+ f"bearings will be worked out[/grey62]")
+ return
+ console.print("[grey62]no grid square set, so no distances and no "
+ "bearings — `--grid IO91` gives both[/grey62]")
+ if band_needs(options.band) == "shortwave":
+ console.print("[grey62]this band is shortwave: a plain receiver "
+ "needs an upconverter or direct sampling to hear it"
+ "[/grey62]")
+
+
+def finish(console, options: Ft8Options, output_dir: str, heard: Heard,
+ log=None) -> int:
+ """Close the log, write the exports, and print the report."""
+ from . import ft8log
+
+ if log is not None:
+ log.close()
+ console.print(f"[grey62]{log.lines} decodes written to "
+ f"{log.path}[/grey62]")
+ if options.csv:
+ where = ft8log.write_csv(log.path.with_suffix(".csv"),
+ heard.band.decodes)
+ if where is not None:
+ console.print(f"[grey62]and as a spreadsheet: {where}"
+ "[/grey62]")
+ if options.adif:
+ where = ft8log.write_adif(log.path.with_suffix(".adi"),
+ heard.band.decodes, band=options.band,
+ frequency=options.frequency,
+ my_grid=options.grid)
+ if where is not None:
+ console.print(f"[grey62]and as ADIF, marked as heard rather "
+ f"than worked: {where}[/grey62]")
+ if options.report:
+ report(console, heard.band, options)
+ return heard.decodes
+
+
+def _away_text(station: Station, options: Ft8Options) -> str:
+ away = station.away(options.grid)
+ if away is None:
+ return ""
+ km, bearing = away
+ if options.imperial:
+ return f"{km * 0.621371:.0f} mi {bearing:.0f}°"
+ return f"{km:.0f} km {bearing:.0f}°"
+
+
+def report(console, band: Band, options: Ft8Options) -> None:
+ """Three tables, because they answer three different questions."""
+ from rich.table import Table
+
+ stations = sorted(band.all(), key=lambda s: (-s.decodes, s.call))
+ if not stations and not band.decodes:
+ console.print("[yellow]nothing decoded[/yellow]")
+ if band.slots:
+ console.print("[grey62]the band was quiet, the aerial is not "
+ "hearing it, or the clock is out: FT8 needs the "
+ "clock right to a second or two[/grey62]")
+ return
+
+ console.print()
+ console.print(f"[bold]Stations heard[/bold] — {len(stations)} in "
+ f"{band.slots} slots")
+ t = Table(box=None, header_style="bold", pad_edge=False)
+ t.add_column("station")
+ t.add_column("grid")
+ if options.grid:
+ t.add_column("away")
+ t.add_column("decodes", justify="right")
+ t.add_column("best", justify="right")
+ t.add_column("worst", justify="right")
+ t.add_column("audio", justify="right")
+ t.add_column("CQ", justify="right")
+ for s in stations[:60]:
+ row = [s.call, s.grid or "—"]
+ if options.grid:
+ row.append(_away_text(s, options) or "—")
+ row += [str(s.decodes), f"{s.best_snr:.0f}", f"{s.worst_snr:.0f}",
+ f"{s.hertz:.0f} Hz", str(s.calling) if s.calling else ""]
+ t.add_row(*row)
+ console.print(t)
+ if len(stations) > 60:
+ console.print(f"[grey62]…and {len(stations) - 60} more[/grey62]")
+
+ calling = [s for s in stations if s.calling]
+ if calling:
+ console.print()
+ console.print(f"[bold]Calling CQ[/bold] — {len(calling)} stations "
+ "looking for a contact")
+ t = Table(box=None, header_style="bold", pad_edge=False)
+ t.add_column("station")
+ t.add_column("grid")
+ if options.grid:
+ t.add_column("away")
+ t.add_column("times", justify="right")
+ t.add_column("best", justify="right")
+ for s in sorted(calling, key=lambda s: -s.calling)[:30]:
+ row = [s.call, s.grid or "—"]
+ if options.grid:
+ row.append(_away_text(s, options) or "—")
+ row += [str(s.calling), f"{s.best_snr:.0f}"]
+ t.add_row(*row)
+ console.print(t)
+
+ worked = [s for s in stations if s.worked]
+ if worked:
+ console.print()
+ console.print("[bold]Who was working whom[/bold] — the exchanges "
+ "that passed while this was listening")
+ t = Table(box=None, header_style="bold", pad_edge=False)
+ t.add_column("station")
+ t.add_column("answered")
+ for s in sorted(worked, key=lambda s: -len(s.worked))[:30]:
+ t.add_row(s.call, ", ".join(sorted(s.worked)[:8]))
+ console.print(t)
+
+ if options.grid:
+ far = [(s, s.away(options.grid)) for s in stations if s.grid]
+ far = [(s, a) for s, a in far if a is not None]
+ if far:
+ s, (km, bearing) = max(far, key=lambda x: x[1][0])
+ unit = (f"{km * 0.621371:.0f} miles" if options.imperial
+ else f"{km:.0f} km")
+ console.print()
+ console.print(f"[grey62]furthest heard: {s.call} in {s.grid}, "
+ f"{unit} on {bearing:.0f}°[/grey62]")
diff --git a/bandsaunter/ft8code.py b/bandsaunter/ft8code.py
new file mode 100644
index 0000000..667e5c0
--- /dev/null
+++ b/bandsaunter/ft8code.py
@@ -0,0 +1,712 @@
+"""What an FT8 transmission carries, and how it is wrapped.
+
+No radio in here at all. This is the part between a string of one hundred
+and seventy-four bits and a line somebody can read: the checksum, the
+error-correcting code that makes FT8 work at signal levels where the
+operator hears nothing, the Gray mapping onto eight tones, and the
+seventy-seven bits that hold two callsigns and a grid square.
+
+The shape of it, outward from the message:
+
+ 77 bits what was said: two callsigns and a report, or free text
+ + 14 bits a CRC, so a wrong answer is caught rather than printed
+ = 91 bits
+ + 83 bits LDPC parity, so a wrong answer is usually *repaired*
+ = 174 bits
+ / 3 three bits to a tone, in Gray order
+ = 58 tones
+ + 21 tones three Costas arrays, at the start, the middle and the end
+ = 79 tones at 6.25 Hz apart and 6.25 to the second: 12.64 seconds
+
+The error correction is the whole trick. Fifty-eight tones of payload
+carried in one hundred and seventy-four bits is a rate of about one half:
+half of what is transmitted is redundancy, and that is what buys a protocol
+that decodes at twenty-odd decibels below the noise in the same bandwidth.
+A receiver that only took the strongest tone in each symbol and hoped would
+decode almost nothing -- which is why the tone detector below reports how
+confident it is, bit by bit, rather than just what it thinks it heard.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+
+import numpy as np
+
+from .ft8tables import (BITS, CHECKS, COSTAS, CRC_BITS, CRC_POLYNOMIAL,
+ GENERATOR, GRAY, PARITY, PAYLOAD, TONES)
+
+__all__ = ["crc14", "with_crc", "crc_holds", "encode", "repair",
+ "parity_holds", "tones_of",
+ "bits_of", "unpack", "pack", "CallBook", "Message", "MESSAGE_BITS",
+ "SYMBOL_HZ", "SYMBOL_S", "SLOT_S", "SENDING_S", "COSTAS", "TONES",
+ "hash_of", "free_text", "is_standard_call"]
+
+MESSAGE_BITS = 77 # what is actually said, before the checksum
+
+# The channel, in numbers. Six and a quarter of everything: the tones are
+# 6.25 Hz apart and there are 6.25 of them a second, which makes each tone
+# exactly one cycle of separation long and the whole thing orthogonal.
+SYMBOL_HZ = 6.25
+SYMBOL_S = 1.0 / SYMBOL_HZ
+SENDING_S = TONES * SYMBOL_S # 12.64 s of transmission
+SLOT_S = 15.0 # in a slot of fifteen
+
+
+# ---------------------------------------------------------------------------
+# The checksum
+# ---------------------------------------------------------------------------
+
+def crc14(bits) -> int:
+ """CRC-14 over a sequence of bits, most significant first.
+
+ The one thing standing between a repaired codeword and a confident lie.
+ The error correction below will happily converge on *a* valid codeword
+ from noise; whether it is the codeword that was sent is what this
+ answers, and one in sixteen thousand wrong answers gets through, which
+ over an evening is a handful of lines nobody should trust. Every
+ decoder prints them anyway, and so does this one -- but only after the
+ CRC has agreed, which is the difference between a handful and a flood.
+ """
+ top = 1 << (CRC_BITS - 1)
+ mask = (top << 1) - 1
+ remainder = 0
+ for i, bit in enumerate(bits):
+ if i % 8 == 0:
+ # A byte at a time into the top of the register, exactly as the
+ # specification frames it: the message is a byte sequence and
+ # the odd bits at the end are zeros.
+ byte = 0
+ for k in range(8):
+ byte = (byte << 1) | (bits[i + k] if i + k < len(bits) else 0)
+ remainder ^= byte << (CRC_BITS - 8)
+ remainder = ((remainder << 1) ^ CRC_POLYNOMIAL if remainder & top
+ else remainder << 1)
+ return remainder & mask
+
+
+def with_crc(payload) -> np.ndarray:
+ """The seventy-seven bits said, plus the fourteen that check them.
+
+ The checksum covers eighty-two bits rather than seventy-seven: the
+ message zero-extended by five, which is what the specification says and
+ is not the same answer as checksumming seventy-seven.
+ """
+ payload = np.asarray(payload, dtype=np.uint8)
+ if payload.size != MESSAGE_BITS:
+ raise ValueError(f"a message is {MESSAGE_BITS} bits, not "
+ f"{payload.size}")
+ extended = np.concatenate([payload, np.zeros(5, dtype=np.uint8)])
+ check = crc14(extended.tolist())
+ tail = np.array([(check >> (CRC_BITS - 1 - k)) & 1
+ for k in range(CRC_BITS)], dtype=np.uint8)
+ return np.concatenate([payload, tail])
+
+
+def crc_holds(bits91) -> bool:
+ """Whether the checksum on a decoded block agrees with its message."""
+ bits91 = np.asarray(bits91, dtype=np.uint8)
+ if bits91.size != PAYLOAD:
+ return False
+ return bool((with_crc(bits91[:MESSAGE_BITS]) == bits91).all())
+
+
+# ---------------------------------------------------------------------------
+# The error-correcting code
+# ---------------------------------------------------------------------------
+
+def _generator() -> np.ndarray:
+ """The parity half of the generator, unpacked to bits."""
+ rows = np.zeros((PARITY, PAYLOAD), dtype=np.uint8)
+ for m, row in enumerate(GENERATOR):
+ packed = bytes.fromhex(row)
+ for k in range(PAYLOAD):
+ rows[m, k] = (packed[k // 8] >> (7 - k % 8)) & 1
+ return rows
+
+
+GEN = _generator()
+
+
+def encode(bits91) -> np.ndarray:
+ """Ninety-one bits in, one hundred and seventy-four out.
+
+ Systematic: the message comes out unchanged at the front and the parity
+ is appended, which is why the decoder can read the message straight off
+ a repaired codeword without undoing anything.
+ """
+ bits91 = np.asarray(bits91, dtype=np.uint8)
+ if bits91.size != PAYLOAD:
+ raise ValueError(f"expected {PAYLOAD} bits, got {bits91.size}")
+ parity = (GEN @ bits91) & 1
+ return np.concatenate([bits91, parity.astype(np.uint8)])
+
+
+def _sparse():
+ """The parity-check matrix as flat slots, for the message passing.
+
+ Every codeword bit sits in exactly three checks and every check covers
+ six or seven bits, so the whole graph fits in two small index arrays and
+ the decoding never needs a scatter-add. The short checks are padded to
+ seven with a slot pointing at a dummy bit that is held at no opinion.
+ """
+ width = max(len(c) for c in CHECKS)
+ bit_of_slot = np.full(PARITY * width, BITS, dtype=np.int64) # BITS = dummy
+ live = np.zeros(PARITY * width, dtype=bool)
+ for m, check in enumerate(CHECKS):
+ for j, one_based in enumerate(check):
+ bit_of_slot[m * width + j] = one_based - 1
+ live[m * width + j] = True
+ # And the other way: for each bit, the slots that talk about it.
+ per_bit = [[] for _ in range(BITS)]
+ for slot in np.flatnonzero(live):
+ per_bit[bit_of_slot[slot]].append(slot)
+ degree = max(len(s) for s in per_bit)
+ slots_of_bit = np.zeros((BITS, degree), dtype=np.int64)
+ for n, slots in enumerate(per_bit):
+ slots_of_bit[n] = slots + [slots[-1]] * (degree - len(slots))
+ return width, bit_of_slot, live, slots_of_bit
+
+
+WIDTH, BIT_OF_SLOT, LIVE, SLOTS_OF_BIT = _sparse()
+
+
+def parity_holds(bits174) -> bool:
+ """Whether every one of the eighty-three checks comes out even.
+
+ Here rather than left to callers because the padding convention -- short
+ checks point their spare slots at a dummy bit past the end -- is an
+ implementation detail of the message passing, and anything outside this
+ module that had to know about it would be a bug waiting to be written.
+ """
+ bits174 = np.asarray(bits174, dtype=np.uint8)
+ if bits174.size != BITS:
+ return False
+ padded = np.concatenate([bits174, np.zeros(1, dtype=np.uint8)])
+ sums = padded[BIT_OF_SLOT].copy()
+ sums[~LIVE] = 0
+ return bool((sums.reshape(PARITY, WIDTH).sum(axis=1) % 2 == 0).all())
+
+
+def repair(llr, rounds: int = 30, alpha: float = 0.75):
+ """Belief propagation over one or many candidate codewords.
+
+ ``llr`` is how strongly each bit is believed to be a zero: positive for
+ zero, negative for one, and the size of it is the confidence. Shape
+ (174,) for one candidate or (n, 174) for a batch, which is how it is
+ actually used -- a busy slot throws up hundreds of candidates and doing
+ them one at a time is most of the decoding time.
+
+ Normalised min-sum rather than the exact sum-product: it is within a
+ few tenths of a decibel of it, and it is multiplication-free in the
+ inner loop, which at this size matters more than the tenths.
+
+ Returns the repaired bits, or None where the checks never came out
+ even. An answer here still has to pass the CRC before it is believed:
+ this finds *a* codeword, and noise has valid codewords in it too.
+ """
+ single = np.ndim(llr) == 1
+ belief = np.atleast_2d(np.asarray(llr, dtype=np.float32))
+ n = belief.shape[0]
+ if belief.shape[1] != BITS:
+ raise ValueError(f"a codeword is {BITS} bits, not {belief.shape[1]}")
+
+ messages = np.zeros((n, PARITY * WIDTH), dtype=np.float32)
+ dead = ~LIVE
+ dead_grid = dead.reshape(PARITY, WIDTH)
+ out = np.zeros((n, BITS), dtype=np.uint8)
+ done = np.zeros(n, dtype=bool)
+
+ def totals(msgs):
+ """What every bit is believed to be, once its three checks are in.
+
+ A gather and a sum rather than a scatter-add: every bit sits in
+ exactly three checks, so the three places to look are known in
+ advance and np.add.at -- which is where an earlier version of this
+ spent most of its time -- is not needed at all.
+ """
+ return belief + msgs[:, SLOTS_OF_BIT].sum(axis=2)
+
+ for _ in range(rounds):
+ total = totals(messages)
+ # What each check hears about a bit, less what it said itself.
+ wide = np.concatenate([total, np.zeros((n, 1), dtype=np.float32)],
+ axis=1)
+ heard = (wide[:, BIT_OF_SLOT] - messages).reshape(n, PARITY, WIDTH)
+
+ size = np.abs(heard)
+ size[:, dead_grid] = np.inf
+ # The smallest and the next smallest in each check, so "the
+ # smallest of the others" is a lookup rather than a loop.
+ order = np.argsort(size, axis=2)
+ smallest = np.take_along_axis(size, order[:, :, :1], axis=2)
+ next_up = np.take_along_axis(size, order[:, :, 1:2], axis=2)
+ others = np.where(size == smallest, next_up, smallest)
+
+ odd = (heard < 0).sum(axis=2, keepdims=True) % 2 == 1
+ mine = np.where(heard < 0, -1.0, 1.0).astype(np.float32)
+ whole = np.where(odd, -1.0, 1.0).astype(np.float32)
+ messages = (alpha * whole * mine
+ * np.minimum(others, 1e4)).astype(np.float32)
+ messages = messages.reshape(n, -1)
+ messages[:, dead] = 0.0
+
+ hard = (totals(messages) < 0).astype(np.uint8)
+ wide_hard = np.concatenate([hard, np.zeros((n, 1), dtype=np.uint8)],
+ axis=1)
+ sums = wide_hard[:, BIT_OF_SLOT].reshape(n, PARITY, WIDTH).copy()
+ sums[:, dead_grid] = 0
+ even = (sums.sum(axis=2) % 2 == 0).all(axis=1)
+ fresh = even & ~done
+ if fresh.any():
+ out[fresh] = hard[fresh]
+ done |= fresh
+ if done.all():
+ break
+
+ if single:
+ return out[0] if done[0] else None
+ return out, done
+
+
+# ---------------------------------------------------------------------------
+# Tones
+# ---------------------------------------------------------------------------
+
+UNGRAY = np.zeros(8, dtype=np.uint8)
+for _value, _tone in enumerate(GRAY):
+ UNGRAY[_tone] = _value
+
+SYNC_AT = (0, 36, 72) # where the three Costas arrays sit
+
+
+def tones_of(bits174) -> np.ndarray:
+ """The seventy-nine tones that carry these bits, sync included."""
+ bits174 = np.asarray(bits174, dtype=np.uint8)
+ if bits174.size != BITS:
+ raise ValueError(f"a codeword is {BITS} bits, not {bits174.size}")
+ trips = bits174.reshape(-1, 3)
+ values = trips[:, 0] * 4 + trips[:, 1] * 2 + trips[:, 2]
+ data = np.array([GRAY[v] for v in values], dtype=np.uint8)
+ out = np.zeros(TONES, dtype=np.uint8)
+ costas = np.array(COSTAS, dtype=np.uint8)
+ taken = 0
+ for i in range(TONES):
+ which = i // 36 if i % 36 < 7 else -1
+ if i in range(0, 7) or i in range(36, 43) or i in range(72, 79):
+ out[i] = costas[(i - (0 if i < 7 else 36 if i < 43 else 72))]
+ else:
+ out[i] = data[taken]
+ taken += 1
+ return out
+
+
+def bits_of(tones) -> np.ndarray:
+ """The bits those tones carry, sync thrown away."""
+ tones = np.asarray(tones, dtype=np.uint8)
+ if tones.size != TONES:
+ raise ValueError(f"a transmission is {TONES} tones, not {tones.size}")
+ data = [tones[i] for i in range(TONES)
+ if not (i < 7 or 36 <= i < 43 or 72 <= i)]
+ out = np.zeros(BITS, dtype=np.uint8)
+ for k, tone in enumerate(data):
+ value = UNGRAY[tone]
+ out[3 * k] = (value >> 2) & 1
+ out[3 * k + 1] = (value >> 1) & 1
+ out[3 * k + 2] = value & 1
+ return out
+
+
+# ---------------------------------------------------------------------------
+# Callsigns
+# ---------------------------------------------------------------------------
+
+ALPHANUM_SPACE = " 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ" # 37
+ALPHANUM = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ" # 36
+NUMERIC = "0123456789" # 10
+LETTERS_SPACE = " ABCDEFGHIJKLMNOPQRSTUVWXYZ" # 27
+FULL = " 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ+-./?" # 42
+WITH_SLASH = " 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ/" # 38
+
+TOKENS = 2_063_592 # DE, QRZ, CQ and the CQ-with-an-argument forms
+MAX22 = 4_194_304 # room for a hashed callsign
+MAX_GRID = 32_400 # four-character grids, before the report forms
+
+
+def hash_of(call: str, width: int = 22) -> int:
+ """The short hash a compound callsign is carried as.
+
+ A callsign that will not fit in twenty-eight bits -- anything with a
+ slash in it, and anything long -- travels as a hash, and the receiver
+ is expected to have heard the full call earlier in the exchange and to
+ remember it. That is why a fresh receiver prints <...>: not a decoding
+ failure, but a station whose full name has not been said yet in
+ anything this receiver heard.
+ """
+ packed = 0
+ padded = (call.upper() + " ")[:11]
+ for ch in padded:
+ packed = packed * 38 + (WITH_SLASH.index(ch) if ch in WITH_SLASH else 0)
+ return int((47_055_833_459 * packed) >> (64 - width)) & ((1 << width) - 1)
+
+
+def is_standard_call(call: str) -> bool:
+ """Whether a callsign fits the twenty-eight bit form.
+
+ One or two characters of prefix, a digit, and up to three letters. Most
+ callsigns on earth; not the ones with a slash in them.
+ """
+ return _pack_standard(call) is not None
+
+
+def _pack_standard(call: str):
+ """A plain callsign as its number, or None if it is not a plain one.
+
+ The form is one or two characters of prefix, exactly one digit, and up
+ to three letters -- and the twenty-eight bit packing wants the digit in
+ the third place. Which character is *the* digit cannot be found by
+ looking at the second one: Z33Z has a digit there and it belongs to the
+ prefix. It is the last character that is not one of the trailing
+ letters, so that is how it is found.
+ """
+ call = (call or "").strip().upper()
+ if not call or "/" in call or len(call) > 6:
+ return None
+ at = len(call) - 1
+ while at >= 0 and call[at].isalpha():
+ at -= 1
+ if at < 0 or at > 2 or not call[at].isdigit():
+ return None
+ call = " " * (2 - at) + call
+ if len(call) > 6:
+ return None
+ call = (call + " ")[:6]
+ if (call[0] not in ALPHANUM_SPACE or call[1] not in ALPHANUM
+ or call[2] not in NUMERIC):
+ return None
+ if any(c not in LETTERS_SPACE for c in call[3:6]):
+ return None
+ n = ALPHANUM_SPACE.index(call[0])
+ n = n * 36 + ALPHANUM.index(call[1])
+ n = n * 10 + NUMERIC.index(call[2])
+ n = n * 27 + LETTERS_SPACE.index(call[3])
+ n = n * 27 + LETTERS_SPACE.index(call[4])
+ n = n * 27 + LETTERS_SPACE.index(call[5])
+ return n
+
+
+def _unpack_standard(n: int) -> str:
+ out = [""] * 6
+ out[5] = LETTERS_SPACE[n % 27]
+ n //= 27
+ out[4] = LETTERS_SPACE[n % 27]
+ n //= 27
+ out[3] = LETTERS_SPACE[n % 27]
+ n //= 27
+ out[2] = NUMERIC[n % 10]
+ n //= 10
+ out[1] = ALPHANUM[n % 36]
+ n //= 36
+ out[0] = ALPHANUM_SPACE[n % 37]
+ return "".join(out).strip()
+
+
+@dataclass
+class CallBook:
+ """Full callsigns heard earlier, so a hash can be turned back into one.
+
+ A compound callsign travels as a hash and is expected to be remembered
+ from earlier in the same exchange. This is that memory. It is only
+ ever added to by callsigns that arrived in full and passed their CRC,
+ so a station that was never heard properly stays <...> rather than being
+ guessed at -- a wrong callsign in a log is worse than a missing one.
+ """
+
+ by_hash: dict = field(default_factory=dict)
+
+ def remember(self, call: str) -> None:
+ call = (call or "").strip().upper()
+ if len(call) < 3 or call in ("CQ", "DE", "QRZ"):
+ return
+ for width in (10, 12, 22):
+ self.by_hash[(width, hash_of(call, width))] = call
+
+ def look_up(self, value: int, width: int = 22) -> str:
+ return self.by_hash.get((width, value), "<...>")
+
+
+def _unpack28(n: int, book: CallBook) -> str:
+ """One of the two callsign fields of a standard message."""
+ if n < TOKENS:
+ if n == 0:
+ return "DE"
+ if n == 1:
+ return "QRZ"
+ if n == 2:
+ return "CQ"
+ if n <= 1002:
+ return f"CQ {n - 3:03d}"
+ if n <= 532_443:
+ rest = n - 1003
+ letters = [""] * 4
+ for i in (3, 2, 1, 0):
+ letters[i] = LETTERS_SPACE[rest % 27]
+ rest //= 27
+ return "CQ " + "".join(letters).strip()
+ return "<...>"
+ n -= TOKENS
+ if n < MAX22:
+ return book.look_up(n, 22)
+ return _unpack_standard(n - MAX22)
+
+
+def _unpack_grid(value: int, rover: int) -> str:
+ """The fifteen-bit field: a grid square, a signal report, or a sign-off."""
+ if value <= MAX_GRID:
+ n = value
+ out = ["", "", "", ""]
+ out[3] = NUMERIC[n % 10]
+ n //= 10
+ out[2] = NUMERIC[n % 10]
+ n //= 10
+ out[1] = chr(ord("A") + n % 18)
+ n //= 18
+ out[0] = chr(ord("A") + n % 18)
+ grid = "".join(out)
+ return ("R " + grid) if rover else grid
+ report = value - MAX_GRID
+ if report == 1:
+ return ""
+ if report == 2:
+ return "RRR"
+ if report == 3:
+ return "RR73"
+ if report == 4:
+ return "73"
+ decibels = report - 35
+ return f"{'R' if rover else ''}{decibels:+03d}"
+
+
+def _pack_grid(text: str):
+ """The inverse: a grid, a report or a token as fifteen bits and a flag."""
+ text = (text or "").strip().upper()
+ if not text:
+ return MAX_GRID + 1, 0
+ if text == "RRR":
+ return MAX_GRID + 2, 0
+ if text == "RR73":
+ return MAX_GRID + 3, 0
+ if text == "73":
+ return MAX_GRID + 4, 0
+ rover = 0
+ if text.startswith("R ") or (text.startswith("R") and len(text) > 1
+ and text[1] in "+-"):
+ rover, text = 1, text[2:].strip() if text[1] == " " else text[1:]
+ if (len(text) == 4 and "A" <= text[0] <= "R" and "A" <= text[1] <= "R"
+ and text[2].isdigit() and text[3].isdigit()):
+ value = ((ord(text[0]) - 65) * 18 + (ord(text[1]) - 65)) * 10
+ value = (value + int(text[2])) * 10 + int(text[3])
+ return value, rover
+ try:
+ return MAX_GRID + 35 + int(text), rover
+ except ValueError:
+ return None
+
+
+# ---------------------------------------------------------------------------
+# Messages
+# ---------------------------------------------------------------------------
+
+@dataclass
+class Message:
+ """One decoded line, and what could be picked out of it."""
+
+ text: str = ""
+ kind: str = "" # standard, free text, telemetry, ...
+ calls: tuple = () # the callsigns in it, in order
+ grid: str = "" # a four-character grid, where there was one
+ report: str = "" # a signal report, where there was one
+ calling: bool = False # whether this is a CQ
+
+
+def _number(bits, start: int, count: int) -> int:
+ value = 0
+ for k in range(count):
+ value = (value << 1) | int(bits[start + k])
+ return value
+
+
+def _put(bits, start: int, count: int, value: int) -> None:
+ for k in range(count):
+ bits[start + k] = (value >> (count - 1 - k)) & 1
+
+
+def free_text(bits77) -> str:
+ """The thirteen-character form: anything at all, at a price.
+
+ Base forty-two over seventy-one bits, which is thirteen characters and
+ no callsign structure -- so a receiver cannot tell who sent it from the
+ message alone, and nor can the network that reports these things.
+ """
+ n = _number(bits77, 0, 71)
+ out = [""] * 13
+ for i in range(12, -1, -1):
+ out[i] = FULL[n % 42]
+ n //= 42
+ return "".join(out).strip()
+
+
+def unpack(bits77, book: CallBook | None = None) -> Message:
+ """Seventy-seven bits as something to read.
+
+ The type is the last three bits, which is an odd place for it until you
+ remember that everything before it is a different length in each form.
+ """
+ bits77 = np.asarray(bits77, dtype=np.uint8)
+ book = book or CallBook()
+ i3 = _number(bits77, 74, 3)
+
+ if i3 == 0:
+ n3 = _number(bits77, 71, 3)
+ if n3 == 0:
+ return Message(text=free_text(bits77), kind="free text")
+ if n3 == 5:
+ value = _number(bits77, 0, 71)
+ return Message(text=f"{value:018X}", kind="telemetry")
+ return Message(text=free_text(bits77), kind="free text")
+
+ if i3 in (1, 2):
+ a = _number(bits77, 0, 28)
+ a_rover = _number(bits77, 28, 1)
+ b = _number(bits77, 29, 28)
+ b_rover = _number(bits77, 57, 1)
+ rover = _number(bits77, 58, 1)
+ grid = _number(bits77, 59, 15)
+ one = _unpack28(a, book)
+ two = _unpack28(b, book)
+ if a_rover:
+ one += "/R"
+ if b_rover:
+ two += "/R"
+ extra = _unpack_grid(grid, rover)
+ text = " ".join(x for x in (one, two, extra) if x)
+ looks_like_grid = (len(extra) == 4 and extra[:2].isalpha()
+ and extra[2:].isdigit())
+ return Message(text=text, kind="standard",
+ calls=tuple(c for c in (one, two)
+ if c not in ("CQ", "DE", "QRZ")
+ and not c.startswith("CQ ")),
+ grid=extra if looks_like_grid else "",
+ report="" if looks_like_grid else extra,
+ calling=one.startswith("CQ") or one == "QRZ")
+
+ if i3 == 4:
+ # A compound callsign: one hashed, one carried in full as eleven
+ # characters of base thirty-eight.
+ short = _number(bits77, 0, 12)
+ n58 = _number(bits77, 12, 58)
+ flip = _number(bits77, 70, 1)
+ kind = _number(bits77, 71, 2)
+ rest = n58
+ letters = [""] * 11
+ for i in range(10, -1, -1):
+ letters[i] = WITH_SLASH[rest % 38]
+ rest //= 38
+ full = "".join(letters).strip()
+ other = book.look_up(short, 12)
+ one, two = (full, other) if flip else (other, full)
+ tail = {1: "RRR", 2: "RR73", 3: "73"}.get(kind, "")
+ text = " ".join(x for x in (one, two, tail) if x)
+ return Message(text=text, kind="non-standard",
+ calls=tuple(c for c in (one, two) if c != "<...>"),
+ report=tail, calling=one.startswith("CQ"))
+
+ return Message(text=free_text(bits77), kind=f"type {i3}")
+
+
+def pack(text: str, book: CallBook | None = None):
+ """The inverse, for the standard and free-text forms.
+
+ Here so the decoder can be tested against something that is not itself.
+ A receiver never needs to build a message -- this one does not transmit
+ -- but a test that encodes with the same misunderstanding it decodes
+ with agrees with itself about anything they are both wrong about.
+ """
+ text = (text or "").strip().upper()
+ bits = np.zeros(MESSAGE_BITS, dtype=np.uint8)
+ words = text.split()
+
+ if len(words) in (2, 3) or (len(words) == 4 and words[0] == "CQ"):
+ made = _pack_standard_message(words, bits)
+ if made is not None:
+ return made
+ return _pack_free_text(text, bits)
+
+
+def _pack_standard_message(words, bits):
+ if words[0] == "CQ" and len(words) >= 3 and len(words[1]) <= 4 \
+ and not _is_call(words[1]):
+ one, rest = f"CQ {words[1]}", words[2:]
+ else:
+ one, rest = words[0], words[1:]
+ if not rest:
+ return None
+ two, extra = rest[0], (rest[1] if len(rest) > 1 else "")
+ a = _pack28(one)
+ b = _pack28(two)
+ if a is None or b is None:
+ return None
+ grid = _pack_grid(extra)
+ if grid is None:
+ return None
+ value, rover = grid
+ _put(bits, 0, 28, a)
+ _put(bits, 28, 1, 0)
+ _put(bits, 29, 28, b)
+ _put(bits, 57, 1, 0)
+ _put(bits, 58, 1, rover)
+ _put(bits, 59, 15, value)
+ _put(bits, 74, 3, 1)
+ return bits
+
+
+def _is_call(word: str) -> bool:
+ return any(c.isdigit() for c in word) and _pack_standard(word) is not None
+
+
+def _pack28(call: str):
+ call = call.strip().upper()
+ if call == "DE":
+ return 0
+ if call == "QRZ":
+ return 1
+ if call == "CQ":
+ return 2
+ if call.startswith("CQ "):
+ rest = call[3:].strip()
+ if rest.isdigit() and len(rest) == 3:
+ return 3 + int(rest)
+ if 1 <= len(rest) <= 4 and all(c in LETTERS_SPACE for c in rest):
+ n = 0
+ for ch in (rest + " ")[:4]:
+ n = n * 27 + LETTERS_SPACE.index(ch)
+ return 1003 + n
+ return None
+ plain = _pack_standard(call)
+ if plain is not None:
+ return TOKENS + MAX22 + plain
+ # Deliberately not a hash. A hash always succeeds, and a packer that
+ # always succeeds would send "HELLO WORLD" as two hashed callsigns
+ # rather than as the free text it plainly is.
+ return None
+
+
+def _pack_free_text(text: str, bits):
+ text = "".join(c if c in FULL else " " for c in text)[:13]
+ text = text.rjust(13)
+ n = 0
+ for ch in text:
+ n = n * 42 + FULL.index(ch)
+ _put(bits, 0, 71, n)
+ _put(bits, 71, 3, 0)
+ _put(bits, 74, 3, 0)
+ return bits
diff --git a/bandsaunter/ft8log.py b/bandsaunter/ft8log.py
new file mode 100644
index 0000000..140cfd6
--- /dev/null
+++ b/bandsaunter/ft8log.py
@@ -0,0 +1,177 @@
+"""Writing FT8 decodes down.
+
+Three forms, because three different things want to read them. The log is
+for a person and for this program's own report; the CSV is for a
+spreadsheet; the ADIF is for the logging and spotting programs every
+amateur already has.
+
+The ADIF is the one worth a note. It is the format for recording contacts,
+and nothing here makes a contact -- this receiver does not transmit. So
+what is written is a record of a station *heard*, marked as such, which is
+what a propagation map or a reverse-beacon feed wants. Passing it off as a
+worked contact would put claims into somebody's log that they cannot make.
+"""
+
+from __future__ import annotations
+
+import csv
+from dataclasses import dataclass
+from datetime import datetime, timezone
+from pathlib import Path
+
+from . import __version__
+
+__all__ = ["Ft8Log", "open_log", "write_csv", "write_adif", "read_log",
+ "LOG_VERSION"]
+
+LOG_VERSION = 1
+
+
+@dataclass
+class Ft8Log:
+ """A file of decodes, one to a line, written as they arrive."""
+
+ path: Path
+ handle: object = None
+ lines: int = 0
+
+ def append(self, found) -> None:
+ if self.handle is None:
+ return
+ when = datetime.fromtimestamp(found.at, timezone.utc)
+ self.handle.write(
+ f"{when.strftime('%Y-%m-%d %H:%M:%S')} "
+ f"{found.snr_db:>4.0f} {found.offset:>5.1f} "
+ f"{found.hertz:>7.1f} ~ {found.text}\n")
+ self.handle.flush()
+ self.lines += 1
+
+ def close(self) -> None:
+ if self.handle is not None:
+ self.handle.close()
+ self.handle = None
+
+
+def open_log(directory, started: float, frequency: float,
+ band: str = "") -> Ft8Log | None:
+ """A new log for this run, or nothing if it cannot be opened."""
+ when = datetime.fromtimestamp(started, timezone.utc)
+ path = Path(directory) / f"ft8-{when.strftime('%Y%m%d-%H%M%S')}.txt"
+ try:
+ path.parent.mkdir(parents=True, exist_ok=True)
+ handle = open(path, "w")
+ except OSError:
+ return None
+ handle.write(f"# bandsaunter {__version__} FT8 log v{LOG_VERSION}\n")
+ handle.write(f"# {frequency / 1e6:g} MHz"
+ f"{(' ' + band) if band else ''}, times are UTC\n")
+ handle.write("# when snr dt hz ~ message\n")
+ handle.flush()
+ return Ft8Log(path=path, handle=handle)
+
+
+def read_log(path) -> list[dict]:
+ """A log back off the disk, for reading an evening again.
+
+ Deliberately forgiving: a line that cannot be parsed is skipped rather
+ than raising, because a log is often read while it is still being
+ written and the last line may be half there.
+ """
+ out = []
+ try:
+ text = Path(path).read_text()
+ except OSError:
+ return out
+ for line in text.splitlines():
+ if line.startswith("#") or "~" not in line:
+ continue
+ head, message = line.split("~", 1)
+ parts = head.split()
+ if len(parts) < 5:
+ continue
+ try:
+ when = datetime.strptime(f"{parts[0]} {parts[1]}",
+ "%Y-%m-%d %H:%M:%S")
+ out.append({"at": when.replace(tzinfo=timezone.utc).timestamp(),
+ "snr_db": float(parts[2]), "offset": float(parts[3]),
+ "hertz": float(parts[4]), "text": message.strip()})
+ except ValueError:
+ continue
+ return out
+
+
+def write_csv(path, decodes) -> Path | None:
+ try:
+ with open(path, "w", newline="") as fh:
+ writer = csv.writer(fh)
+ writer.writerow(["utc", "snr_db", "offset_s", "hertz", "kind",
+ "calls", "grid", "report", "cq", "message"])
+ for d in decodes:
+ when = datetime.fromtimestamp(d.at, timezone.utc)
+ writer.writerow([when.strftime("%Y-%m-%d %H:%M:%S"),
+ f"{d.snr_db:.0f}", f"{d.offset:.1f}",
+ f"{d.hertz:.1f}", d.kind,
+ " ".join(d.calls), d.grid, d.report,
+ "yes" if d.calling else "", d.text])
+ except OSError:
+ return None
+ return Path(path)
+
+
+def _adif(field: str, value: str) -> str:
+ value = str(value)
+ return f"<{field}:{len(value)}>{value} "
+
+
+def write_adif(path, decodes, band: str = "", frequency: float = 0.0,
+ my_grid: str = "") -> Path | None:
+ """The decodes as ADIF records of stations heard.
+
+ One record per station rather than per decode: a logging program that
+ was handed forty identical records for one evening's CQ calls would be
+ right to complain. The strongest report is the one kept, that being
+ the one worth passing on.
+ """
+ best: dict = {}
+ for d in decodes:
+ sender = d.calls[1] if len(d.calls) > 1 else (
+ d.calls[0] if d.calls else "")
+ if not sender or sender == "<...>":
+ continue
+ was = best.get(sender)
+ if was is None or d.snr_db > was.snr_db:
+ best[sender] = d
+ if d.grid and not getattr(best[sender], "grid", ""):
+ best[sender].grid = d.grid
+ try:
+ with open(path, "w") as fh:
+ fh.write(f"bandsaunter {__version__} — FT8 stations heard.\n")
+ fh.write("These are receptions, not contacts: this program "
+ "does not transmit.\n")
+ fh.write(_adif("ADIF_VER", "3.1.4") + "\n")
+ fh.write(_adif("PROGRAMID", "bandsaunter") + "\n")
+ fh.write(_adif("PROGRAMVERSION", __version__) + "\n")
+ fh.write("\n")
+ for call, d in sorted(best.items()):
+ when = datetime.fromtimestamp(d.at, timezone.utc)
+ row = (_adif("CALL", call)
+ + _adif("QSO_DATE", when.strftime("%Y%m%d"))
+ + _adif("TIME_ON", when.strftime("%H%M%S"))
+ + _adif("MODE", "FT8"))
+ if band:
+ row += _adif("BAND", band)
+ if frequency:
+ row += _adif("FREQ", f"{frequency / 1e6:.6f}")
+ if d.grid:
+ row += _adif("GRIDSQUARE", d.grid)
+ if my_grid:
+ row += _adif("MY_GRIDSQUARE", my_grid.upper())
+ row += _adif("RST_RCVD", f"{d.snr_db:.0f}")
+ # Said plainly in the record itself, so that a log this is
+ # imported into cannot quietly become a claim.
+ row += _adif("COMMENT", "heard only, not worked")
+ row += _adif("QSO_RANDOM", "Y")
+ fh.write(row + "\n")
+ except OSError:
+ return None
+ return Path(path)
diff --git a/bandsaunter/ft8sim.py b/bandsaunter/ft8sim.py
new file mode 100644
index 0000000..4711cdb
--- /dev/null
+++ b/bandsaunter/ft8sim.py
@@ -0,0 +1,172 @@
+"""A made-up FT8 band, on a real clock.
+
+For trying the display, the report and the exports without an aerial, and
+for testing the whole path from samples to tables. It stands in for a
+receiver: same `tune`, `read_samples` and `close`, same sample rate, same
+complex samples.
+
+What it cannot tell you is whether your receiver hears anything, which is
+the one question a simulator is never allowed to answer. It is also the
+lesson this project learned expensively on another band: an encoder tested
+against its own decoder agrees with it about anything they are both wrong
+about. So the decoding is tested against real off-air recordings; this is
+here for the parts above the decoding.
+"""
+
+from __future__ import annotations
+
+import math
+import time
+
+import numpy as np
+
+from . import ft8code as code
+from . import ft8wave as wave
+
+__all__ = ["SimulatedBand", "CALLS", "GRIDS"]
+
+# A plausible spread: a few continents, a few strengths, a few habits.
+CALLS = ("G4UJS", "SP4FCA", "IK4LZH", "EA1ABT", "DL1UDO", "OH8GDU",
+ "JA2GQT", "VK4BLE", "RA6ABO", "F4FSY", "K1ABC", "W2XYZ",
+ "PA3EPP", "SM0ABC", "LZ1LZ", "9A9TT", "ZS6ABC", "LU1AAA")
+GRIDS = ("IO83", "KO03", "JN54", "IN73", "JO31", "KP24",
+ "PM85", "QG62", "KN96", "JN25", "FN42", "FN20",
+ "JO21", "JO99", "KN12", "JN75", "KG33", "GF05")
+
+
+class SimulatedBand:
+ """A receiver that invents a band of FT8 stations.
+
+ Paced against the real clock, because everything about FT8 is: the
+ slots are quarter-minutes of UTC, and a simulator that produced fifteen
+ seconds of audio instantly would make the section look as though it
+ worked while testing none of the timing that makes it work.
+ """
+
+ def __init__(self, rate: float = 240_000.0, seed: int = 0,
+ stations: int = 12, realtime: bool = True,
+ clock=time.time, sleep=time.sleep):
+ self.rate = float(rate)
+ self.frequency = 0.0
+ self.rng = np.random.default_rng(seed)
+ self.stations = max(1, int(stations))
+ self.realtime = realtime
+ self.clock = clock
+ self.sleep = sleep
+ self.closed = False
+ self._audio_rate = 12_000
+ self._made: dict[float, np.ndarray] = {}
+ self._one_sided: dict[float, np.ndarray] = {}
+ self._at = None
+
+ # -- the receiver's shape -------------------------------------------
+ def tune(self, hz: float) -> None:
+ self.frequency = float(hz)
+
+ def close(self) -> None:
+ self.closed = True
+
+ def read_samples(self, count: int):
+ """A block of samples, paced to take as long as it would on the air."""
+ if self.closed:
+ return None
+ now = self.clock()
+ if self._at is None:
+ self._at = now
+ wanted = count / self.rate
+ if self.realtime:
+ behind = self._at + wanted - self.clock()
+ if behind > 0:
+ self.sleep(behind)
+ block = self._samples_for(self._at, count)
+ self._at += wanted
+ return block
+
+ # -- making the band ------------------------------------------------
+ def _slot_audio(self, slot_at: float) -> np.ndarray:
+ """One slot of the band, at twelve kilohertz, made once and kept."""
+ have = self._made.get(slot_at)
+ if have is not None:
+ return have
+ rng = np.random.default_rng(int(slot_at) & 0xFFFFFFFF)
+ total = int(self._audio_rate * code.SLOT_S)
+ audio = rng.standard_normal(total).astype(np.float32) * 0.02
+ # Stations pick a frequency and keep it, mostly, and take turns.
+ even = int(slot_at / code.SLOT_S) % 2 == 0
+ for i in range(self.stations):
+ if (i % 2 == 0) != even:
+ continue
+ call = CALLS[(i + int(slot_at / code.SLOT_S) // 2) % len(CALLS)]
+ grid = GRIDS[CALLS.index(call)]
+ other = CALLS[(i * 7 + 3) % len(CALLS)]
+ pick = rng.integers(0, 3)
+ if pick == 0:
+ text = f"CQ {call} {grid}"
+ elif pick == 1:
+ text = f"{other} {call} {grid}"
+ else:
+ text = f"{other} {call} {rng.integers(-24, 5):+03d}"
+ hertz = 300.0 + i * 190.0 + float(rng.integers(-30, 30))
+ strength = float(rng.uniform(-18.0, 10.0))
+ one = wave.transmit(text, rate=float(self._audio_rate),
+ hertz=hertz,
+ offset=0.5 + float(rng.uniform(-0.3, 0.6)),
+ seconds=code.SLOT_S)
+ scale = 10.0 ** (strength / 20.0) * 0.05
+ audio[:one.size] += (one * scale).astype(np.float32)
+ if len(self._made) > 2:
+ self._made.clear()
+ self._made[slot_at] = audio
+ return audio
+
+ def _samples_for(self, at: float, count: int):
+ """Complex samples covering ``count`` samples from ``at``.
+
+ The band is made as audio and then put back on a carrier, which is
+ the reverse of what the section does to it. One-sided, so that
+ what comes out of the demodulator is what went in: a real audio
+ tone at 1000 Hz has to arrive as a complex tone at +1000 Hz, or
+ upper sideband and lower sideband would be the same thing.
+
+ Held at the audio rate and stepped through, rather than properly
+ resampled. The images that leaves sit at multiples of twelve
+ kilohertz and the demodulator's own decimation filter removes them
+ -- which is worth saying out loud, because it means this simulator
+ leans on the thing it is being used to test. That is tolerable for
+ the display and the report, and is exactly why the decoding itself
+ is tested against real recordings instead.
+ """
+ out = np.zeros(count, dtype=np.complex64)
+ step = 1.0 / self.rate
+ ratio = self._audio_rate / self.rate
+ for k in range(0, count, 4096):
+ piece = min(4096, count - k)
+ when = at + k * step
+ slot_at = math.floor(when / code.SLOT_S) * code.SLOT_S
+ audio = self._analytic_slot(slot_at)
+ into = (when - slot_at) * self._audio_rate
+ idx = np.clip((into + np.arange(piece) * ratio).astype(np.int64),
+ 0, audio.size - 1)
+ out[k:k + piece] = audio[idx].astype(np.complex64)
+ return out
+
+ def _analytic_slot(self, slot_at: float) -> np.ndarray:
+ have = self._one_sided.get(slot_at)
+ if have is None:
+ have = _analytic(self._slot_audio(slot_at))
+ # Two slots kept: a block of samples commonly straddles a
+ # boundary, and rebuilding a slot for every such block is most
+ # of the simulator's time.
+ if len(self._one_sided) > 2:
+ self._one_sided.clear()
+ self._one_sided[slot_at] = have
+ return have
+
+
+def _analytic(real_audio: np.ndarray) -> np.ndarray:
+ """The one-sided version of a real signal: no energy below zero."""
+ spectrum = np.fft.fft(real_audio)
+ n = spectrum.size
+ spectrum[n // 2 + 1:] = 0.0
+ spectrum[1:(n + 1) // 2] *= 2.0
+ return np.fft.ifft(spectrum)
diff --git a/bandsaunter/ft8tables.py b/bandsaunter/ft8tables.py
new file mode 100644
index 0000000..20286be
--- /dev/null
+++ b/bandsaunter/ft8tables.py
@@ -0,0 +1,219 @@
+"""The two fixed tables the FT8 code is defined by.
+
+Everything else in this section is worked out from first principles -- the
+tones, the sync, the parity arithmetic, the way a callsign is squeezed into
+twenty-eight bits. These two cannot be, because they are not derived from
+anything. They are the code, chosen once by its designers and published, and
+a receiver that guessed at them would be speaking a different protocol.
+
+Taken from ft8_lib (https://github.com/kgoba/ft8_lib), MIT licensed,
+Copyright (c) 2018 K\u0101rlis Goba, which took them in turn from WSJT-X.
+Reproduced here under that licence; the MIT terms permit it and this file
+carries the attribution they ask for. Nothing else in bandsaunter comes from
+there: the decoder below is its own.
+
+GENERATOR is the parity half of the systematic generator: row m gives the
+message bits that are exclusive-ored to make parity bit m, as ninety-one bits
+in twelve bytes, most significant bit first.
+
+CHECKS is the sparse parity-check matrix, as the bit positions each of the
+eighty-three checks covers, one-based as published. It is *not* derivable
+from GENERATOR even though both describe the same code: the generator's
+parity half runs to a weight of fifty-odd bits per row, and belief
+propagation on a matrix that dense says nothing useful. The sparse one is a
+different basis for the same dual space, six or seven bits to a check, and it
+is the one the decoding works on. That they agree is checked by a test
+rather than assumed -- encode at random, and every sparse check must come out
+even.
+"""
+
+__all__ = ["GENERATOR", "CHECKS", "COSTAS", "GRAY", "BITS", "PAYLOAD",
+ "PARITY", "TONES", "CRC_POLYNOMIAL", "CRC_BITS"]
+
+# The 7x7 Costas array that starts, middles and ends every transmission.
+COSTAS = (3, 1, 4, 0, 6, 5, 2)
+
+# Three bits to a tone, in Gray order, so that mistaking a tone for its
+# neighbour costs one bit rather than three.
+GRAY = (0, 1, 3, 2, 5, 6, 4, 7)
+
+BITS = 174 # bits in a codeword
+PAYLOAD = 91 # of which are message and checksum
+PARITY = BITS - PAYLOAD
+TONES = 79 # channel symbols, sync included
+
+# CRC-14, without the leading bit.
+CRC_POLYNOMIAL = 0x2757
+CRC_BITS = 14
+
+GENERATOR: tuple[str, ...] = (
+ "8329CE11BF31EAF509F27FC0",
+ "761C264E25C2593354931320",
+ "DC265902FB277C6410A1BDC0",
+ "1B3F417858CD2DD33EC7F620",
+ "09FDA4FEE04195FD034783A0",
+ "077CCCC11B8873ED5C3D48A0",
+ "29B62AFE3CA036F4FE1A9DA0",
+ "6054FAF5F35D96D3B0C8C3E0",
+ "E20798E4310EED27884AE900",
+ "775C9C08E80E26DDAE563180",
+ "B0B811028C2BF997213487C0",
+ "18A0C9231FC60ADF5C5EA320",
+ "76471E8302A0721E01B12B80",
+ "FFBCCB80CA8341FAFB47B2E0",
+ "66A72A158F9325A2BF671700",
+ "C4243689FE85B1C51363A180",
+ "0DFF739414D1A1B34B1C2700",
+ "15B48830636C8B99894972E0",
+ "29A89C0D3DE81D665489B0E0",
+ "4F126F37FA51CBE61BD6B940",
+ "99C47239D0D97D3C84E09400",
+ "1919B75119765621BB4F1E80",
+ "09DB12D731FAEE0B86DF6B80",
+ "488FC33DF43FBDEEA4EAFB40",
+ "827423EE40B675F756EB5FE0",
+ "ABE197C484CB74757144A9A0",
+ "2B500E4BC0EC5A6D2BDBDD00",
+ "C474AA53D702187616693600",
+ "8EBA1A13DB3390BD6718CEC0",
+ "753844673A27782CC42012E0",
+ "06FF83A145C37035A5C12680",
+ "3B37417858CC2DD33EC3F620",
+ "9A4A5A28EE17CA9C324842C0",
+ "BC29F465309C977E89610A40",
+ "2663AE6DDF8B5CE2BB294880",
+ "46F231EFE457034C18144180",
+ "3FB2CE85ABE9B0C72E06FBE0",
+ "DE87481F282C153971A0A2E0",
+ "FCD7CCF23C69FA99BBA14120",
+ "F0261447E9490CA8E474CEC0",
+ "4410115818196F95CDD70120",
+ "088FC31DF4BFBDE2A4EAFB40",
+ "B8FEF1B6307729FB0A078C00",
+ "5AFEA7ACCCB77BBC9D99A900",
+ "49A7016AC653F65ECDC90760",
+ "1944D085BE4E7DA8D6CC7D00",
+ "251F62ADC4032F0EE7140020",
+ "56471F8702A0721E00B12B80",
+ "2B8E4923F2DD51E2D537FA00",
+ "6B550A40A66F4755DE95C260",
+ "A18AD28D4E27FE92A4F6C840",
+ "10C2E586388CB82A3D807580",
+ "EF34A41817EE02133DB2EB00",
+ "7E9C0C54325A9C15836E0000",
+ "3693E572D1FDE4CDF079E860",
+ "BFB2CEC5ABE1B0C72E07FBE0",
+ "7EE18230C583CCCC57D4B080",
+ "A066CB2FEDAFC9F526641260",
+ "BB23725ABC47CC5F4CC4CD20",
+ "DED9DBA3BEE40C59B5609B40",
+ "D9A7016AC653E6DECDC90360",
+ "9AD46AED5F707F280AB5FC40",
+ "E5921C77822587316D7D3C20",
+ "4F14DA8242A8B86DCA733520",
+ "8B8B507AD467D4441DF770E0",
+ "22831C9CF1169467AD04B680",
+ "213B838FE2AE54C38EE71800",
+ "5D926B6DD71F085181A4E120",
+ "66AB79D4B29EE6E69509E560",
+ "958148682D748A38DD68BAA0",
+ "B8CE020CF069C32A723AB140",
+ "F4331D6D461607E957527460",
+ "6DA23BA424B9596133CF9C80",
+ "A636BCBC7B30C5FBEAE67FE0",
+ "5CB0D86A07DF654A9089A200",
+ "F11F106848780FC9ECDD80A0",
+ "1FBB5364FB8D2C9D730D5BA0",
+ "FCB86BC70A50C9D02A5D0340",
+ "A534433029EAC15F322E34C0",
+ "C989D9C7C3D3B8C55D751300",
+ "7BB38B2F0186D46643AE9620",
+ "2644EBADEB44B9467D1F42C0",
+ "608CC857594BFBB55D696000",
+)
+
+CHECKS: tuple[tuple[int, ...], ...] = (
+ (4, 31, 59, 91, 92, 96, 153),
+ (5, 32, 60, 93, 115, 146),
+ (6, 24, 61, 94, 122, 151),
+ (7, 33, 62, 95, 96, 143),
+ (8, 25, 63, 83, 93, 96, 148),
+ (6, 32, 64, 97, 126, 138),
+ (5, 34, 65, 78, 98, 107, 154),
+ (9, 35, 66, 99, 139, 146),
+ (10, 36, 67, 100, 107, 126),
+ (11, 37, 67, 87, 101, 139, 158),
+ (12, 38, 68, 102, 105, 155),
+ (13, 39, 69, 103, 149, 162),
+ (8, 40, 70, 82, 104, 114, 145),
+ (14, 41, 71, 88, 102, 123, 156),
+ (15, 42, 59, 106, 123, 159),
+ (1, 33, 72, 106, 107, 157),
+ (16, 43, 73, 108, 141, 160),
+ (17, 37, 74, 81, 109, 131, 154),
+ (11, 44, 75, 110, 121, 166),
+ (45, 55, 64, 111, 130, 161, 173),
+ (8, 46, 71, 112, 119, 166),
+ (18, 36, 76, 89, 113, 114, 143),
+ (19, 38, 77, 104, 116, 163),
+ (20, 47, 70, 92, 138, 165),
+ (2, 48, 74, 113, 128, 160),
+ (21, 45, 78, 83, 117, 121, 151),
+ (22, 47, 58, 118, 127, 164),
+ (16, 39, 62, 112, 134, 158),
+ (23, 43, 79, 120, 131, 145),
+ (19, 35, 59, 73, 110, 125, 161),
+ (20, 36, 63, 94, 136, 161),
+ (14, 31, 79, 98, 132, 164),
+ (3, 44, 80, 124, 127, 169),
+ (19, 46, 81, 117, 135, 167),
+ (7, 49, 58, 90, 100, 105, 168),
+ (12, 50, 61, 118, 119, 144),
+ (13, 51, 64, 114, 118, 157),
+ (24, 52, 76, 129, 148, 149),
+ (25, 53, 69, 90, 101, 130, 156),
+ (20, 46, 65, 80, 120, 140, 170),
+ (21, 54, 77, 100, 140, 171),
+ (35, 82, 133, 142, 171, 174),
+ (14, 30, 83, 113, 125, 170),
+ (4, 29, 68, 120, 134, 173),
+ (1, 4, 52, 57, 86, 136, 152),
+ (26, 51, 56, 91, 122, 137, 168),
+ (52, 84, 110, 115, 145, 168),
+ (7, 50, 81, 99, 132, 173),
+ (23, 55, 67, 95, 172, 174),
+ (26, 41, 77, 109, 141, 148),
+ (2, 27, 41, 61, 62, 115, 133),
+ (27, 40, 56, 124, 125, 126),
+ (18, 49, 55, 124, 141, 167),
+ (6, 33, 85, 108, 116, 156),
+ (28, 48, 70, 85, 105, 129, 158),
+ (9, 54, 63, 131, 147, 155),
+ (22, 53, 68, 109, 121, 174),
+ (3, 13, 48, 78, 95, 123),
+ (31, 69, 133, 150, 155, 169),
+ (12, 43, 66, 89, 97, 135, 159),
+ (5, 39, 75, 102, 136, 167),
+ (2, 54, 86, 101, 135, 164),
+ (15, 56, 87, 108, 119, 171),
+ (10, 44, 82, 91, 111, 144, 149),
+ (23, 34, 71, 94, 127, 153),
+ (11, 49, 88, 92, 142, 157),
+ (29, 34, 87, 97, 147, 162),
+ (30, 50, 60, 86, 137, 142, 162),
+ (10, 53, 66, 84, 112, 128, 165),
+ (22, 57, 85, 93, 140, 159),
+ (28, 32, 72, 103, 132, 166),
+ (28, 29, 84, 88, 117, 143, 150),
+ (1, 26, 45, 80, 128, 147),
+ (17, 27, 89, 103, 116, 153),
+ (51, 57, 98, 163, 165, 172),
+ (21, 37, 73, 138, 152, 169),
+ (16, 47, 76, 130, 137, 154),
+ (3, 24, 30, 72, 104, 139),
+ (9, 40, 90, 106, 134, 151),
+ (15, 58, 60, 74, 111, 150, 163),
+ (18, 42, 79, 144, 146, 152),
+ (25, 38, 65, 99, 122, 160),
+ (17, 42, 75, 129, 170, 172),
+)
diff --git a/bandsaunter/ft8wave.py b/bandsaunter/ft8wave.py
new file mode 100644
index 0000000..d142a04
--- /dev/null
+++ b/bandsaunter/ft8wave.py
@@ -0,0 +1,411 @@
+"""Finding FT8 in fifteen seconds of audio.
+
+The coding is in ft8code; this is the part that has to deal with the air.
+A slot of audio three kilohertz wide holds anything from nothing to forty
+transmissions, each fifty hertz wide, each starting when its operator's
+clock said to rather than when ours did, and most of them below the noise.
+
+The shape of the search:
+
+ 1. a waterfall -- the whole slot as power against time and frequency, at
+ half a symbol and half a tone, so nothing falls between two stools;
+ 2. the sync search -- every place a Costas array could be, scored;
+ 3. for each candidate, the eight tone powers of each of seventy-nine
+ symbols, turned into how strongly each bit is believed;
+ 4. the error correction, which either converges or does not;
+ 5. the checksum, which decides whether to believe it.
+
+Steps four and five are why this works at all. Nothing up to step three is
+clever enough to read a signal that is twenty decibels under the noise; what
+it produces is a wash of barely-tilted opinions, and the code turns a
+hundred and seventy-four of those into ninety-one bits of certainty -- or
+into nothing, which is the other important answer.
+"""
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass
+
+import numpy as np
+
+from . import ft8code as code
+from .ft8code import COSTAS, SYMBOL_S, TONES
+
+__all__ = ["Waterfall", "Candidate", "Decode", "listen_to", "waterfall",
+ "candidates", "soft_bits", "TIME_STEPS", "FREQ_STEPS"]
+
+# Half a symbol and half a tone. A transmission that started a quarter of a
+# symbol late, or sits a quarter of a tone off, is still caught: with steps
+# a whole symbol wide the worst case loses most of the signal to the gap
+# between two bins, and the worst case is common because nobody's clock and
+# nobody's dial agree with ours.
+TIME_STEPS = 2
+FREQ_STEPS = 2
+
+SYNC_AT = (0, 36, 72) # where the three Costas arrays sit, in symbols
+
+# A perfectly-timed station does not start at the top of the slot. The
+# transmission is 12.64 seconds long in a slot of fifteen, and the
+# convention every FT8 program follows is to begin half a second in and to
+# report lateness against that -- so a station whose clock is right reports
+# as dt 0.0 rather than dt 0.5. Followed here so the figures mean the same
+# thing as everybody else's.
+NOMINAL_START = 0.5
+
+
+@dataclass
+class Waterfall:
+ """The slot as power against time and frequency."""
+
+ power: np.ndarray # (times, bins), already in decibels
+ rate: float
+ hz_per_bin: float
+ seconds_per_step: float
+
+ @property
+ def times(self) -> int:
+ return self.power.shape[0]
+
+ @property
+ def bins(self) -> int:
+ return self.power.shape[1]
+
+
+@dataclass
+class Candidate:
+ """Somewhere a transmission might start, and how much it looks like one."""
+
+ step: int # in half-symbols from the top of the slot
+ bin: int # in half-tones up the waterfall
+ score: float
+
+ def seconds(self, fall: Waterfall) -> float:
+ """How late this started, against the nominal start of sending."""
+ return self.step * fall.seconds_per_step - NOMINAL_START
+
+ def hertz(self, fall: Waterfall) -> float:
+ return self.bin * fall.hz_per_bin
+
+
+@dataclass
+class Decode:
+ """One transmission, read."""
+
+ text: str = ""
+ kind: str = ""
+ calls: tuple = ()
+ grid: str = ""
+ report: str = ""
+ calling: bool = False
+ hertz: float = 0.0
+ offset: float = 0.0 # how late it started, in seconds
+ snr_db: float = 0.0
+ score: float = 0.0
+ at: float = 0.0 # when the slot began, as a clock time
+
+
+def waterfall(audio, rate: float) -> Waterfall:
+ """The slot, as power against time and frequency.
+
+ One symbol of samples to a row, stepped half a symbol at a time, and
+ zero-padded to twice its length before the transform so the bins come
+ out half a tone apart. Padding rather than a longer window on purpose:
+ a longer window would average two symbols together, and the thing being
+ looked for changes every symbol.
+ """
+ audio = np.asarray(audio, dtype=np.float32)
+ per_symbol = int(round(rate * SYMBOL_S))
+ if per_symbol < 8:
+ raise ValueError("the audio rate is too low to hold FT8 tones")
+ step = per_symbol // TIME_STEPS
+ size = per_symbol * FREQ_STEPS
+ window = np.hanning(per_symbol).astype(np.float32)
+
+ rows = 1 + max(0, (audio.size - per_symbol) // step)
+ if rows < 1:
+ raise ValueError("not enough audio for a single symbol")
+ # One strided view of the whole slot, transformed in a single call:
+ # a loop over two hundred rows in Python is most of the decode time.
+ frames = np.lib.stride_tricks.sliding_window_view(
+ audio, per_symbol)[::step][:rows]
+ spectrum = np.fft.rfft(frames * window, n=size, axis=1)
+ power = np.abs(spectrum).astype(np.float32)
+ # Decibels, because everything downstream adds and subtracts these and
+ # the useful comparisons are all ratios.
+ floor = max(float(np.median(power)) * 1e-4, 1e-12)
+ power = 20.0 * np.log10(np.maximum(power, floor))
+ return Waterfall(power=power, rate=float(rate),
+ hz_per_bin=rate / size,
+ seconds_per_step=step / rate)
+
+
+def _sync_offsets():
+ """Where the twenty-one sync tones sit, relative to a candidate."""
+ steps, bins = [], []
+ for block in SYNC_AT:
+ for k, tone in enumerate(COSTAS):
+ steps.append((block + k) * TIME_STEPS)
+ bins.append(tone * FREQ_STEPS)
+ return np.array(steps), np.array(bins)
+
+
+SYNC_STEP, SYNC_BIN = _sync_offsets()
+
+
+def candidates(fall: Waterfall, most: int = 300, lowest: float = 200.0,
+ highest: float = 3000.0, earliest: float = -2.5,
+ latest: float = 3.5) -> list[Candidate]:
+ """Every place a transmission plausibly starts, best first.
+
+ Scored as how far the twenty-one sync tones stand above the other seven
+ tones at the same instants. Not above the noise floor of the band --
+ that would rank a strong signal's neighbours above a weak signal, and
+ the weak ones are the whole point of the protocol.
+
+ Only local peaks are kept. A real transmission lights up every
+ neighbouring offset as well, and without this one loud station would
+ fill the list and crowd out forty quiet ones.
+ """
+ power = fall.power
+ span = TONES * TIME_STEPS
+ first = int(math.floor(earliest / fall.seconds_per_step))
+ last = int(math.ceil(latest / fall.seconds_per_step))
+ starts = np.arange(first, last + 1)
+ starts = starts[(starts >= 0) | (starts + SYNC_STEP.min() >= 0)]
+ starts = starts[starts + span <= power.shape[0]]
+ starts = starts[starts >= 0]
+ low = max(0, int(lowest / fall.hz_per_bin))
+ high = min(power.shape[1] - 8 * FREQ_STEPS,
+ int(highest / fall.hz_per_bin))
+ if starts.size == 0 or high <= low:
+ return []
+ tops = np.arange(low, high)
+
+ # The sync tones, and all eight tones at the same instants, for every
+ # (start, frequency) at once.
+ times = starts[:, None] + SYNC_STEP[None, :] # (S, 21)
+ here = power[times] # (S, 21, bins)
+ want = tops[None, None, :] + SYNC_BIN[None, :, None]
+ sync = np.take_along_axis(here, want, axis=2).mean(axis=1) # (S, tops)
+ whole = np.zeros_like(sync)
+ for tone in range(8):
+ idx = tops[None, None, :] + tone * FREQ_STEPS
+ whole += np.take_along_axis(here, idx, axis=2).mean(axis=1)
+ score = sync - whole / 8.0
+
+ # Local peaks only, in both directions.
+ keep = np.ones_like(score, dtype=bool)
+ keep[1:, :] &= score[1:, :] >= score[:-1, :]
+ keep[:-1, :] &= score[:-1, :] >= score[1:, :]
+ keep[:, 1:] &= score[:, 1:] >= score[:, :-1]
+ keep[:, :-1] &= score[:, :-1] >= score[:, 1:]
+ rows, cols = np.nonzero(keep)
+ values = score[rows, cols]
+ order = np.argsort(values)[::-1][:most]
+ return [Candidate(step=int(starts[rows[i]]), bin=int(tops[cols[i]]),
+ score=float(values[i])) for i in order]
+
+
+def soft_bits(fall: Waterfall, spot: Candidate) -> np.ndarray:
+ """How strongly each of the hundred and seventy-four bits is a zero.
+
+ Every symbol gives eight tone powers. A bit is a zero in four of the
+ eight tones and a one in the other four, so its evidence is the best of
+ the four against the best of the other four -- in decibels, which is
+ already a log-likelihood up to a scale factor, and the scale does not
+ matter to the min-sum decoding downstream.
+
+ Taking only the loudest tone and calling that three bits, which is the
+ obvious thing to do, throws away exactly the information the error
+ correction runs on, and decodes almost nothing.
+ """
+ power = fall.power
+ steps = spot.step + np.arange(TONES) * TIME_STEPS
+ bins = spot.bin + np.arange(8) * FREQ_STEPS
+ if steps[-1] >= power.shape[0] or bins[-1] >= power.shape[1]:
+ return np.zeros(code.BITS, dtype=np.float32)
+ grid = power[np.ix_(steps, bins)] # (79, 8)
+
+ # Sync symbols carry nothing; drop them and put the tones back in the
+ # order the bits were in before the Gray mapping.
+ data = np.array([i for i in range(TONES)
+ if not (i < 7 or 36 <= i < 43 or 72 <= i)])
+ values = grid[data][:, list(code.GRAY)] # (58, 8) by value
+ out = np.zeros(code.BITS, dtype=np.float32)
+ for b in range(3):
+ mask = (np.arange(8) >> (2 - b)) & 1
+ zero = values[:, mask == 0].max(axis=1)
+ one = values[:, mask == 1].max(axis=1)
+ out[b::3] = zero - one
+ return out
+
+
+# Worked out by measuring against transmissions of known strength and
+# against off-air recordings with published readings, not derived. Turning
+# a ratio of bins into the decibels-in-2500-Hz everybody quotes depends on
+# the window, the padding and the shape of the noise, and calibrating it is
+# both easier and more honest than deriving it and hoping.
+SNR_TRIM = -37.0
+
+# Making the transmitter above label its signals the way the reading below
+# reports them. The reading is the one tied to reality -- it matches the
+# published figures for the off-air recordings to within half a decibel
+# across a hundred of them -- and a bare ratio of signal power to noise in
+# 2500 Hz comes out eight decibels away from it, consistently enough that
+# the difference is a definition rather than an error. Whose definition is
+# "right" is not a question this can settle; what matters is that a signal
+# built here at -15 reads as -15, so a test of sensitivity means what it
+# says.
+SNR_MATCHES = 10.0 ** (8.0 / 10.0)
+SNR_NEAR_HZ = 200.0 # how far either side to look for the noise
+SNR_PERCENTILE = 30 # low enough to ignore the neighbours
+
+
+def _snr(fall: Waterfall, spot: Candidate, tones) -> float:
+ """A signal-to-noise in the usual 2500 Hz reference bandwidth.
+
+ Measured in the tone that was actually sent, which is known by the time
+ this is called because the message decoded. Taking the loudest of the
+ eight instead looks like the same thing and is not: the largest of
+ eight noisy numbers is well above their mean even when there is no
+ signal at all, so weak transmissions all read as though they were
+ stronger than they are, and the scale compresses where it matters.
+
+ The noise is measured beside the signal rather than across the band. A
+ receiver with a three-kilohertz passband has nothing above it but the
+ noise floor of the sound card, and a median taken over the whole
+ spectrum sits twelve to sixteen decibels below the real one -- which is
+ a receiver flattering every station it hears by that much.
+
+ Good to a few decibels, honestly: the readings here track the published
+ ones for the same recordings with a correlation of about 0.83 and no
+ systematic bias between -25 and +10, which is the range that matters.
+ Very strong signals read a little low, as they do everywhere.
+ """
+ power = (10.0 ** (fall.power / 20.0)) ** 2
+ steps = spot.step + np.arange(TONES) * TIME_STEPS
+ live = (steps >= 0) & (steps < power.shape[0])
+ steps, sent = steps[live], np.asarray(tones)[live]
+ bins = spot.bin + np.arange(8) * FREQ_STEPS
+ if steps.size == 0 or bins[-1] >= power.shape[1]:
+ return -99.0
+ near = int(SNR_NEAR_HZ / fall.hz_per_bin)
+ lo = max(0, spot.bin - near)
+ high = min(power.shape[1], bins[-1] + near)
+ columns = np.arange(lo, high)
+ # A guard either side of the signal: its own skirts are not noise.
+ beside = ((columns < spot.bin - 2 * FREQ_STEPS)
+ | (columns > bins[-1] + 2 * FREQ_STEPS))
+ if beside.sum() < 8:
+ return -99.0
+ noise = float(np.percentile(power[np.ix_(steps, columns[beside])],
+ SNR_PERCENTILE))
+ grid = power[np.ix_(steps, bins)]
+ here = float(grid[np.arange(grid.shape[0]), sent].mean())
+ if noise <= 0 or here <= noise:
+ return -99.0
+ return round(10.0 * math.log10((here - noise) / noise) + SNR_TRIM, 0)
+
+
+def listen_to(audio, rate: float, book=None, most: int = 300,
+ rounds: int = 30, at: float = 0.0) -> list[Decode]:
+ """Everything decodable in one slot of audio.
+
+ Candidates are decoded in one batch rather than one at a time: a busy
+ slot throws up two or three hundred of them, the error correction is
+ most of the work, and doing them together is the difference between a
+ slot that decodes in a second and one that does not keep up with the
+ air.
+ """
+ book = book if book is not None else code.CallBook()
+ fall = waterfall(audio, rate)
+ spots = candidates(fall, most=most)
+ if not spots:
+ return []
+
+ beliefs = np.stack([soft_bits(fall, spot) for spot in spots])
+ repaired, worked = code.repair(beliefs, rounds=rounds)
+
+ out: list[Decode] = []
+ seen: dict[str, Decode] = {}
+ for i, spot in enumerate(spots):
+ if not worked[i]:
+ continue
+ bits = repaired[i]
+ if not code.crc_holds(bits[:code.PAYLOAD]):
+ continue
+ message = code.unpack(bits[:code.MESSAGE_BITS], book)
+ if not message.text:
+ continue
+ found = Decode(text=message.text, kind=message.kind,
+ calls=message.calls, grid=message.grid,
+ report=message.report, calling=message.calling,
+ hertz=round(spot.hertz(fall), 1),
+ offset=round(spot.seconds(fall), 2),
+ snr_db=_snr(fall, spot, code.tones_of(bits)),
+ score=spot.score, at=at)
+ # The same transmission is found at several neighbouring offsets.
+ # Keep the one with the best sync, which is the one nearest the
+ # truth about where and when it actually was.
+ was = seen.get(found.text)
+ if was is None or found.score > was.score:
+ seen[found.text] = found
+ out = sorted(seen.values(), key=lambda d: d.hertz)
+ for found in out:
+ for call in found.calls:
+ book.remember(call)
+ return out
+
+
+def transmit(text: str, rate: float = 12000.0, hertz: float = 1000.0,
+ offset: float = 0.5, seconds: float = code.SLOT_S,
+ snr_db: float | None = None, seed: int = 0,
+ book=None) -> np.ndarray:
+ """A slot of audio with one transmission in it.
+
+ Here rather than in a test because the tests need it, the simulator
+ needs it, and something that generates the signal this module claims to
+ understand is the only way to ask a question of the decoder whose answer
+ is known in advance -- where a transmission started, to the sample.
+
+ It does not transmit anything anywhere. It builds samples.
+ """
+ payload = code.pack(text, book)
+ tones = code.tones_of(code.encode(code.with_crc(payload)))
+ per_symbol = int(round(rate * SYMBOL_S))
+ total = int(round(rate * seconds))
+ audio = np.zeros(total, dtype=np.float64)
+
+ # Continuous phase across the whole transmission. A tone that restarted
+ # its phase every symbol would splatter across the band, which is both
+ # unneighbourly on the air and not what a decoder should be tested with.
+ phase = 0.0
+ start = int(round(offset * rate))
+ for i, tone in enumerate(tones):
+ freq = hertz + float(tone) * code.SYMBOL_HZ
+ at = start + i * per_symbol
+ if at >= total or at + per_symbol <= 0:
+ phase += 2 * math.pi * freq * per_symbol / rate
+ continue
+ step = 2 * math.pi * freq / rate
+ angles = phase + step * np.arange(per_symbol)
+ phase = angles[-1] + step
+ first = max(0, at)
+ last = min(total, at + per_symbol)
+ audio[first:last] += np.sin(angles[first - at:last - at])
+
+ if snr_db is not None:
+ # The usual reference: the signal's own power against the noise in
+ # 2500 Hz, which is how every FT8 program quotes a report. The
+ # noise added here is white across the whole sampled band, so the
+ # part of it that lands in the reference 2500 Hz is that fraction
+ # of the total -- which is the step it is easy to get wrong, and
+ # getting it wrong makes a decoder look deaf when it is the test
+ # signal that was quietly attenuated.
+ rng = np.random.default_rng(seed)
+ signal = float(np.mean(audio[audio != 0] ** 2)) if audio.any() else 1.0
+ bandwidth = rate / 2.0
+ whole = (signal / (10.0 ** (snr_db / 10.0))
+ * (bandwidth / 2500.0) / SNR_MATCHES)
+ audio = audio + rng.standard_normal(total) * math.sqrt(whole)
+ return audio.astype(np.float32)
diff --git a/bandsaunter/tui.py b/bandsaunter/tui.py
index d6e18f8..a6fc964 100644
--- a/bandsaunter/tui.py
+++ b/bandsaunter/tui.py
@@ -24,7 +24,8 @@ from .config import (DEFAULT_CONFIG_DIR, DEFAULT_CONFIG_PATH,
from .ranges import RangeError, ScanRange, parse_frequency
__all__ = ["run_tui", "show_ranges", "settings_menu", "help_screen",
- "aircraft_menu", "weather_menu", "aprs_menu", "first_run_setup",
+ "aircraft_menu", "weather_menu", "aprs_menu", "ft8_menu",
+ "first_run_setup",
"TUIAbort"]
_BACK = ("", "b", "back", "q", "quit", "x")
@@ -1676,6 +1677,9 @@ def _main_loop(console: Console, cfg: ScanConfig) -> ScanConfig | None:
f" [cyan]7[/cyan] APRS (144 MHz packet) "
f"[grey62]positions, weather and messages from amateurs"
f"[/grey62]\n"
+ f" [cyan]8[/cyan] FT8 "
+ f"[grey62]fifteen-second slots, whole bands at once, mostly "
+ f"under the noise[/grey62]\n"
f" [cyan]h[/cyan] Help\n"
f" [cyan]s[/cyan] [bold green]Start scanning[/bold green]\n"
f" [cyan]q[/cyan] Quit\n")
@@ -1695,6 +1699,8 @@ def _main_loop(console: Console, cfg: ScanConfig) -> ScanConfig | None:
weather_menu(console, cfg)
elif choice == "7":
aprs_menu(console, cfg)
+ elif choice == "8":
+ ft8_menu(console, cfg)
elif choice in ("h", "?", "help"):
help_screen(console)
elif choice in ("s", "start", "go"):
@@ -1706,3 +1712,195 @@ def _main_loop(console: Console, cfg: ScanConfig) -> ScanConfig | None:
return cfg
elif choice in ("q", "quit", "exit"):
return None
+
+
+_FT8_INTRO = (
+ "[bold]FT8[/bold] is fifteen seconds of everybody at once. Every station "
+ "on the band transmits in the same quarter-minute slots, on the same "
+ "dial frequency, fifty hertz wide each, stacked across three kilohertz "
+ "of audio — so one receiver parked on one frequency hears the whole "
+ "band's worth of stations at the same time.\n\n"
+ "Most of them arrive [bold]below the noise[/bold], and decode anyway: "
+ "half of what is sent is error-correcting code, which is what buys a "
+ "mode that works twenty decibels under what an operator can hear.\n\n"
+ "Two things it needs. The [bold]clock[/bold] has to be right to a second "
+ "or two, because the slots are quarter-minutes of UTC and every station "
+ "on earth agrees about which one it is. And almost all the activity is "
+ "on [bold]shortwave[/bold], which a plain receiver of this kind cannot "
+ "reach without an upconverter or direct sampling — so the default here "
+ "is the two-metre channel, which it can.\n\n"
+ "Nothing here transmits. It listens, decodes and writes down."
+)
+
+
+def ft8_menu(console: Console, cfg: ScanConfig) -> None:
+ """Listen to FT8, without a command line."""
+ from . import ft8
+
+ options = ft8.load_options()
+ while True:
+ _rule(console, "FT8")
+ console.print(Panel(Text.from_markup(_FT8_INTRO),
+ border_style="blue", padding=(0, 1)))
+ _option_groups(console, options, ft8)
+ logs = ft8.logs_in(cfg.output_dir)
+ kept = "no logs yet" if not logs else \
+ f"{len(logs)} log{'s' if len(logs) != 1 else ''}"
+ console.print(
+ f"\n [cyan]l[/cyan] [bold green]Listen[/bold green]"
+ f" [grey62]{ft8.describe(options)}[/grey62]\n"
+ # The band is the setting that decides whether anything is heard
+ # at all, and whether the receiver can reach it, so it sits here
+ # rather than a level down among the gain and the sample rate.
+ f" [cyan]c[/cyan] Band / channel "
+ f"[bold]{ft8.band_text(options)}[/bold]\n"
+ f" [cyan]g[/cyan] Where you are "
+ f"[grey62]{options.grid.upper() or 'not set — no distances'}"
+ f"[/grey62]\n"
+ f" [cyan]r[/cyan] Read a log back [grey62]{kept} in "
+ f"{cfg.output_dir}[/grey62]\n"
+ f" [cyan]N[/cyan] open group N "
+ f"[grey62]or type part of an option's name to find it[/grey62]\n"
+ f" [cyan]s[/cyan] Save these as default "
+ f"[grey62]kept in {ft8.options_path()}[/grey62]\n"
+ f" [cyan]d[/cyan] Reset them\n"
+ f" [cyan]b[/cyan] Back\n")
+ answer = _ask(console, " choice", "l").strip().lower()
+
+ if answer in _BACK:
+ return
+ if answer in ("l", "listen", "p"):
+ _ft8_listen(console, cfg, options)
+ elif answer in ("c", "channel", "band"):
+ _ft8_band(console, options)
+ elif answer in ("g", "grid", "where"):
+ _ft8_grid(console, options)
+ elif answer in ("r", "read", "m"):
+ _ft8_read(console, cfg, options, logs)
+ elif answer == "s":
+ try:
+ where = ft8.save_options(options)
+ console.print(f" [green]saved to {where}[/green]")
+ except OSError as exc:
+ console.print(f" [red]could not save: {exc}[/red]")
+ elif answer == "d":
+ if _confirm(" reset every FT8 option"):
+ options = ft8.Ft8Options()
+ console.print(" [green]reset[/green]")
+ elif answer.isdigit() and 1 <= int(answer) <= len(ft8.OPTION_GROUPS):
+ _option_group_menu(console, options,
+ ft8.OPTION_GROUPS[int(answer) - 1], ft8)
+ elif answer.lstrip("?").strip().isdigit():
+ _edit_option(console, options, answer, ft8)
+ elif answer:
+ found = _find_options(answer, ft8)
+ if not found:
+ console.print(f" [yellow]nothing matches {answer!r} — "
+ f"enter a group number, or l, c, g, r, s, d "
+ f"or b[/yellow]")
+ elif len(found) == 1:
+ _edit_option(console, options,
+ str(ft8.OPTIONS.index(found[0]) + 1), ft8)
+ else:
+ _option_list(console, options, found, f"matching {answer!r}",
+ ft8)
+ _pick_option(console, options, ft8)
+
+
+def _ft8_listen(console: Console, cfg: ScanConfig, options) -> None:
+ from . import ft8
+
+ errs = options.validate()
+ if errs:
+ for e in errs:
+ console.print(f" [red]{e}[/red]")
+ return
+ try:
+ ft8.listen(console, options, cfg.output_dir)
+ except Exception as exc: # a menu must survive it
+ console.print(f" [red]{exc}[/red]")
+
+
+def _ft8_band(console: Console, options) -> None:
+ """Choose the band, which sets the dial frequency with it.
+
+ Which band is also the question of whether the receiver can hear it at
+ all, so the list says so rather than leaving somebody to find out by
+ listening to silence for ten minutes.
+ """
+ from . import ft8
+
+ console.print("\n [bold]Which band[/bold]\n")
+ for i, (name, hz, needs) in enumerate(ft8.BANDS, 1):
+ note = ("[yellow]needs an upconverter or direct sampling[/yellow]"
+ if needs == "shortwave"
+ else "[green]a plain receiver reaches this[/green]")
+ here = " [green]<- now[/green]" if options.band == name else ""
+ console.print(f" [cyan]{i:>2}[/cyan] {name:<5} "
+ f"{hz / 1e6:>10.3f} MHz {note}{here}")
+ console.print("\n [grey62]Almost all the activity is on shortwave, and "
+ "20m is the busiest band on earth. Two metres is quiet by "
+ "comparison and is what a plain dongle can reach.[/grey62]")
+ answer = _ask(console, "\n band (number, name, or blank to keep)",
+ "").strip().lower()
+ if not answer:
+ return
+ if answer.isdigit() and 1 <= int(answer) <= len(ft8.BANDS):
+ ft8.use_band(options, ft8.BANDS[int(answer) - 1][0])
+ elif any(answer == name for name, _hz, _n in ft8.BANDS):
+ ft8.use_band(options, answer)
+ else:
+ console.print(f" [yellow]no band called {answer!r}[/yellow]")
+ return
+ console.print(f" [green]{ft8.band_text(options)}[/green]")
+
+
+def _ft8_grid(console: Console, options) -> None:
+ """Set the grid square, which is what distances are measured from."""
+ from . import ft8
+
+ console.print("\n [grey62]Your Maidenhead grid square: four characters "
+ "like IO91 or FN31. Every station calling CQ says where it "
+ "is this way, so with yours filled in the report can say "
+ "how far each of them is and on what bearing — which on "
+ "shortwave is the whole interest of the thing.[/grey62]")
+ answer = _ask(console, "\n grid square (blank to clear)",
+ options.grid).strip()
+ if not answer:
+ options.grid = ""
+ console.print(" [green]cleared — no distances[/green]")
+ return
+ if ft8.grid_at(answer) is None:
+ console.print(f" [yellow]{answer!r} is not a grid square[/yellow]")
+ return
+ options.grid = answer.upper()
+ lat, lon = ft8.grid_at(options.grid)
+ console.print(f" [green]{options.grid} — {lat:.2f},{lon:.2f}[/green]")
+
+
+def _ft8_read(console: Console, cfg: ScanConfig, options, logs) -> None:
+ """Read an evening's decodes back off the disk."""
+ from . import ft8, ft8log
+
+ if not logs:
+ console.print(" [yellow]no FT8 logs yet — listen first[/yellow]")
+ return
+ console.print("\n [bold]Which log[/bold]\n")
+ for i, path in enumerate(logs[-20:], 1):
+ console.print(f" [cyan]{i:>2}[/cyan] {path.name}")
+ answer = _ask(console, "\n log (number, or blank to go back)",
+ "").strip()
+ if not answer.isdigit():
+ return
+ picked = logs[-20:][int(answer) - 1] if 1 <= int(answer) <= len(logs[-20:]) \
+ else None
+ if picked is None:
+ return
+ rows = ft8log.read_log(picked)
+ console.print(f"\n [grey62]{len(rows)} decodes in {picked.name}"
+ f"[/grey62]\n")
+ for row in rows[:40]:
+ console.print(f" {row['snr_db']:>4.0f} {row['offset']:>5.1f} "
+ f"{row['hertz']:>7.1f} ~ {row['text']}")
+ if len(rows) > 40:
+ console.print(f" [grey62]…and {len(rows) - 40} more[/grey62]")
diff --git a/bandsaunter/ui.py b/bandsaunter/ui.py
index e2976da..df2c594 100755
--- a/bandsaunter/ui.py
+++ b/bandsaunter/ui.py
@@ -1071,3 +1071,118 @@ class AprsDisplay:
if not out.plain:
out.append(station.kind, style="grey62")
return out
+
+
+class Ft8Display:
+ """The last slot's decodes, above a table of who has been heard.
+
+ Two things at once because FT8 is two things at once. A slot is an
+ event -- forty stations transmitted, here is what they said -- and the
+ evening is an accumulation. A display that showed only the latest slot
+ would throw away the band; one that showed only the running table would
+ never show the thing that just happened.
+ """
+
+ def __init__(self, console: Console, hold: float = 3600.0,
+ imperial: bool = False, grid: str = "", band: str = ""):
+ self.console = console
+ self.hold = hold
+ self.imperial = imperial
+ self.grid = grid
+ self.band = band
+ self.started = time.time()
+ self.heard = None
+ self.slot_at = 0.0
+
+ def update(self, heard, slot_at: float = 0.0) -> None:
+ self.heard = heard
+ if slot_at:
+ self.slot_at = slot_at
+
+ def showing(self, now: float | None = None) -> list:
+ if self.heard is None:
+ return []
+ now = time.time() if now is None else now
+ return [s for s in self.heard.band.all()
+ if now - s.last <= self.hold]
+
+ def render(self, now: float | None = None, width: int | None = None):
+ now = time.time() if now is None else now
+ parts = [self._header(now)]
+ latest = list(getattr(self.heard, "latest", []) or [])
+ if latest:
+ parts.append(self._slot(latest))
+ here = self.showing(now)
+ if here:
+ parts.append(self._table(here, now))
+ elif not latest:
+ parts.append(Panel(_one_line(
+ "[grey62]nothing decoded yet — the first decodes arrive "
+ "when the slot after next ends, up to thirty seconds away. "
+ "If nothing comes at all, check the clock: FT8 needs it "
+ "right to a second or two[/grey62]"),
+ border_style="grey37", padding=(0, 1)))
+ return Group(*parts)
+
+ def _header(self, now: float):
+ heard = self.heard
+ slots = getattr(heard, "slots", 0)
+ decodes = getattr(heard, "decodes", 0)
+ stations = len(getattr(heard, "band", None).stations) if heard else 0
+ elapsed = max(1e-9, now - self.started)
+ bits = [f"[bold]FT8[/bold] {self.band}",
+ f"{slots} slots",
+ f"{decodes} decodes",
+ f"{stations} stations"]
+ if slots:
+ bits.append(f"{decodes / max(1, slots):.1f} a slot")
+ bits.append(_dur(elapsed))
+ if self.grid:
+ bits.append(f"from {self.grid.upper()}")
+ return Text.from_markup(" ".join(f"[grey62]{b}[/grey62]"
+ if i else b
+ for i, b in enumerate(bits)))
+
+ def _slot(self, latest):
+ table = Table(box=None, header_style="bold", pad_edge=False,
+ title=f"last slot — {len(latest)} decoded",
+ title_justify="left", title_style="bold")
+ table.add_column("snr", justify="right")
+ table.add_column("dt", justify="right")
+ table.add_column("hz", justify="right")
+ table.add_column("message")
+ for d in latest[:24]:
+ table.add_row(f"{d.snr_db:.0f}", f"{d.offset:+.1f}",
+ f"{d.hertz:.0f}",
+ f"[bold]{d.text}[/bold]" if d.calling else d.text)
+ return table
+
+ def _table(self, here, now: float):
+ from .ft8 import grid_away
+
+ here = sorted(here, key=lambda s: (-s.last, s.call))
+ table = Table(box=None, header_style="bold", pad_edge=False,
+ title=f"heard so far — {len(here)} stations",
+ title_justify="left", title_style="bold")
+ table.add_column("station")
+ table.add_column("grid")
+ if self.grid:
+ table.add_column("away", justify="right")
+ table.add_column("n", justify="right")
+ table.add_column("best", justify="right")
+ table.add_column("hz", justify="right")
+ table.add_column("last", justify="right")
+ for s in here[:20]:
+ row = [s.call, s.grid or "—"]
+ if self.grid:
+ away = grid_away(self.grid, s.grid) if s.grid else None
+ if away is None:
+ row.append("—")
+ elif self.imperial:
+ row.append(f"{away[0] * 0.621371:.0f} mi")
+ else:
+ row.append(f"{away[0]:.0f} km")
+ row += [str(s.decodes), f"{s.best_snr:.0f}", f"{s.hertz:.0f}",
+ f"{now - s.last:.0f}s"]
+ table.add_row(*row)
+ return table
diff --git a/packaging/bandsaunter.1 b/packaging/bandsaunter.1
index f788817..dd3409d 100644
--- a/packaging/bandsaunter.1
+++ b/packaging/bandsaunter.1
@@ -1,5 +1,5 @@
.\" Generated by packaging/make-man.py -- do not edit by hand.
-.TH BANDSAUNTER 1 "2026-09-21" "bandsaunter 2026-09-21_03" "User Commands"
+.TH BANDSAUNTER 1 "2026-09-21" "bandsaunter 2026-09-21_04" "User Commands"
.SH NAME
bandsaunter \- scan, record and identify radio signals with an RTL-SDR
.SH SYNOPSIS
@@ -2958,6 +2958,196 @@ Also write a map \[em] write the stations and their tracks for Google Earth.
.br
Setting name \fBkml\fR, default \fBno\fR.
.PP
+.SH FT8
+Fifteen seconds of everybody at once. Every station on the band transmits in
+the same quarter-minute slots, on the same dial frequency, fifty hertz wide
+each, stacked across three kilohertz of audio \[em] so one receiver parked on
+one frequency hears the whole band's worth of stations at the same time, and
+hears most of them well below the noise.
+.PP
+Half of what is transmitted is error-correcting code, and that is the trick:
+it is what buys a mode that decodes twenty-odd decibels under what an operator
+can hear. A receiver that took the loudest tone of each symbol and hoped would
+decode almost nothing, which is why the tone detector reports how confident it
+is bit by bit rather than what it thinks it heard.
+.SS What it needs
+The clock has to be right to a second or two. The slots are quarter-minutes of
+UTC and every station on earth agrees about which one it is. A receiver a
+second out still decodes; one a slot out hears every transmission split across
+two captures and decodes none of them. This is the one failure that looks
+exactly like a dead band, so the display and the report both say so when
+nothing arrives.
+.PP
+Almost all the activity is on shortwave, which a plain receiver of this kind
+cannot reach. The default is therefore the two-metre channel at 144.174 MHz,
+which it can. All thirteen channels are in the list and the shortwave ones
+work through an upconverter or a receiver in direct sampling mode; the menu
+says which is which rather than leaving somebody to find out by listening to
+silence.
+.SS What comes out
+.BI \-\-grid " SQUARE"
+is what turns decodes into geography: every station calling CQ says where it
+is, so with your own square filled in each gets a distance and a bearing and
+the furthest heard is named. Three tables afterwards \[em] stations heard,
+calling CQ, and who was working whom.
+.PP
+.B \-\-adif
+writes the log again in the form every amateur logging program imports,
+marked as heard rather than worked. Nothing here transmits, so nothing here is
+a contact, and an ADIF that let a logging program treat these as worked would
+put claims into somebody's log that they cannot make.
+.SS How well it works
+Checked against eleven off-air recordings with published decodes, which is the
+only honest way to test a decoder: an encoder tested against its own decoder
+agrees with it about anything they are both wrong about. Ninety-seven of a
+hundred and fifty messages, with no false decodes; timing within a hundredth
+of a second, frequency within a hertz, signal reports within half a decibel on
+average. The third not decoded are the weakest in each slot: a mature decoder
+subtracts what it has decoded and looks again in the remainder, which is not
+built here.
+.SH FT8 OPTIONS
+Every option the FT8 side takes, in the five groups the menu shows them in.
+Each is a flag here and a line in the menu, and both come from one table in
+the program, so they cannot disagree.
+.SS Receiver
+.TP
+.B --device
+Receiver \[em] which receiver to use, when more than one is plugged in.
+.br
+Setting name \fBdevice\fR, default \fB0\fR.
+.br
+Accepts: at least 0.
+.TP
+.B --gain
+Gain \[em] tuner gain in dB, or automatic.
+.br
+Setting name \fBgain\fR, default \fBauto\fR.
+.TP
+.B --rate
+Sample rate \[em] how fast to sample; 96 kS/s is the least that holds the channel (Hz).
+.br
+Setting name \fBrate\fR, default \fB240 kHz\fR.
+.br
+Accepts: at least 96000.
+.TP
+.B --band
+Band \[em] which FT8 channel to listen on.
+.br
+Setting name \fBband\fR, default \fB2m\fR.
+.br
+Accepts: one of: 160m, 80m, 60m, 40m, 30m, 20m, 17m, 15m, 12m, 10m, 6m, 2m, 70cm.
+.TP
+.B --frequency --freq
+Dial frequency \[em] the dial frequency, if you want one the band list does not have (Hz).
+.br
+Setting name \fBfrequency\fR, default \fB144.174 MHz\fR.
+.br
+Accepts: at least 100000, at most 2e+09.
+.TP
+.B --simulate / --no-simulate
+Simulate \[em] invent a band instead of using a receiver.
+.br
+Setting name \fBsimulate\fR, default \fBno\fR.
+.PP
+.SS Listening
+.TP
+.B --seconds
+Listen for \[em] how long to listen; zero means until stopped (s).
+.br
+Setting name \fBseconds\fR, default \fBuntil stopped\fR.
+.br
+Accepts: at least 0.
+.TP
+.B --slots
+Or this many slots \[em] stop after this many fifteen-second slots; zero means no limit.
+.br
+Setting name \fBslots\fR, default \fBno limit\fR.
+.br
+Accepts: at least 0.
+.TP
+.B --log / --no-log
+Write a log \[em] keep every decode in a file.
+.br
+Setting name \fBlog\fR, default \fByes\fR.
+.TP
+.B --decodes-seen / --no-decodes-seen
+Print each decode \[em] a line per decode instead of a table that refreshes.
+.br
+Setting name \fBdecodes_seen\fR, default \fBno\fR.
+.TP
+.B --hold
+Keep on display for \[em] how long a station stays on the table after its last decode (s).
+.br
+Setting name \fBhold\fR, default \fB3600 s\fR.
+.br
+Accepts: at least 1.
+.PP
+.SS Decoding
+.TP
+.B --lowest
+Search from \[em] the lowest audio frequency to look for signals at (Hz).
+.br
+Setting name \fBlowest\fR, default \fB200 Hz\fR.
+.br
+Accepts: at least 0.
+.TP
+.B --highest
+Search to \[em] the highest audio frequency to look for signals at (Hz).
+.br
+Setting name \fBhighest\fR, default \fB3 kHz\fR.
+.br
+Accepts: at least 100.
+.TP
+.B --most
+Candidates per slot \[em] how many possible transmissions to try to decode each slot.
+.br
+Setting name \fBmost\fR, default \fB300\fR.
+.br
+Accepts: at least 1.
+.TP
+.B --rounds
+Repair passes \[em] how many passes of error correction to make before giving up.
+.br
+Setting name \fBrounds\fR, default \fB30\fR.
+.br
+Accepts: at least 1.
+.PP
+.SS Showing
+.TP
+.B --grid
+Aerial at \[em] your own grid square, so distances can be worked out.
+.br
+Setting name \fBgrid\fR, default \fBnot set, so no distances\fR.
+.TP
+.B --units
+Show readings in \[em] metric or imperial, for the display and the export.
+.br
+Setting name \fBunits\fR, default \fBmetric\fR.
+.br
+Accepts: one of: metric, imperial.
+.TP
+.B --calls-only / --no-calls-only
+Callsigns only \[em] leave out free text and telemetry.
+.br
+Setting name \fBcalls_only\fR, default \fBno\fR.
+.PP
+.SS Afterwards
+.TP
+.B --report / --no-report
+Report at the end \[em] print the tables when the listening stops.
+.br
+Setting name \fBreport\fR, default \fByes\fR.
+.TP
+.B --csv / --no-csv
+Also write a CSV \[em] a spreadsheet of every decode beside the log.
+.br
+Setting name \fBcsv\fR, default \fBno\fR.
+.TP
+.B --adif / --no-adif
+Also write an ADIF \[em] the log again, in the form logging programs read.
+.br
+Setting name \fBadif\fR, default \fBno\fR.
+.PP
.SH FILES
.TP
.I ~/.config/bandsaunter/config.yaml
@@ -3090,6 +3280,16 @@ terms, in
or at
.UR https://www.gnu.org/licenses/
.UE .
+.PP
+One file is not original work.
+.I ft8tables.py
+holds the two fixed tables that define the FT8 error-correcting code, taken
+from ft8_lib (https://github.com/kgoba/ft8_lib), MIT licensed, copyright 2018
+K\[u0101]rlis Goba, which took them in turn from WSJT\-X. They are reproduced
+under the MIT terms and that file carries the attribution they ask for. They
+are there because they cannot be derived: everything else about FT8 here is
+worked out from first principles, but those two tables are the code itself,
+chosen once by its designers and published. No decoding logic was taken.
.SH BUGS
The DVB-T television driver claims these dongles on sight. If the receiver
cannot be opened, that is almost always why: the package blacklists the
diff --git a/packaging/make-man.py b/packaging/make-man.py
index 737ef06..7f5c48f 100755
--- a/packaging/make-man.py
+++ b/packaging/make-man.py
@@ -77,6 +77,13 @@ def aprs_section() -> list[str]:
return options_section(ap)
+def ft8_section() -> list[str]:
+ """Every FT8 option, from the same table again."""
+ from bandsaunter import ft8
+
+ return options_section(ft8)
+
+
def options_section(air) -> list[str]:
"""One section's options, written out from the table the program uses.
@@ -1792,6 +1799,58 @@ Every option the APRS side takes, in the four groups the menu shows them in.
Each is a flag here and a line in the menu, and both come from one table in
the program, so they cannot disagree.
.APRS_OPTIONS_HERE
+.SH FT8
+Fifteen seconds of everybody at once. Every station on the band transmits in
+the same quarter-minute slots, on the same dial frequency, fifty hertz wide
+each, stacked across three kilohertz of audio \[em] so one receiver parked on
+one frequency hears the whole band's worth of stations at the same time, and
+hears most of them well below the noise.
+.PP
+Half of what is transmitted is error-correcting code, and that is the trick:
+it is what buys a mode that decodes twenty-odd decibels under what an operator
+can hear. A receiver that took the loudest tone of each symbol and hoped would
+decode almost nothing, which is why the tone detector reports how confident it
+is bit by bit rather than what it thinks it heard.
+.SS What it needs
+The clock has to be right to a second or two. The slots are quarter-minutes of
+UTC and every station on earth agrees about which one it is. A receiver a
+second out still decodes; one a slot out hears every transmission split across
+two captures and decodes none of them. This is the one failure that looks
+exactly like a dead band, so the display and the report both say so when
+nothing arrives.
+.PP
+Almost all the activity is on shortwave, which a plain receiver of this kind
+cannot reach. The default is therefore the two-metre channel at 144.174 MHz,
+which it can. All thirteen channels are in the list and the shortwave ones
+work through an upconverter or a receiver in direct sampling mode; the menu
+says which is which rather than leaving somebody to find out by listening to
+silence.
+.SS What comes out
+.BI \-\-grid " SQUARE"
+is what turns decodes into geography: every station calling CQ says where it
+is, so with your own square filled in each gets a distance and a bearing and
+the furthest heard is named. Three tables afterwards \[em] stations heard,
+calling CQ, and who was working whom.
+.PP
+.B \-\-adif
+writes the log again in the form every amateur logging program imports,
+marked as heard rather than worked. Nothing here transmits, so nothing here is
+a contact, and an ADIF that let a logging program treat these as worked would
+put claims into somebody's log that they cannot make.
+.SS How well it works
+Checked against eleven off-air recordings with published decodes, which is the
+only honest way to test a decoder: an encoder tested against its own decoder
+agrees with it about anything they are both wrong about. Ninety-seven of a
+hundred and fifty messages, with no false decodes; timing within a hundredth
+of a second, frequency within a hertz, signal reports within half a decibel on
+average. The third not decoded are the weakest in each slot: a mature decoder
+subtracts what it has decoded and looks again in the remainder, which is not
+built here.
+.SH FT8 OPTIONS
+Every option the FT8 side takes, in the five groups the menu shows them in.
+Each is a flag here and a line in the menu, and both come from one table in
+the program, so they cannot disagree.
+.FT8_OPTIONS_HERE
.SH FILES
.TP
.I ~/.config/bandsaunter/config.yaml
@@ -1924,6 +1983,16 @@ terms, in
or at
.UR https://www.gnu.org/licenses/
.UE .
+.PP
+One file is not original work.
+.I ft8tables.py
+holds the two fixed tables that define the FT8 error-correcting code, taken
+from ft8_lib (https://github.com/kgoba/ft8_lib), MIT licensed, copyright 2018
+K\[u0101]rlis Goba, which took them in turn from WSJT\-X. They are reproduced
+under the MIT terms and that file carries the attribution they ask for. They
+are there because they cannot be derived: everything else about FT8 here is
+worked out from first principles, but those two tables are the code itself,
+chosen once by its designers and published. No decoding logic was taken.
.SH BUGS
The DVB-T television driver claims these dongles on sight. If the receiver
cannot be opened, that is almost always why: the package blacklists the
@@ -1944,6 +2013,7 @@ def main() -> int:
text = text.replace(".WEATHER_OPTIONS_HERE",
"\n".join(weather_section()))
text = text.replace(".APRS_OPTIONS_HERE", "\n".join(aprs_section()))
+ text = text.replace(".FT8_OPTIONS_HERE", "\n".join(ft8_section()))
text = text.replace("\n\n", "\n") # troff dislikes blank lines
target = Path(sys.argv[1] if len(sys.argv) > 1
else Path(__file__).parent / "bandsaunter.1")
diff --git a/tests/test_ft8.py b/tests/test_ft8.py
new file mode 100644
index 0000000..0a7b352
--- /dev/null
+++ b/tests/test_ft8.py
@@ -0,0 +1,839 @@
+"""The FT8 section: the code, the radio, the options and the front ends.
+
+The important thing about this file is where its truth comes from. An
+encoder tested against its own decoder agrees with it about anything they
+are both wrong about -- this project learned that expensively on 433 MHz --
+so the decoding here is checked against signals that came off the air and
+were decoded by somebody else's program, not against anything written here.
+
+Those recordings live outside the repository, and the tests that use them
+skip when they are absent. What does not skip is REAL: whole codewords
+lifted off the air, which pin the checksum, the parity and every field of
+the message format to reality in forty-four characters apiece.
+"""
+import math
+import time
+import wave
+from pathlib import Path
+
+import numpy as np
+import pytest
+
+from bandsaunter import ft8, ft8code as code, ft8log, ft8sim, ft8wave
+from bandsaunter import ft8tables as tables
+
+
+REFERENCE = Path("/mnt/global/bandsaunter/ft8ref")
+needs_recordings = pytest.mark.skipif(
+ not REFERENCE.is_dir(), reason="the off-air recordings are not here")
+
+
+# Whole codewords, off the air, with what they say. A hundred and
+# seventy-four bits each: seventy-seven of message, fourteen of checksum and
+# eighty-three of parity, so one of these exercises the lot.
+REAL = [
+ ("B79A7EA67655CA92DB4D342FDD55B071779A631CFB4C",
+ "PA3EPP SP8NFO KN09", "standard"),
+ ("0000002046E853111D48153FD733DA62C110A5E053FC",
+ "CQ F4FSY JN25", "standard"),
+ ("E20AFC7590F9DA9FA7C9EEEA8A1A96D06F6855C82B58",
+ "VK4BLE OH1EDK -20", "standard"),
+ ("70D84EAC55E5FB9FA70CD7148B9180DD1CD36A7EFC6C",
+ "ET3RFG/R IN3ADG -23", "standard"),
+ ("1E6191F629601212F10D6992973A9B5AEA0596877498",
+ "2M0OGG RA6ABO KN96", "standard"),
+ ("70AFEC9338AD521FA50DE077F5C94CAD362B2CC3A364",
+ "ES5GI DD3SF 73", "standard"),
+ ("00000024519BF311248987ED8D2C4AA596CE262E1E68",
+ "CQ IK4LZH JN54", "standard"),
+ ("4B9ACCF4519BF31FAA4F0C50A7B6CEA7F9C51BED8E80",
+ "9A9TT IK4LZH -10", "standard"),
+ ("0B13C7046826781FA50F539DFE6D9FAF63F7C21EC268",
+ "R2EA IZ4OUL 73", "standard"),
+ ("CBEC04F45AC9D49FA509770E71FEDE65EE2EFE66E8A8",
+ "SA5QED IQ5PJ 73", "standard"),
+]
+
+
+def bits_of_hex(text: str) -> np.ndarray:
+ raw = np.unpackbits(np.frombuffer(bytes.fromhex(text), dtype=np.uint8))
+ return raw[:code.BITS].astype(np.uint8)
+
+
+# ---------------------------------------------------------------------------
+# The two tables the code is defined by
+# ---------------------------------------------------------------------------
+
+def test_the_generator_and_the_parity_matrix_describe_the_same_code():
+ """The two published tables are not derivable from one another -- the
+ generator's parity half runs to fifty-odd bits a row and the sparse one
+ to six or seven -- so that they agree is checked rather than assumed.
+
+ If they disagreed, every codeword this builds would fail every check
+ and nothing would ever decode, which is a long way to find out."""
+ rng = np.random.default_rng(11)
+ for _ in range(50):
+ message = rng.integers(0, 2, code.PAYLOAD).astype(np.uint8)
+ assert code.parity_holds(code.encode(message))
+
+
+def test_every_codeword_bit_is_checked_three_times():
+ """A regular column weight, which is what the message passing assumes
+ when it gathers three slots per bit instead of scattering."""
+ assert code.SLOTS_OF_BIT.shape == (code.BITS, 3)
+ assert sorted({len(c) for c in tables.CHECKS}) == [6, 7]
+
+
+def test_a_broken_codeword_fails_its_parity():
+ rng = np.random.default_rng(3)
+ word = code.encode(rng.integers(0, 2, code.PAYLOAD).astype(np.uint8))
+ for flip in (0, 40, 90, 173):
+ bad = word.copy()
+ bad[flip] ^= 1
+ assert not code.parity_holds(bad)
+
+
+# ---------------------------------------------------------------------------
+# Reality
+# ---------------------------------------------------------------------------
+
+@pytest.mark.parametrize("packed,text,kind", REAL)
+def test_a_codeword_off_the_air_says_what_it_said(packed, text, kind):
+ """The whole chain against something nobody here made up."""
+ word = bits_of_hex(packed)
+ assert code.parity_holds(word), "the parity of a real transmission"
+ assert code.crc_holds(word[:code.PAYLOAD]), "the checksum of a real one"
+ message = code.unpack(word[:code.MESSAGE_BITS])
+ assert message.text == text
+ assert message.kind == kind
+
+
+def test_a_real_codeword_is_repaired_from_a_damaged_copy():
+ """What the error correction is for, on a real transmission rather than
+ a made-up one.
+
+ Ten of the hundred and seventy-four bits, each of them *confidently*
+ wrong, which is the hard case: a bit the detector is unsure about costs
+ the decoding far less than one it is certain about and mistaken. Real
+ errors arrive uncertain, which is why the same code reads signals off
+ the air with far more than ten bits astray."""
+ word = bits_of_hex(REAL[0][0])
+ rng = np.random.default_rng(5)
+ belief = np.where(word == 1, -4.0, 4.0).astype(np.float32)
+ for flip in rng.choice(code.BITS, 10, replace=False):
+ belief[flip] = -belief[flip]
+ got = code.repair(belief)
+ assert got is not None and (got == word).all()
+ assert code.unpack(got[:code.MESSAGE_BITS]).text == REAL[0][1]
+
+
+def test_noise_does_not_decode_into_a_message():
+ """The checksum's job. The error correction will converge on *a*
+ codeword given enough noise, and the fourteen bits are what stop that
+ becoming a line in somebody's log."""
+ rng = np.random.default_rng(17)
+ lied = 0
+ for _ in range(200):
+ belief = rng.standard_normal(code.BITS).astype(np.float32) * 2.0
+ got = code.repair(belief)
+ if got is not None and code.crc_holds(got[:code.PAYLOAD]):
+ lied += 1
+ # One in sixteen thousand gets through by arithmetic; two hundred tries
+ # should see none, and seeing several would mean the CRC is not being
+ # checked at all.
+ assert lied == 0, f"{lied} runs of noise passed the checksum"
+
+
+# ---------------------------------------------------------------------------
+# The checksum and the coding
+# ---------------------------------------------------------------------------
+
+def test_the_checksum_covers_the_message_zero_extended():
+ """Eighty-two bits, not seventy-seven -- which is not the same answer
+ and is the sort of thing that decodes nothing at all."""
+ payload = np.zeros(code.MESSAGE_BITS, dtype=np.uint8)
+ payload[3] = payload[70] = 1
+ block = code.with_crc(payload)
+ assert block.size == code.PAYLOAD
+ assert code.crc_holds(block)
+ broken = block.copy()
+ broken[10] ^= 1
+ assert not code.crc_holds(broken)
+
+
+def test_the_tones_carry_the_bits_and_the_sync_is_where_it_should_be():
+ rng = np.random.default_rng(2)
+ word = code.encode(rng.integers(0, 2, code.PAYLOAD).astype(np.uint8))
+ tones = code.tones_of(word)
+ assert tones.size == code.TONES
+ assert (code.bits_of(tones) == word).all()
+ for start in (0, 36, 72):
+ assert list(tones[start:start + 7]) == list(tables.COSTAS)
+
+
+def test_a_tone_mistaken_for_its_neighbour_costs_one_bit():
+ """Which is the whole reason for the Gray mapping, and is worth a test
+ because getting the map backwards still round-trips perfectly."""
+ for tone in range(7):
+ a = code.UNGRAY[tone]
+ b = code.UNGRAY[tone + 1]
+ assert bin(int(a) ^ int(b)).count("1") == 1
+
+
+# ---------------------------------------------------------------------------
+# Messages
+# ---------------------------------------------------------------------------
+
+@pytest.mark.parametrize("text", [
+ "CQ DL1UDO JO31", "CQ DX Z33Z KN11", "CQ JA OH1LWZ KP11",
+ "VK4BLE OH8JK R-17", "JR5MJS OH8NW 73", "LZ1CWK DC8VA RR73",
+ "G4CUS SP4FCA +10", "2M0OGG RA6ABO KN96", "YO7CGS A41ZZ -11",
+ "R2EA IZ4OUL R-08", "W2XYZ K1ABC RRR",
+])
+def test_a_standard_message_survives_the_whole_chain(text):
+ payload = code.pack(text)
+ word = code.encode(code.with_crc(payload))
+ assert code.parity_holds(word)
+ back = code.bits_of(code.tones_of(word))
+ assert (back == word).all()
+ got = code.repair(np.where(word == 1, -4.0, 4.0).astype(np.float32))
+ assert got is not None and (got == word).all()
+ assert code.unpack(got[:code.MESSAGE_BITS]).text == text
+
+
+def test_the_digit_of_a_callsign_is_found_rather_than_guessed_at():
+ """Z33Z has a digit in the second place that belongs to the prefix, so
+ a packer that assumes the second character is the separating digit
+ turns it into a hash and loses the callsign."""
+ assert code.is_standard_call("Z33Z")
+ assert code.is_standard_call("K1ABC")
+ assert code.is_standard_call("DL1UDO")
+ assert code.is_standard_call("2M0OGG")
+ assert not code.is_standard_call("ET3RFG/R")
+ assert not code.is_standard_call("")
+
+
+def test_free_text_is_not_dressed_up_as_two_callsigns():
+ """A packer that falls back to hashing any word it does not recognise
+ always succeeds, and would send HELLO WORLD as two hashed callsigns."""
+ got = code.unpack(code.pack("HELLO WORLD"))
+ assert got.kind == "free text"
+ assert got.text == "HELLO WORLD"
+
+
+def test_free_text_is_thirteen_characters_and_says_so_by_truncating():
+ got = code.unpack(code.pack("TNX FER QSO 73"))
+ assert got.kind == "free text"
+ assert len(got.text) <= 13
+
+
+@pytest.mark.parametrize("text,grid,report,calling", [
+ ("CQ DL1UDO JO31", "JO31", "", True),
+ ("VK4BLE OH8JK R-17", "", "R-17", False),
+ ("JR5MJS OH8NW 73", "", "73", False),
+ ("W2XYZ K1ABC RRR", "", "RRR", False),
+])
+def test_what_can_be_picked_out_of_a_message(text, grid, report, calling):
+ got = code.unpack(code.pack(text))
+ assert got.grid == grid and got.report == report
+ assert got.calling is calling
+
+
+def test_a_hashed_callsign_stays_unknown_until_it_has_been_heard_in_full():
+ """A wrong callsign in a log is worse than a missing one, so a hash
+ that has not been heard spelled out stays <...> rather than guessed."""
+ book = code.CallBook()
+ assert book.look_up(code.hash_of("GM4ABC", 22), 22) == "<...>"
+ book.remember("GM4ABC")
+ assert book.look_up(code.hash_of("GM4ABC", 22), 22) == "GM4ABC"
+ assert book.look_up(code.hash_of("GM4ABD", 22), 22) == "<...>"
+
+
+def test_a_callsign_too_short_to_mean_anything_is_not_remembered():
+ book = code.CallBook()
+ for rubbish in ("", "A", "CQ", "DE", "QRZ"):
+ book.remember(rubbish)
+ assert book.by_hash == {}
+
+
+# ---------------------------------------------------------------------------
+# The radio
+# ---------------------------------------------------------------------------
+
+def test_a_transmission_is_found_where_it_was_put():
+ """The decoder's own clock and dial, against a signal whose time and
+ frequency are known to the sample."""
+ for hertz in (500.0, 1200.0, 2400.0):
+ for offset in (0.2, 0.5, 1.4):
+ audio = ft8wave.transmit("CQ DL1UDO JO31", hertz=hertz,
+ offset=offset)
+ got = ft8wave.listen_to(audio, 12000.0)
+ assert len(got) == 1, (hertz, offset)
+ assert got[0].text == "CQ DL1UDO JO31"
+ assert abs(got[0].hertz - hertz) < 3.5
+ # Reported against the nominal start of sending, half a second
+ # into the slot, which is what every FT8 program reports.
+ assert abs(got[0].offset - (offset - 0.5)) < 0.12
+
+
+def test_the_nominal_start_is_half_a_second_into_the_slot():
+ """A station whose clock is right reports as dt 0.0, not dt 0.5: the
+ transmission is 12.64 seconds long in a slot of fifteen and everybody
+ begins half a second in."""
+ audio = ft8wave.transmit("CQ DL1UDO JO31", offset=ft8wave.NOMINAL_START)
+ got = ft8wave.listen_to(audio, 12000.0)
+ assert got and abs(got[0].offset) < 0.1
+
+
+def test_several_transmissions_in_one_slot_all_come_out():
+ """Which is the whole point of the mode: they overlap in time entirely
+ and are told apart by frequency alone."""
+ said = [("CQ DL1UDO JO31", 500.0), ("VK4BLE OH8JK R-17", 1000.0),
+ ("JR5MJS OH8NW 73", 1600.0), ("CQ DX Z33Z KN11", 2300.0)]
+ audio = np.zeros(int(12000 * code.SLOT_S), dtype=np.float32)
+ for text, hertz in said:
+ audio += ft8wave.transmit(text, hertz=hertz, offset=0.5)
+ got = {d.text for d in ft8wave.listen_to(audio, 12000.0)}
+ assert got == {t for t, _hz in said}
+
+
+def test_a_slot_of_noise_decodes_to_nothing():
+ rng = np.random.default_rng(9)
+ noise = rng.standard_normal(int(12000 * code.SLOT_S)).astype(np.float32)
+ assert ft8wave.listen_to(noise, 12000.0) == []
+
+
+def test_it_hears_a_signal_well_below_the_noise():
+ """The claim the mode is built on. Fifteen decibels under is a signal
+ an operator hears nothing of at all."""
+ heard = 0
+ for seed in range(5):
+ audio = ft8wave.transmit("CQ DL1UDO JO31", hertz=1200.0, offset=0.5,
+ snr_db=-15.0, seed=seed)
+ got = ft8wave.listen_to(audio, 12000.0)
+ heard += any(d.text == "CQ DL1UDO JO31" for d in got)
+ assert heard == 5
+
+
+def test_the_signal_report_tracks_the_signal():
+ """Not to a decibel -- it is an estimate -- but a stronger signal has
+ to read stronger, or the number is worse than not printing one."""
+ readings = []
+ for want in (-15.0, -5.0, 5.0):
+ audio = ft8wave.transmit("CQ DL1UDO JO31", hertz=1200.0, offset=0.5,
+ snr_db=want, seed=1)
+ got = ft8wave.listen_to(audio, 12000.0)
+ assert got
+ readings.append(got[0].snr_db)
+ assert readings[0] < readings[1] < readings[2]
+ assert all(abs(r - w) < 8.0
+ for r, w in zip(readings, (-15.0, -5.0, 5.0)))
+
+
+def test_the_waterfall_is_half_a_symbol_and_half_a_tone():
+ """A transmission a quarter of a symbol late, or a quarter of a tone
+ off, still has to be caught."""
+ fall = ft8wave.waterfall(np.zeros(12000 * 15, dtype=np.float32), 12000.0)
+ assert fall.hz_per_bin == pytest.approx(code.SYMBOL_HZ
+ / ft8wave.FREQ_STEPS)
+ assert fall.seconds_per_step == pytest.approx(code.SYMBOL_S
+ / ft8wave.TIME_STEPS)
+
+
+def test_audio_too_short_to_hold_a_symbol_is_refused_rather_than_guessed():
+ with pytest.raises(ValueError):
+ ft8wave.waterfall(np.zeros(10, dtype=np.float32), 12000.0)
+
+
+# ---------------------------------------------------------------------------
+# Against the air
+# ---------------------------------------------------------------------------
+
+def _recording(name):
+ with wave.open(str(REFERENCE / f"{name}.wav")) as handle:
+ raw = handle.readframes(handle.getnframes())
+ rate = float(handle.getframerate())
+ audio = np.frombuffer(raw, dtype=np.int16).astype(np.float32) / 32768.0
+ want = {}
+ for line in (REFERENCE / f"{name}.txt").read_text().splitlines():
+ parts = line.split(None, 4)
+ if len(parts) >= 5:
+ want[" ".join(parts[4].lstrip("~ ").split())] = (
+ float(parts[1]), float(parts[2]), float(parts[3]))
+ return audio, rate, want
+
+
+@needs_recordings
+@pytest.mark.parametrize("name,least", [
+ ("191111_110145", 2), ("websdr_test3", 7), ("20m_busy_test_02", 14),
+])
+def test_it_decodes_signals_that_came_off_the_air(name, least):
+ """The only test here whose answers were not written by this program."""
+ audio, rate, want = _recording(name)
+ got = {d.text: d for d in ft8wave.listen_to(audio, rate)}
+ matched = [w for w in want if any(w.startswith(t) for t in got)]
+ assert len(matched) >= least, f"decoded {len(matched)} of {len(want)}"
+ # And nothing it decoded may be something nobody else heard: a false
+ # line in a log is worse than a missing one.
+ unknown = [t for t in got if not any(w.startswith(t) for w in want)]
+ assert len(unknown) <= 1, unknown
+
+
+@needs_recordings
+def test_the_time_and_frequency_agree_with_what_else_heard_them():
+ audio, rate, want = _recording("websdr_test3")
+ errors = []
+ for d in ft8wave.listen_to(audio, rate):
+ for text, (snr, dt, hz) in want.items():
+ if text.startswith(d.text):
+ errors.append((d.offset - dt, d.hertz - hz))
+ assert len(errors) >= 6
+ assert max(abs(dt) for dt, _hz in errors) < 0.2
+ assert max(abs(hz) for _dt, hz in errors) < 4.0
+
+
+# ---------------------------------------------------------------------------
+# Slots and grids
+# ---------------------------------------------------------------------------
+
+def test_slots_are_quarter_minutes_of_utc():
+ """Not of the local clock and not of when this program started: the
+ protocol is built on every station on earth agreeing which one it is."""
+ assert ft8.slot_start(1_700_000_007.0) == 1_700_000_000.0 - \
+ (1_700_000_000.0 % 15)
+ for when in (0.0, 1_700_000_000.0, 1_700_000_014.9):
+ assert ft8.slot_start(when) % ft8.SLOT == 0
+ assert 0 <= when - ft8.slot_start(when) < ft8.SLOT
+ assert ft8.next_slot(when) == ft8.slot_start(when) + ft8.SLOT
+
+
+@pytest.mark.parametrize("grid,lat,lon", [
+ ("IO91", 51.5, -1.0), ("FN31", 41.5, -73.0), ("JN54", 44.5, 11.0),
+ ("RE78", -41.5, 175.0),
+])
+def test_a_grid_square_is_where_it_says(grid, lat, lon):
+ got = ft8.grid_at(grid)
+ assert got == pytest.approx((lat, lon), abs=0.01)
+
+
+def test_a_grid_that_is_not_one_is_refused():
+ for rubbish in ("", "IO", "9191", "ZZ99", "IO9", "ABCD"):
+ assert ft8.grid_at(rubbish) is None
+
+
+def test_the_distance_between_two_squares_is_the_great_circle_one():
+ km, bearing = ft8.grid_away("IO91", "FN31")
+ assert km == pytest.approx(5390, abs=60)
+ assert bearing == pytest.approx(288, abs=3)
+ assert ft8.grid_away("IO91", "IO91")[0] < 1.0
+ assert ft8.grid_away("IO91", "") is None
+
+
+# ---------------------------------------------------------------------------
+# The options
+# ---------------------------------------------------------------------------
+
+def test_every_option_names_a_field_that_exists():
+ fields = set(ft8.Ft8Options().__dict__)
+ for option in ft8.OPTIONS:
+ assert option.key in fields, f"{option.key} is not an option"
+
+
+def test_every_field_is_an_option_somebody_can_reach():
+ assert set(ft8.Ft8Options().__dict__) == {o.key for o in ft8.OPTIONS}
+
+
+def test_every_option_is_in_a_group_and_says_what_it_does():
+ for option in ft8.OPTIONS:
+ assert option.group in ft8.OPTION_GROUPS
+ assert option.help and option.detail and option.flags
+ if option.kind == "bool":
+ assert option.off_flags, f"{option.key} cannot be turned off"
+ for group in ft8.OPTION_GROUPS:
+ assert ft8.in_group(group)
+
+
+def test_every_option_flag_is_one_the_parser_accepts():
+ from bandsaunter.cli import build_parser
+
+ known = set()
+ for action in build_parser()._subparsers._group_actions[0].choices[
+ "ft8"]._actions:
+ known.update(action.option_strings)
+ for option in ft8.OPTIONS:
+ for flag in option.flags + option.off_flags:
+ assert flag in known, f"{flag} is in the table and not the parser"
+
+
+def test_every_command_line_option_is_reachable_from_the_menu():
+ """Asked for explicitly, and worth a test rather than a promise: the
+ menu builds itself from the groups, so an option in no group would be
+ settable from the command line and invisible in the menu."""
+ from bandsaunter import tui
+
+ text = Path(tui.__file__).read_text()
+ assert "def ft8_menu(" in text
+ assert "ft8_menu(console, cfg)" in text, "not reachable from the front page"
+ reachable = set()
+ for group in ft8.OPTION_GROUPS:
+ reachable |= {o.key for o in ft8.in_group(group)}
+ assert reachable == {o.key for o in ft8.OPTIONS}
+
+
+def test_the_options_survive_being_saved_and_read_back(tmp_path):
+ options = ft8.Ft8Options(band="20m", frequency=14_074_000.0,
+ grid="IO91", units="imperial", slots=12)
+ ft8.save_options(options, tmp_path)
+ assert ft8.load_options(tmp_path) == options
+
+
+def test_a_settings_file_with_a_mistake_in_it_costs_the_defaults(tmp_path):
+ (tmp_path / "ft8.yaml").write_text("band: [not a band\n")
+ assert ft8.load_options(tmp_path) == ft8.Ft8Options()
+
+
+@pytest.mark.parametrize("options,broken", [
+ (ft8.Ft8Options(rate=48_000.0), "sample rate"),
+ (ft8.Ft8Options(seconds=-1.0), "negative"),
+ (ft8.Ft8Options(hold=0.0), "display"),
+ (ft8.Ft8Options(frequency=10.0), "frequency"),
+ (ft8.Ft8Options(lowest=2000.0, highest=500.0), "above the bottom"),
+ (ft8.Ft8Options(lowest=1000.0, highest=1050.0), "narrower"),
+ (ft8.Ft8Options(most=0), "candidate"),
+ (ft8.Ft8Options(rounds=0), "error correction"),
+ (ft8.Ft8Options(grid="nowhere"), "grid square"),
+])
+def test_a_setting_that_cannot_work_is_refused(options, broken):
+ assert any(broken in err for err in options.validate())
+
+
+def test_the_defaults_are_all_workable():
+ assert ft8.Ft8Options().validate() == []
+
+
+def test_choosing_a_band_sets_the_frequency_with_it():
+ options = ft8.defaults()
+ ft8.use_band(options, "20m")
+ assert options.frequency == 14_074_000.0 and options.band == "20m"
+ ft8.use_band(options, "nonsense")
+ assert options.frequency == 144_174_000.0
+
+
+def test_the_band_list_says_which_ones_a_plain_receiver_can_reach():
+ """Almost all the activity is on shortwave, which this kind of receiver
+ cannot hear without help, and finding that out by listening to silence
+ for ten minutes is the wrong way to learn it."""
+ assert "upconverter" in ft8.band_text(ft8.Ft8Options(band="20m"))
+ assert "upconverter" not in ft8.band_text(ft8.Ft8Options(band="2m"))
+ assert ft8.defaults().band == "2m"
+
+
+def test_the_command_line_beats_the_saved_settings(tmp_path, monkeypatch):
+ from bandsaunter.cli import build_parser
+
+ ft8.save_options(ft8.Ft8Options(band="2m", grid="AA00"), tmp_path)
+ monkeypatch.setattr(ft8, "options_path",
+ lambda directory=None: tmp_path / "ft8.yaml")
+ args = build_parser().parse_args(["ft8", "--band", "20m",
+ "--grid", "IO91"])
+ assert args.band == "20m" and args.grid == "IO91"
+
+
+# ---------------------------------------------------------------------------
+# End to end
+# ---------------------------------------------------------------------------
+
+class _Clock:
+ """A clock the test winds, so a run of slots takes no real time."""
+
+ def __init__(self, at=1_700_000_000.0):
+ self.at = at
+
+ def now(self):
+ return self.at
+
+ def sleep(self, seconds):
+ self.at += seconds
+
+
+def _run(slots=2, **kw):
+ clock = _Clock()
+ band = ft8sim.SimulatedBand(rate=240_000.0, realtime=True, stations=12,
+ clock=clock.now, sleep=clock.sleep)
+ options = ft8.defaults()
+ options.slots = slots
+ for key, value in kw.items():
+ setattr(options, key, value)
+ heard = ft8.Heard()
+ heard.started = clock.now()
+ total = ft8.pump(band, options, heard, None, clock.now(),
+ clock=clock.now)
+ return heard, total
+
+
+def test_a_made_up_band_comes_out_the_other_end():
+ """The whole path: samples, sideband, slot boundaries, sync, decoding,
+ and into the book of stations."""
+ heard, total = _run(slots=2)
+ assert heard.slots == 2
+ assert total >= 8, f"only {total} decodes in two slots"
+ assert len(heard.band.stations) >= 6
+ for station in heard.band.all():
+ assert station.decodes >= 1
+ assert station.call and station.call != "<...>"
+
+
+def test_the_slot_boundaries_are_found_rather_than_assumed():
+ """A receiver that dated its samples a block out would slice every slot
+ late and cut the first half-second -- three symbols and part of the
+ opening sync -- off every transmission on the band."""
+ heard, _total = _run(slots=2)
+ assert heard.band.decodes
+ # Everything on this band starts within half a second of the nominal
+ # start, so everything should be reported within about that of zero.
+ worst = max(abs(d.offset) for d in heard.band.decodes)
+ assert worst < 1.0, f"the worst decode was {worst:.2f} s out"
+
+
+def test_callsigns_only_leaves_out_the_free_text():
+ heard, _ = _run(slots=1, calls_only=True)
+ assert all(d.kind in ("standard", "non-standard")
+ for d in heard.band.decodes)
+
+
+def test_the_search_can_be_narrowed_and_the_narrowing_is_obeyed():
+ heard, _ = _run(slots=1, lowest=800.0, highest=1600.0)
+ assert heard.band.decodes
+ assert all(800.0 <= d.hertz <= 1600.0 for d in heard.band.decodes)
+
+
+def test_a_station_that_calls_cq_is_counted_as_calling():
+ heard, _ = _run(slots=2)
+ calling = [s for s in heard.band.all() if s.calling]
+ assert calling, "nobody called CQ on a band that is mostly CQ"
+ assert all(s.grid for s in calling), "a CQ says where it is"
+
+
+# ---------------------------------------------------------------------------
+# Writing it down
+# ---------------------------------------------------------------------------
+
+def _decode(text="CQ DL1UDO JO31", **kw):
+ fields = dict(text=text, kind="standard", calls=("DL1UDO",),
+ grid="JO31", report="", calling=True, hertz=1200.0,
+ offset=0.2, snr_db=-7.0, at=1_700_000_000.0)
+ fields.update(kw)
+ return ft8wave.Decode(**fields)
+
+
+def test_a_log_is_written_and_read_back(tmp_path):
+ log = ft8log.open_log(tmp_path, 1_700_000_000.0, 14_074_000.0, "20m")
+ assert log is not None
+ for i in range(3):
+ log.append(_decode(hertz=1000.0 + i * 100))
+ log.close()
+ rows = ft8log.read_log(log.path)
+ assert len(rows) == 3
+ assert rows[0]["text"] == "CQ DL1UDO JO31"
+ assert rows[1]["hertz"] == 1100.0
+
+
+def test_a_half_written_log_line_is_skipped_rather_than_raising(tmp_path):
+ path = tmp_path / "ft8-part.txt"
+ path.write_text("# header\n2023-11-14 22:13:20 -7 0.2 1200.0 ~ OK\n"
+ "2023-11-14 22:13:35 -7 0.2\n")
+ rows = ft8log.read_log(path)
+ assert len(rows) == 1 and rows[0]["text"] == "OK"
+
+
+def test_a_csv_has_a_row_for_every_decode(tmp_path):
+ where = ft8log.write_csv(tmp_path / "x.csv",
+ [_decode(), _decode("W2XYZ K1ABC RRR")])
+ assert where is not None
+ body = where.read_text().splitlines()
+ assert len(body) == 3 and body[0].startswith("utc,")
+
+
+def test_an_adif_says_these_were_heard_rather_than_worked(tmp_path):
+ """Nothing here transmits, so nothing here is a contact. An ADIF that
+ let a logging program treat these as worked would put claims into
+ somebody's log that they cannot make."""
+ where = ft8log.write_adif(tmp_path / "x.adi",
+ [_decode(), _decode("W2XYZ K1ABC RRR",
+ calls=("W2XYZ", "K1ABC"))],
+ band="20m", frequency=14_074_000.0,
+ my_grid="IO91")
+ body = where.read_text()
+ assert "heard only, not worked" in body
+ assert "not contacts" in body
+ assert "DL1UDO" in body
+ assert "FT8" in body
+ assert "IO91" in body
+
+
+def test_an_adif_keeps_one_record_a_station_rather_than_one_a_decode(tmp_path):
+ many = [_decode(snr_db=s) for s in (-20.0, -3.0, -11.0)]
+ where = ft8log.write_adif(tmp_path / "y.adi", many)
+ body = where.read_text()
+ assert body.count("") == 1
+ # And the one kept is the best report, that being the one worth passing on.
+ assert "-3" in body
+
+
+# ---------------------------------------------------------------------------
+# The display
+# ---------------------------------------------------------------------------
+
+def test_the_display_says_what_it_is_waiting_for_before_anything_arrives():
+ from rich.console import Console
+ from bandsaunter.ui import Ft8Display
+
+ console = Console(width=160, force_terminal=False)
+ display = Ft8Display(console, band="20m — 14.074 MHz")
+ display.update(ft8.Heard())
+ with console.capture() as caught:
+ console.print(display.render())
+ shown = caught.get()
+ assert "nothing decoded yet" in shown
+ # And the one thing that silently stops FT8 working is worth saying.
+ assert "clock" in shown
+
+
+def test_the_display_shows_the_last_slot_and_the_running_total():
+ from rich.console import Console
+ from bandsaunter.ui import Ft8Display
+
+ console = Console(width=110, force_terminal=False)
+ display = Ft8Display(console, grid="IO91", band="20m — 14.074 MHz")
+ heard = ft8.Heard()
+ for text in ("CQ DL1UDO JO31", "W2XYZ K1ABC RRR"):
+ found = _decode(text, calls=tuple(text.split()[1:2] or ["DL1UDO"]),
+ grid="JO31" if "CQ" in text else "")
+ heard.band.add(found)
+ heard.decodes += 1
+ heard.latest = heard.band.decodes
+ heard.slots = 4
+ display.update(heard, 1_700_000_000.0)
+ with console.capture() as caught:
+ console.print(display.render(now=1_700_000_005.0))
+ shown = caught.get()
+ assert "last slot" in shown and "heard so far" in shown
+ assert "DL1UDO" in shown and "JO31" in shown
+ assert "4 slots" in shown
+
+
+def test_the_report_says_nothing_rather_than_an_empty_table():
+ from rich.console import Console
+
+ console = Console(width=100, force_terminal=False)
+ band = ft8.Band()
+ band.slots = 12
+ with console.capture() as caught:
+ ft8.report(console, band, ft8.defaults())
+ shown = caught.get()
+ assert "nothing decoded" in shown
+ assert "clock" in shown, "the usual cause is worth naming"
+
+
+def test_the_report_names_the_furthest_station_when_it_can():
+ from rich.console import Console
+
+ console = Console(width=120, force_terminal=False)
+ band = ft8.Band()
+ band.slots = 4
+ band.add(_decode("CQ VK4BLE QG62", calls=("VK4BLE",), grid="QG62"))
+ band.add(_decode("CQ DL1UDO JO31", calls=("DL1UDO",), grid="JO31"))
+ options = ft8.defaults()
+ options.grid = "IO91"
+ with console.capture() as caught:
+ ft8.report(console, band, options)
+ shown = caught.get()
+ assert "furthest heard" in shown and "VK4BLE" in shown
+
+
+def test_the_report_leaves_distances_out_when_it_has_nowhere_to_measure_from():
+ from rich.console import Console
+
+ console = Console(width=120, force_terminal=False)
+ band = ft8.Band()
+ band.add(_decode("CQ DL1UDO JO31", calls=("DL1UDO",), grid="JO31"))
+ with console.capture() as caught:
+ ft8.report(console, band, ft8.defaults())
+ shown = caught.get()
+ assert "DL1UDO" in shown and "away" not in shown
+
+
+# ---------------------------------------------------------------------------
+# Opening a receiver
+# ---------------------------------------------------------------------------
+
+def test_every_name_these_modules_import_actually_exists():
+ """Imports inside a function are not checked until that function runs.
+
+ Which is how `open_rtlsdr` -- a name nothing in this program has ever
+ exported -- sat in the one line that opens the receiver, past every
+ test, until somebody chose Listen and was told the section was broken.
+ The tests reached that line only through the simulator, which takes the
+ other branch.
+ """
+ import ast
+ import importlib
+
+ missing = []
+ for name in ("ft8", "ft8code", "ft8wave", "ft8log", "ft8sim",
+ "ft8tables"):
+ module = importlib.import_module(f"bandsaunter.{name}")
+ tree = ast.parse(Path(module.__file__).read_text())
+ for node in ast.walk(tree):
+ if not isinstance(node, ast.ImportFrom) or node.level != 1:
+ continue
+ if not node.module:
+ continue
+ target = importlib.import_module(f"bandsaunter.{node.module}")
+ for alias in node.names:
+ if not hasattr(target, alias.name):
+ missing.append(f"{name}: {node.module}.{alias.name}")
+ assert not missing, missing
+
+
+def test_the_made_up_band_needs_no_receiver_and_says_it_is_made_up():
+ from rich.console import Console
+
+ console = Console(width=90, force_terminal=False)
+ options = ft8.defaults()
+ options.simulate = True
+ with console.capture() as caught:
+ device = ft8.open_device(console, options)
+ assert isinstance(device, ft8sim.SimulatedBand)
+ assert "not there" in caught.get(), "a simulation must say so"
+ device.close()
+
+
+def test_a_receiver_that_cannot_be_opened_is_reported_rather_than_raising():
+ """A menu has to survive it, and the person has to be told which of the
+ two things went wrong: the receiver, or the listening."""
+ from rich.console import Console
+
+ console = Console(width=90, force_terminal=False)
+ options = ft8.defaults()
+ options.device = 99 # no such dongle
+ with console.capture() as caught:
+ assert ft8.open_device(console, options) is None
+ assert "cannot open the receiver" in caught.get()
+
+
+def test_nothing_claims_to_be_listening_before_there_is_a_receiver(tmp_path):
+ """Announcing the frequency and then failing to open a dongle reads as
+ though the listening went wrong, when what went wrong happened before
+ any of it started."""
+ from rich.console import Console
+
+ console = Console(width=90, force_terminal=False)
+ options = ft8.defaults()
+ options.device = 99
+ options.log = False
+ with console.capture() as caught:
+ assert ft8.listen(console, options, str(tmp_path)) == 0
+ shown = caught.get()
+ assert "cannot open the receiver" in shown
+ assert "listening on" not in shown