dustcouncil/bandsaunter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
bandsaunter/bandsaunter/ft8.py
The Dust Council 93120b80a6 Listen to FT8: fifteen seconds of everybody at once 
A new section, alongside the aircraft, the weather sensors and APRS.

FT8 is the odd one out among the things this program listens to, and the
reason is worth stating because it shapes everything below.  Every station on
the band transmits in the same quarter-minute slots, on the same dial
frequency, fifty hertz wide each, stacked across three kilohertz of audio.
One receiver parked on one frequency therefore hears the whole band's worth of
stations at once -- and hears most of them well below the noise, because half
of what is sent is error-correcting code.  That is the entire trick: a rate of
about one half buys a mode that decodes twenty-odd decibels under what an
operator can hear.  A receiver that took the loudest tone of each symbol and
hoped would decode almost nothing, which is why the tone detector reports how
confident it is bit by bit rather than what it thinks it heard.

Written from first principles except for two tables.  The checksum, the
belief propagation over the sparse graph, the Costas sync search, the
waterfall, the soft-bit metric, and the seventy-seven bits that hold two
callsigns and a grid square are all here.  The generator and the parity-check
matrix are not: they cannot be derived, being the code itself rather than
consequences of anything, so they are taken from ft8_lib under its MIT licence
with the attribution it asks for, and said so in the readme, the manual and
the file.  No decoding logic came with them.  That the two agree -- and they
are not derivable from one another, the generator's parity half running to
fifty-odd bits a row against the sparse matrix's six or seven -- is a test
rather than an assumption.

Tested against the air, not against itself.  Eleven off-air recordings with
published decodes: ninety-seven of a hundred and fifty messages, no false
decodes, timing within a hundredth of a second, frequency within a hertz,
signal reports within half a decibel on average.  The third not decoded are
the weakest in each slot; a mature decoder subtracts what it has decoded and
looks again in the remainder, and does ordered-statistics decoding where
belief propagation fails, and neither is built here.  What is here decodes
nothing that other receivers did not also hear, which is the property that
matters in a log.  Ten whole codewords lifted off the air are in the tests as
a permanent fixture, so the recordings can go missing and the regression
cannot.

Three things that looked like bugs and were not, and three that were.  The
half-second timing discrepancy was the convention: a transmission is 12.64
seconds in a slot of fifteen and everybody starts half a second in, so
lateness is reported against that.  Synthetic signals at known offsets proved
the clock self-consistent before anything was changed.  The signal reports
were twenty-one decibels optimistic because those recordings have a receiver
passband above three kilohertz, putting a whole-band median twelve to sixteen
decibels below the real noise floor -- so noise is now measured beside the
signal, and in the tone that was actually sent rather than the loudest of
eight, the largest of eight noisy numbers being well above their mean even
with no signal at all.  And the test transmitter was thirteen decibels
pessimistic, scaling its noise into a fifty-hertz reference instead of the
sampled bandwidth, which made the decoder look deaf when it was the test
signal that had been quietly attenuated.

The real bug the simulator caught was a one-block timestamp error: samples
were dated a block earlier than they were taken, which slid every slot slice a
second late and cut the first half-second -- three symbols, part of the
opening Costas array -- off every transmission on the band.  One decode a slot
became six.

Reachable both ways, as everything here is.  Twenty-one options, every one of
them a command-line flag and a line in the menu, both built from one table so
they cannot disagree -- and a test that says so, since an option in no group
would be settable from the command line and invisible in the menu.  The band
list says which channels a plain dongle can reach and which need an
upconverter, because almost all the activity is on shortwave and finding that
out by listening to silence for ten minutes is the wrong way to learn it.  The
default is two metres, which a plain dongle can hear.

--grid turns decodes into geography: every CQ says where it is, so each gets a
distance and a bearing and the furthest is named.  --adif writes the log in
the form every amateur logging program imports, marked as heard rather than
worked, because nothing here transmits and an ADIF that let a logging program
treat these as contacts would put claims into somebody's log that they cannot
make.

One bug shipped and found by being used rather than by being tested: the
line that opens the receiver called a function this program has never had.
Every test reached it through the simulator, which takes the other branch, so
the one line that matters to somebody with an aerial was the one line never
run.  There is now a check that every name these modules import actually
exists -- it names the missing one rather than failing somewhere downstream --
and two that say a receiver which cannot be opened is reported rather than
raised, and that nothing claims to be listening before there is one.  It had
been announcing the frequency first, so a dongle that would not open read as
listening that had gone wrong.

Ninety-six new tests.  Full suite 2781 passed.  Built as 2026-09-21_04.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PsWPTweCT6pwxKngvVxcg
2026-09-21 16:09:57 -07:00

919 lines
37 KiB
Python
Raw Blame History

"""Listening to FT8, from either front end.
Fifteen seconds of everybody at once. Every station on the band transmits
in the same quarter-minute slots, on the same dial frequency, fifty hertz
wide each, stacked across three kilohertz of audio -- so one receiver
parked on one frequency hears the whole band's worth of stations at the
same time, and hears most of them well below the noise.
That is why this is a section rather than something the scanner stops on.
A sweep catches whatever was keyed up as it went past; FT8 is forty
stations transmitting simultaneously for twelve and a half seconds out of
every fifteen, and the interesting thing is all of them.
The band plan is mostly shortwave, which a plain receiver of this kind
cannot reach without help -- so the default here is the two-metre channel
at 144.174 MHz, which it can. The shortwave channels are all in the list
and work perfectly well through an upconverter or a receiver in direct
sampling mode; they are simply not what you get by not choosing.
Nothing here transmits. This listens, decodes and writes down.
"""
from __future__ import annotations
import math
import time
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
from pathlib import Path
import numpy as np
import yaml
from . import ft8code as code
from . import ft8wave as wave
from .settings import Setting, format_value
__all__ = ["Ft8Options", "OPTIONS", "OPTION_GROUPS", "defaults", "in_group",
"by_key", "format_option", "describe", "summarise",
"Station", "Band", "Heard", "listen", "open_device", "open_log",
"pump", "finish", "report", "load_options", "save_options",
"options_path", "logs_in", "BANDS", "band_named", "band_text",
"use_band", "slot_start", "next_slot", "grid_at", "grid_away",
"SLOT", "audio_from"]
# ---------------------------------------------------------------------------
# Where FT8 lives
# ---------------------------------------------------------------------------
# name, hertz, what it needs. The dial frequency, in the usual convention:
# signals sit from about 200 to 3000 Hz above it.
BANDS: tuple[tuple[str, float, str], ...] = (
("160m", 1_840_000.0, "shortwave"),
("80m", 3_573_000.0, "shortwave"),
("60m", 5_357_000.0, "shortwave"),
("40m", 7_074_000.0, "shortwave"),
("30m", 10_136_000.0, "shortwave"),
("20m", 14_074_000.0, "shortwave"),
("17m", 18_100_000.0, "shortwave"),
("15m", 21_074_000.0, "shortwave"),
("12m", 24_915_000.0, "shortwave"),
("10m", 28_074_000.0, "shortwave"),
("6m", 50_313_000.0, "direct"),
("2m", 144_174_000.0, "direct"),
("70cm", 432_174_000.0, "direct"),
)
SLOT = code.SLOT_S # fifteen seconds, and everybody keeps to it
def band_named(name: str) -> float:
"""The dial frequency for a band, or the two-metre one."""
wanted = (name or "").strip().lower()
for key, hz, _needs in BANDS:
if wanted == key:
return hz
return 144_174_000.0
def band_needs(name: str) -> str:
wanted = (name or "").strip().lower()
for key, _hz, needs in BANDS:
if wanted == key:
return needs
return "direct"
def use_band(options: "Ft8Options", name: str) -> None:
"""Point the receiver at a band, frequency and all."""
options.band = (name or "").strip().lower()
options.frequency = band_named(options.band)
def band_text(options: "Ft8Options") -> str:
where = f"{options.frequency / 1e6:g} MHz"
needs = band_needs(options.band)
if needs == "shortwave":
return f"{options.band} — {where}, needs an upconverter"
return f"{options.band} — {where}"
# ---------------------------------------------------------------------------
# Slots
# ---------------------------------------------------------------------------
def slot_start(when: float | None = None) -> float:
"""The top of the fifteen-second slot that ``when`` falls in.
Against UTC, not against the local clock and not against when this
program happened to start: the whole protocol is built on every station
on earth agreeing about which quarter-minute it is. A receiver whose
clock is a second out still decodes -- the search covers a few seconds
either way -- but one that is a slot out hears every transmission
split across two captures and decodes none of them.
"""
when = time.time() if when is None else when
return math.floor(when / SLOT) * SLOT
def next_slot(when: float | None = None) -> float:
return slot_start(when) + SLOT
def slot_name(at: float) -> str:
"""The slot, as the six digits every FT8 program labels it with."""
return datetime.fromtimestamp(at, timezone.utc).strftime("%H%M%S")
# ---------------------------------------------------------------------------
# Grid squares
# ---------------------------------------------------------------------------
def grid_at(grid: str):
"""The middle of a Maidenhead square, as latitude and longitude.
Four characters is about seventy miles by a hundred, which is all the
position FT8 carries and quite enough to say which continent somebody
is on. Six characters is finer and turns up in free text rather than
in the position field.
"""
grid = (grid or "").strip().upper()
if len(grid) < 4 or not grid[:2].isalpha() or not grid[2:4].isdigit():
return None
if not ("A" <= grid[0] <= "R" and "A" <= grid[1] <= "R"):
return None
lon = (ord(grid[0]) - 65) * 20.0 + int(grid[2]) * 2.0
lat = (ord(grid[1]) - 65) * 10.0 + int(grid[3]) * 1.0
if len(grid) >= 6 and grid[4].isalpha() and grid[5].isalpha():
lon += (ord(grid[4]) - 65) * (2.0 / 24.0) + (1.0 / 24.0)
lat += (ord(grid[5]) - 65) * (1.0 / 24.0) + (0.5 / 24.0)
else:
lon += 1.0
lat += 0.5
return lat - 90.0, lon - 180.0
def grid_away(here: str, there: str):
"""How far apart two grid squares are, in kilometres, and on what bearing.
Great-circle, from the middles of the squares -- so it is good to the
size of a square and no better, which for FT8 is the honest answer and
is why nothing here prints a distance to the nearest kilometre.
"""
a, b = grid_at(here), grid_at(there)
if a is None or b is None:
return None
lat1, lon1 = math.radians(a[0]), math.radians(a[1])
lat2, lon2 = math.radians(b[0]), math.radians(b[1])
d = 2 * math.asin(math.sqrt(
math.sin((lat2 - lat1) / 2) ** 2
+ math.cos(lat1) * math.cos(lat2) * math.sin((lon2 - lon1) / 2) ** 2))
y = math.sin(lon2 - lon1) * math.cos(lat2)
x = (math.cos(lat1) * math.sin(lat2)
- math.sin(lat1) * math.cos(lat2) * math.cos(lon2 - lon1))
return 6371.0088 * d, (math.degrees(math.atan2(y, x)) + 360.0) % 360.0
# ---------------------------------------------------------------------------
# The options
# ---------------------------------------------------------------------------
@dataclass
class Ft8Options:
"""Everything the FT8 mode can be told, in one place."""
# -- receiver -------------------------------------------------------
device: int = 0
gain: str = "auto"
rate: float = 240_000.0
band: str = "2m"
frequency: float = 144_174_000.0
simulate: bool = False
# -- listening ------------------------------------------------------
seconds: float = 0.0
slots: int = 0 # or stop after this many slots
log: bool = True
decodes_seen: bool = False # a line per decode rather than a table
hold: float = 3600.0
# -- decoding -------------------------------------------------------
lowest: float = 200.0
highest: float = 3000.0
most: int = 300 # candidates examined per slot
rounds: int = 30 # how hard to try to repair one
# -- showing --------------------------------------------------------
grid: str = "" # where the aerial is, as a grid square
units: str = "metric"
calls_only: bool = False # leave out free text and telemetry
# -- afterwards -----------------------------------------------------
report: bool = True
csv: bool = False
adif: bool = False
@property
def imperial(self) -> bool:
return str(self.units).lower().startswith("imp")
def validate(self) -> list[str]:
out = []
if self.rate < 96_000:
out.append("the sample rate must be at least 96 kS/s to hold "
"three kilohertz of audio with room to filter it")
if self.seconds < 0 or self.slots < 0:
out.append("times cannot be negative")
if self.hold <= 0:
out.append("a station must stay on the display for some time")
if not 1e5 < self.frequency < 2e9:
out.append("that is not a frequency a receiver can be tuned to")
if self.highest <= self.lowest:
out.append("the top of the search must be above the bottom")
if self.highest - self.lowest < 100.0:
out.append("a search narrower than 100 Hz would miss almost "
"everything: one signal is 50 Hz wide and they are "
"spread across three kilohertz")
if self.most < 1:
out.append("at least one candidate has to be looked at")
if self.rounds < 1:
out.append("the error correction needs at least one pass")
if self.grid and grid_at(self.grid) is None:
out.append(f"{self.grid!r} is not a grid square — four "
"characters like IO91 or FN31")
return out
def to_dict(self) -> dict:
return asdict(self)
def defaults() -> Ft8Options:
return Ft8Options()
O = Setting
_BANDS = tuple(key for key, _hz, _needs in BANDS)
OPTIONS: tuple[Setting, ...] = (
# -- receiver -------------------------------------------------------
O("device", "Receiver", "Receiver", "int",
"which receiver to use, when more than one is plugged in",
"The index shown by `bandsaunter devices`. Zero unless you have "
"several dongles.",
minimum=0, flags=("--device",), example="0"),
O("gain", "Gain", "Receiver", "gain",
"tuner gain in dB, or automatic",
"FT8 is forty signals at once spanning fifty decibels of strength, "
"and what matters is that none of them clips rather than that any of "
"them is loud. Automatic copes. A fixed gain makes the signal "
"reports comparable between one evening and the next, which is the "
"whole point of them if you are judging an aerial or a band opening.",
flags=("--gain",), example="auto"),
O("rate", "Sample rate", "Receiver", "float",
"how fast to sample; 96 kS/s is the least that holds the channel",
"Only three kilohertz of it is used, so this is about having room to "
"filter cleanly rather than about bandwidth. The default costs "
"little and decimates to the twelve kilohertz the decoding wants by "
"a whole number.",
unit="Hz", minimum=96_000.0, flags=("--rate",), example="240000"),
O("band", "Band", "Receiver", "choice",
"which FT8 channel to listen on",
"Every band has one agreed dial frequency and everybody uses it. "
"Two metres is the default because a plain receiver of this kind can "
"reach it; the shortwave bands, which is where almost all the "
"activity is, need an upconverter or a receiver in direct sampling "
"mode. Choosing a band sets the frequency with it.",
choices=_BANDS, flags=("--band",), example="20m"),
O("frequency", "Dial frequency", "Receiver", "float",
"the dial frequency, if you want one the band list does not have",
"Signals sit from about two hundred to three thousand hertz above "
"this, which is the upper-sideband convention the whole mode is "
"built on. Setting a band sets this; setting this directly leaves "
"the band label alone, so use it for an upconverter's offset or a "
"channel that is not in the list.",
unit="Hz", minimum=1e5, maximum=2e9, flags=("--frequency", "--freq"),
example="14074000"),
O("simulate", "Simulate", "Receiver", "bool",
"invent a band instead of using a receiver",
"A made-up band of stations, on a real clock, for trying the display "
"and the report without an aerial. What it cannot tell you is "
"whether your receiver hears anything.",
flags=("--simulate",), off_flags=("--no-simulate",)),
# -- listening ------------------------------------------------------
O("seconds", "Listen for", "Listening", "float",
"how long to listen; zero means until stopped",
"Rounded up to a whole slot, because half a transmission decodes to "
"nothing at all.",
unit="s", minimum=0.0, flags=("--seconds",), example="600"),
O("slots", "Or this many slots", "Listening", "int",
"stop after this many fifteen-second slots; zero means no limit",
"The same thing as the time limit, counted the way the band counts "
"it. Whichever of the two is reached first stops the run.",
minimum=0, flags=("--slots",), example="40"),
O("log", "Write a log", "Listening", "bool",
"keep every decode in a file",
"One line per decode, with the slot, the frequency, how late it was "
"and how strongly it came in. The report afterwards is built from "
"this, and so is anything you want to do with it later.",
flags=("--log",), off_flags=("--no-log",)),
O("decodes_seen", "Print each decode", "Listening", "bool",
"a line per decode instead of a table that refreshes",
"What to use when the output is going into a pipe or a file. The "
"table is easier to watch; the lines are easier to grep.",
flags=("--decodes-seen",), off_flags=("--no-decodes-seen",)),
O("hold", "Keep on display for", "Listening", "float",
"how long a station stays on the table after its last decode",
"A station transmits every other slot at most, and many call once "
"and go away. Long enough that the table is a picture of the "
"evening rather than of the last thirty seconds.",
unit="s", minimum=1.0, flags=("--hold",), example="3600"),
# -- decoding -------------------------------------------------------
O("lowest", "Search from", "Decoding", "float",
"the lowest audio frequency to look for signals at",
"Below a couple of hundred hertz there is nothing but the "
"receiver's own rumble, and searching it costs time for nothing.",
unit="Hz", minimum=0.0, flags=("--lowest",), example="200"),
O("highest", "Search to", "Decoding", "float",
"the highest audio frequency to look for signals at",
"Three kilohertz is where a normal sideband receiver stops. Going "
"higher finds nothing unless your receiver is wider than that, and "
"costs time in proportion.",
unit="Hz", minimum=100.0, flags=("--highest",), example="3000"),
O("most", "Candidates per slot", "Decoding", "int",
"how many possible transmissions to try to decode each slot",
"The sync search ranks every place a transmission could be and this "
"is how far down the list to go. A busy shortwave band puts forty "
"real signals in a slot and several hundred plausible-looking "
"places; more than a few hundred has been measured to add nothing "
"but time.",
minimum=1, flags=("--most",), example="300"),
O("rounds", "Repair passes", "Decoding", "int",
"how many passes of error correction to make before giving up",
"The code either converges in twenty or so passes or it does not "
"converge at all -- raising this has been measured to find nothing "
"further, and it is here so that can be checked rather than taken "
"on trust.",
minimum=1, flags=("--rounds",), example="30"),
# -- showing --------------------------------------------------------
O("grid", "Aerial at", "Showing", "text",
"your own grid square, so distances can be worked out",
"Four characters, like IO91 or FN31. Every FT8 station that calls "
"CQ says where it is this way, so with your own square filled in "
"the report can say how far each of them is and on what bearing — "
"which on shortwave is the whole interest of the thing. Left blank "
"the decodes are still recorded; there is simply nothing to measure "
"them from.",
flags=("--grid",), example="IO91", metavar="SQUARE"),
O("units", "Show readings in", "Showing", "choice",
"metric or imperial, for the display and the export",
"Distances only. Signal reports are in decibels either way, that "
"being what the mode measures in and what gets sent back on the "
"air.",
choices=("metric", "imperial"), flags=("--units",), example="metric"),
O("calls_only", "Callsigns only", "Showing", "bool",
"leave out free text and telemetry",
"Most of what passes on FT8 is two callsigns and a report. The rest "
"is free text -- thirteen characters, no callsign structure, so "
"nothing can be said about who sent it -- and telemetry, which is "
"eighteen hex digits. Turn this on to see only the exchanges.",
flags=("--calls-only",), off_flags=("--no-calls-only",)),
# -- afterwards -----------------------------------------------------
O("report", "Report at the end", "Afterwards", "bool",
"print the tables when the listening stops",
"Who was heard, how far off, and what was exchanged.",
flags=("--report",), off_flags=("--no-report",)),
O("csv", "Also write a CSV", "Afterwards", "bool",
"a spreadsheet of every decode beside the log",
"The same rows as the log, in the form a spreadsheet opens without "
"being told anything.",
flags=("--csv",), off_flags=("--no-csv",)),
O("adif", "Also write an ADIF", "Afterwards", "bool",
"the log again, in the form logging programs read",
"ADIF is what every amateur logging program imports. What is "
"written is what was heard rather than a contact made -- nothing "
"here transmits, so nothing here is a QSO — so the records are "
"marked as received-only and are for feeding a propagation map or a "
"spotting tool rather than for claiming anything.",
flags=("--adif",), off_flags=("--no-adif",)),
)
OPTION_GROUPS = ("Receiver", "Listening", "Decoding", "Showing", "Afterwards")
def in_group(group: str) -> tuple[Setting, ...]:
return tuple(o for o in OPTIONS if o.group == group)
def by_key(key: str):
for option in OPTIONS:
if option.key == key:
return option
return None
# Options where nothing set means something, rather than nothing.
_ZERO_MEANS = {
"seconds": "until stopped",
"slots": "no limit",
"grid": "not set, so no distances",
}
def format_option(option: Setting, value) -> str:
"""Render an option the way the menu and the manual should show it."""
if not value and option.key in _ZERO_MEANS:
return _ZERO_MEANS[option.key]
return format_value(option, value)
def describe(options: Ft8Options) -> str:
"""The one-line summary the menu shows beside Listen."""
bits = [band_text(options)]
if options.seconds:
bits.append(f"{options.seconds:g} s")
elif options.slots:
bits.append(f"{options.slots} slots")
if options.grid:
bits.append(f"from {options.grid.upper()}")
if options.simulate:
bits.append("simulated")
return ", ".join(bits)
def summarise(options: Ft8Options) -> str:
return describe(options)
def options_path(directory=None) -> Path:
from .config import DEFAULT_CONFIG_DIR
return Path(directory or DEFAULT_CONFIG_DIR) / "ft8.yaml"
def load_options(directory=None) -> Ft8Options:
"""The saved options, or the defaults. A broken file is not an error."""
options = Ft8Options()
try:
body = yaml.safe_load(options_path(directory).read_text()) or {}
except (OSError, ValueError, yaml.YAMLError):
return options
if not isinstance(body, dict):
return options
known = set(options.__dict__)
for key, value in body.items():
if key in known and value is not None:
try:
setattr(options, key, type(getattr(options, key))(value))
except (TypeError, ValueError):
pass
return options
def save_options(options: Ft8Options, directory=None) -> Path:
path = options_path(directory)
path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w") as fh:
yaml.safe_dump(options.to_dict(), fh, sort_keys=False,
default_flow_style=False)
return path
def logs_in(directory) -> list[Path]:
try:
return sorted(Path(directory).glob("ft8-*.txt"))
except OSError:
return []
# ---------------------------------------------------------------------------
# Who was heard
# ---------------------------------------------------------------------------
@dataclass
class Station:
"""One callsign, and everything heard from it."""
call: str = ""
grid: str = ""
first: float = 0.0
last: float = 0.0
decodes: int = 0
best_snr: float = -99.0
worst_snr: float = 99.0
hertz: float = 0.0
calling: int = 0 # how many times it called CQ
worked: set = field(default_factory=set) # who it was talking to
said: list = field(default_factory=list) # the lines, newest last
def away(self, here: str):
"""How far off and on what bearing, if both squares are known."""
if not here or not self.grid:
return None
return grid_away(here, self.grid)
@dataclass
class Band:
"""Every station heard, and every exchange between them.
Kept apart the way the APRS section keeps messages apart from stations,
and for the same reason: a decode is about a moment and a station is
about an evening, and a table that tried to be both would be neither.
"""
stations: dict = field(default_factory=dict)
decodes: list = field(default_factory=list)
book: code.CallBook = field(default_factory=code.CallBook)
slots: int = 0
def add(self, found) -> None:
self.decodes.append(found)
# The caller is the second callsign in a standard exchange -- the
# first is who is being answered. A CQ has no first, so the one
# station named is the one transmitting.
sender = found.calls[1] if len(found.calls) > 1 else (
found.calls[0] if found.calls else "")
if not sender or sender == "<...>":
return
station = self.stations.get(sender)
if station is None:
station = Station(call=sender, first=found.at)
self.stations[sender] = station
station.last = found.at
station.decodes += 1
station.hertz = found.hertz
station.best_snr = max(station.best_snr, found.snr_db)
station.worst_snr = min(station.worst_snr, found.snr_db)
if found.grid:
station.grid = found.grid
if found.calling:
station.calling += 1
elif len(found.calls) > 1 and found.calls[0] != "<...>":
station.worked.add(found.calls[0])
station.said.append(found.text)
if len(station.said) > 32:
del station.said[0]
def all(self) -> list:
return list(self.stations.values())
@dataclass
class Heard:
"""What the front ends share: the running totals of one run."""
band: Band = field(default_factory=Band)
decodes: int = 0
slots: int = 0
started: float = 0.0
latest: list = field(default_factory=list)
# ---------------------------------------------------------------------------
# The receiver
# ---------------------------------------------------------------------------
AUDIO_RATE = 12_000 # what the decoding wants, and a whole division
def open_device(console, options: Ft8Options):
"""The receiver, or a made-up band, or None if neither can be had."""
from rich.panel import Panel
from rich.text import Text
from .device import RtlSdrDevice, RtlSdrError
if options.simulate:
console.print("[yellow]simulated: these stations are not there."
"[/yellow]")
from .ft8sim import SimulatedBand
return SimulatedBand(rate=options.rate, realtime=True)
try:
device = RtlSdrDevice(index=options.device,
sample_rate=int(options.rate),
gain=options.gain,
agc=options.gain == "auto",
# Which is what makes the shortwave bands
# reachable at all on a dongle that supports
# it: below 24 MHz the tuner is bypassed and
# the sampler is fed straight off the aerial.
direct_sampling="auto")
device.open()
except RtlSdrError as exc:
console.print(Panel(Text(str(exc)),
title="[red]cannot open the receiver",
border_style="red"))
return None
return device
def audio_from(samples, options: Ft8Options, demod=None):
"""Upper-sideband audio at twelve kilohertz, from the receiver's samples.
Upper sideband because that is the convention the whole mode is built
on: the dial frequency is the bottom edge and every signal sits above
it, which is why two stations three hundred hertz apart are two
different signals rather than the same one heard twice.
"""
from .demod import make_demodulator
if demod is None:
demod = make_demodulator("usb", sample_rate=int(options.rate),
bandwidth=3200.0, audio_rate=AUDIO_RATE)
audio, _iq = demod.step(samples)
return audio, demod
def pump(device, options: Ft8Options, heard: Heard, log, started: float,
on_slot=None, on_decode=None, stopping=None, clock=time.time) -> int:
"""Read the receiver, a slot at a time, until it stops or is told to.
The one loop both front ends are driven from, so that what is written
to the log cannot depend on which one you happened to be looking at.
Audio is collected continuously and cut on the slot boundaries rather
than being captured slot by slot. A receiver that started reading when
the slot began would miss however long it took to start reading, and
everything on the band begins in the first half-second.
"""
demod = None
audio = np.zeros(0, dtype=np.float32)
audio_at = 0.0 # wall time of audio[0]
want = next_slot(clock()) # the first whole slot we can get
total = 0
device.tune(options.frequency)
block = int(options.rate) # a second of samples at a time
while stopping is None or not stopping():
at = clock()
samples = device.read_samples(block)
if samples is None or samples.size == 0:
break
fresh, demod = audio_from(samples, options, demod)
if fresh.size:
if audio.size == 0:
# The clock was read before asking for the samples, and the
# read returns once they have arrived -- so they cover the
# block that *starts* here. Dating them a block earlier
# instead slides every slice a second late, which cuts the
# first half-second off every transmission: three symbols,
# part of the opening Costas array, and most of the
# decodes with it.
audio_at = at
audio = np.concatenate([audio, fresh])
# Every whole slot now sitting in the buffer.
while audio.size and audio_at <= want and \
audio_at + audio.size / AUDIO_RATE >= want + SLOT:
first = int(round((want - audio_at) * AUDIO_RATE))
piece = audio[first:first + int(round(SLOT * AUDIO_RATE))]
found = wave.listen_to(piece, AUDIO_RATE, book=heard.band.book,
most=options.most, rounds=options.rounds,
at=want)
found = [d for d in found
if options.lowest <= d.hertz <= options.highest]
if options.calls_only:
found = [d for d in found if d.kind in ("standard",
"non-standard")]
heard.slots += 1
heard.band.slots += 1
heard.latest = found
for d in found:
heard.band.add(d)
heard.decodes += 1
total += 1
if log is not None:
log.append(d)
if on_decode is not None:
on_decode(d)
if on_slot is not None:
on_slot(want, found)
want += SLOT
# Keep a little before the next slot: a transmission can start
# before the slot it belongs to.
keep = int(round(max(0.0, want - 3.0 - audio_at) * AUDIO_RATE))
if keep > 0:
audio = audio[keep:]
audio_at += keep / AUDIO_RATE
if options.seconds and clock() - started >= options.seconds:
break
if options.slots and heard.slots >= options.slots:
break
return total
def open_log(console, options: Ft8Options, output_dir, started: float):
from . import ft8log
if not options.log:
return None
log = ft8log.open_log(output_dir, started, options.frequency,
options.band)
if log is None:
console.print("[yellow]could not open a log; carrying on without "
"one[/yellow]")
return log
def listen(console, options: Ft8Options, output_dir: str,
heard: Heard | None = None, stopping=None) -> int:
"""Listen on one channel and write down everything decoded."""
from .ui import Ft8Display
started = time.time()
heard = heard if heard is not None else Heard()
heard.started = started
# The receiver first. Announcing that it is listening and then failing
# to open a dongle reads as though the listening went wrong, when what
# went wrong happened before any of it started.
device = open_device(console, options)
if device is None:
return 0
log = open_log(console, options, output_dir, started)
_say_where(console, options)
console.print(f"[grey62]listening on {band_text(options)} at "
f"{options.rate / 1e6:g} MS/s — control-C to stop"
"[/grey62]")
if not options.decodes_seen:
console.print("[grey62]the first decodes appear when the slot after "
"next ends, which is up to thirty seconds away"
"[/grey62]")
display = None
live = None
if not options.decodes_seen and getattr(console, "is_terminal", False):
from rich.live import Live
display = Ft8Display(console, hold=options.hold,
imperial=options.imperial, grid=options.grid,
band=band_text(options))
display.started = started
live = Live(display.render(), console=console, refresh_per_second=2,
screen=False, transient=False, vertical_overflow="crop")
live.start()
def show_slot(at, found):
if display is not None:
display.update(heard, at)
live.update(display.render())
def show_decode(d):
if options.decodes_seen:
console.print(f"{slot_name(d.at)} {d.snr_db:>4.0f} "
f"{d.offset:>5.1f} {d.hertz:>7.1f} ~ {d.text}")
try:
pump(device, options, heard, log, started, on_slot=show_slot,
on_decode=show_decode, stopping=stopping)
except KeyboardInterrupt:
console.print("\n[grey62]stopped[/grey62]")
finally:
if live is not None:
live.stop()
try:
device.close()
except Exception:
pass
return finish(console, options, output_dir, heard, log)
def _say_where(console, options: Ft8Options) -> None:
"""Whether distances can be worked out, and what it costs if not."""
if options.grid:
where = grid_at(options.grid)
console.print(f"[grey62]aerial at {options.grid.upper()} "
f"({where[0]:.2f},{where[1]:.2f}) — distances and "
f"bearings will be worked out[/grey62]")
return
console.print("[grey62]no grid square set, so no distances and no "
"bearings — `--grid IO91` gives both[/grey62]")
if band_needs(options.band) == "shortwave":
console.print("[grey62]this band is shortwave: a plain receiver "
"needs an upconverter or direct sampling to hear it"
"[/grey62]")
def finish(console, options: Ft8Options, output_dir: str, heard: Heard,
log=None) -> int:
"""Close the log, write the exports, and print the report."""
from . import ft8log
if log is not None:
log.close()
console.print(f"[grey62]{log.lines} decodes written to "
f"{log.path}[/grey62]")
if options.csv:
where = ft8log.write_csv(log.path.with_suffix(".csv"),
heard.band.decodes)
if where is not None:
console.print(f"[grey62]and as a spreadsheet: {where}"
"[/grey62]")
if options.adif:
where = ft8log.write_adif(log.path.with_suffix(".adi"),
heard.band.decodes, band=options.band,
frequency=options.frequency,
my_grid=options.grid)
if where is not None:
console.print(f"[grey62]and as ADIF, marked as heard rather "
f"than worked: {where}[/grey62]")
if options.report:
report(console, heard.band, options)
return heard.decodes
def _away_text(station: Station, options: Ft8Options) -> str:
away = station.away(options.grid)
if away is None:
return ""
km, bearing = away
if options.imperial:
return f"{km * 0.621371:.0f} mi {bearing:.0f}°"
return f"{km:.0f} km {bearing:.0f}°"
def report(console, band: Band, options: Ft8Options) -> None:
"""Three tables, because they answer three different questions."""
from rich.table import Table
stations = sorted(band.all(), key=lambda s: (-s.decodes, s.call))
if not stations and not band.decodes:
console.print("[yellow]nothing decoded[/yellow]")
if band.slots:
console.print("[grey62]the band was quiet, the aerial is not "
"hearing it, or the clock is out: FT8 needs the "
"clock right to a second or two[/grey62]")
return
console.print()
console.print(f"[bold]Stations heard[/bold] — {len(stations)} in "
f"{band.slots} slots")
t = Table(box=None, header_style="bold", pad_edge=False)
t.add_column("station")
t.add_column("grid")
if options.grid:
t.add_column("away")
t.add_column("decodes", justify="right")
t.add_column("best", justify="right")
t.add_column("worst", justify="right")
t.add_column("audio", justify="right")
t.add_column("CQ", justify="right")
for s in stations[:60]:
row = [s.call, s.grid or "—"]
if options.grid:
row.append(_away_text(s, options) or "—")
row += [str(s.decodes), f"{s.best_snr:.0f}", f"{s.worst_snr:.0f}",
f"{s.hertz:.0f} Hz", str(s.calling) if s.calling else ""]
t.add_row(*row)
console.print(t)
if len(stations) > 60:
console.print(f"[grey62]…and {len(stations) - 60} more[/grey62]")
calling = [s for s in stations if s.calling]
if calling:
console.print()
console.print(f"[bold]Calling CQ[/bold] — {len(calling)} stations "
"looking for a contact")
t = Table(box=None, header_style="bold", pad_edge=False)
t.add_column("station")
t.add_column("grid")
if options.grid:
t.add_column("away")
t.add_column("times", justify="right")
t.add_column("best", justify="right")
for s in sorted(calling, key=lambda s: -s.calling)[:30]:
row = [s.call, s.grid or "—"]
if options.grid:
row.append(_away_text(s, options) or "—")
row += [str(s.calling), f"{s.best_snr:.0f}"]
t.add_row(*row)
console.print(t)
worked = [s for s in stations if s.worked]
if worked:
console.print()
console.print("[bold]Who was working whom[/bold] — the exchanges "
"that passed while this was listening")
t = Table(box=None, header_style="bold", pad_edge=False)
t.add_column("station")
t.add_column("answered")
for s in sorted(worked, key=lambda s: -len(s.worked))[:30]:
t.add_row(s.call, ", ".join(sorted(s.worked)[:8]))
console.print(t)
if options.grid:
far = [(s, s.away(options.grid)) for s in stations if s.grid]
far = [(s, a) for s, a in far if a is not None]
if far:
s, (km, bearing) = max(far, key=lambda x: x[1][0])
unit = (f"{km * 0.621371:.0f} miles" if options.imperial
else f"{km:.0f} km")
console.print()
console.print(f"[grey62]furthest heard: {s.call} in {s.grid}, "
f"{unit} on {bearing:.0f}°[/grey62]")
Reference in a new issue View git blame Copy permalink