Listen to FT8: fifteen seconds of everybody at once
A new section, alongside the aircraft, the weather sensors and APRS. FT8 is the odd one out among the things this program listens to, and the reason is worth stating because it shapes everything below. Every station on the band transmits in the same quarter-minute slots, on the same dial frequency, fifty hertz wide each, stacked across three kilohertz of audio. One receiver parked on one frequency therefore hears the whole band's worth of stations at once -- and hears most of them well below the noise, because half of what is sent is error-correcting code. That is the entire trick: a rate of about one half buys a mode that decodes twenty-odd decibels under what an operator can hear. A receiver that took the loudest tone of each symbol and hoped would decode almost nothing, which is why the tone detector reports how confident it is bit by bit rather than what it thinks it heard. Written from first principles except for two tables. The checksum, the belief propagation over the sparse graph, the Costas sync search, the waterfall, the soft-bit metric, and the seventy-seven bits that hold two callsigns and a grid square are all here. The generator and the parity-check matrix are not: they cannot be derived, being the code itself rather than consequences of anything, so they are taken from ft8_lib under its MIT licence with the attribution it asks for, and said so in the readme, the manual and the file. No decoding logic came with them. That the two agree -- and they are not derivable from one another, the generator's parity half running to fifty-odd bits a row against the sparse matrix's six or seven -- is a test rather than an assumption. Tested against the air, not against itself. Eleven off-air recordings with published decodes: ninety-seven of a hundred and fifty messages, no false decodes, timing within a hundredth of a second, frequency within a hertz, signal reports within half a decibel on average. The third not decoded are the weakest in each slot; a mature decoder subtracts what it has decoded and looks again in the remainder, and does ordered-statistics decoding where belief propagation fails, and neither is built here. What is here decodes nothing that other receivers did not also hear, which is the property that matters in a log. Ten whole codewords lifted off the air are in the tests as a permanent fixture, so the recordings can go missing and the regression cannot. Three things that looked like bugs and were not, and three that were. The half-second timing discrepancy was the convention: a transmission is 12.64 seconds in a slot of fifteen and everybody starts half a second in, so lateness is reported against that. Synthetic signals at known offsets proved the clock self-consistent before anything was changed. The signal reports were twenty-one decibels optimistic because those recordings have a receiver passband above three kilohertz, putting a whole-band median twelve to sixteen decibels below the real noise floor -- so noise is now measured beside the signal, and in the tone that was actually sent rather than the loudest of eight, the largest of eight noisy numbers being well above their mean even with no signal at all. And the test transmitter was thirteen decibels pessimistic, scaling its noise into a fifty-hertz reference instead of the sampled bandwidth, which made the decoder look deaf when it was the test signal that had been quietly attenuated. The real bug the simulator caught was a one-block timestamp error: samples were dated a block earlier than they were taken, which slid every slot slice a second late and cut the first half-second -- three symbols, part of the opening Costas array -- off every transmission on the band. One decode a slot became six. Reachable both ways, as everything here is. Twenty-one options, every one of them a command-line flag and a line in the menu, both built from one table so they cannot disagree -- and a test that says so, since an option in no group would be settable from the command line and invisible in the menu. The band list says which channels a plain dongle can reach and which need an upconverter, because almost all the activity is on shortwave and finding that out by listening to silence for ten minutes is the wrong way to learn it. The default is two metres, which a plain dongle can hear. --grid turns decodes into geography: every CQ says where it is, so each gets a distance and a bearing and the furthest is named. --adif writes the log in the form every amateur logging program imports, marked as heard rather than worked, because nothing here transmits and an ADIF that let a logging program treat these as contacts would put claims into somebody's log that they cannot make. One bug shipped and found by being used rather than by being tested: the line that opens the receiver called a function this program has never had. Every test reached it through the simulator, which takes the other branch, so the one line that matters to somebody with an aerial was the one line never run. There is now a check that every name these modules import actually exists -- it names the missing one rather than failing somewhere downstream -- and two that say a receiver which cannot be opened is reported rather than raised, and that nothing claims to be listening before there is one. It had been announcing the frequency first, so a dongle that would not open read as listening that had gone wrong. Ninety-six new tests. Full suite 2781 passed. Built as 2026-09-21_04. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016PsWPTweCT6pwxKngvVxcg
This commit is contained in:
The Dust Council
parent
e203b3e581
commit
93120b80a6
14 changed files with 4247 additions and 3 deletions
105
README.md
105
README.md
|
|
@ -2630,6 +2630,93 @@ dongles ship with; the stock telescopic one extended properly does well.
|
|||
`--packets` shows each frame as it arrives, which is the thing to watch while
|
||||
moving an aerial about.
|
||||
|
||||
## FT8
|
||||
|
||||
Fifteen seconds of everybody at once.
|
||||
|
||||
```
|
||||
bandsaunter ft8 --band 20m --grid IO91
|
||||
```
|
||||
|
||||
Every station on the band transmits in the same quarter-minute slots, on the
|
||||
same dial frequency, fifty hertz wide each, stacked across three kilohertz of
|
||||
audio. So one receiver parked on one frequency hears the whole band's worth
|
||||
of stations simultaneously — and hears most of them **well below the noise**.
|
||||
|
||||
```
|
||||
FT8 20m — 14.074 MHz 12 slots 187 decodes 63 stations 15.6 a slot 3:00 from IO91
|
||||
|
||||
last slot — 16 decoded
|
||||
snr dt hz message
|
||||
-19 -0.5 309 G4CUS SP4FCA RRR
|
||||
-6 0.1 528 VK3EVE SQ3MZM RR73
|
||||
18 0.1 1110 9A9TT IK4LZH -10
|
||||
6 0.0 2535 CQ IZ3XJM JN55
|
||||
|
||||
heard so far — 63 stations
|
||||
station grid away n best hz last
|
||||
IZ3XJM JN55 1180 km 4 6 2535 2s
|
||||
IK4LZH JN54 1140 km 7 18 1110 2s
|
||||
```
|
||||
|
||||
Half of what is transmitted is error-correcting code, and that is the whole
|
||||
trick: it is what buys a mode that decodes twenty-odd decibels under what an
|
||||
operator can hear. A receiver that took the loudest tone of each symbol and
|
||||
hoped would decode almost nothing.
|
||||
|
||||
### What it needs
|
||||
|
||||
**The clock has to be right**, to a second or two. The slots are
|
||||
quarter-minutes of UTC and every station on earth agrees about which one it
|
||||
is. A receiver a second out still decodes; one a slot out hears every
|
||||
transmission split across two captures and decodes none of them. This is the
|
||||
one failure that looks exactly like a dead band, so the display and the
|
||||
report both say so when nothing arrives.
|
||||
|
||||
**Almost all the activity is on shortwave**, which a plain dongle cannot
|
||||
reach. The default here is therefore the two-metre channel at 144.174 MHz,
|
||||
which it can. All thirteen channels are in the list and the shortwave ones
|
||||
work perfectly well through an upconverter or a receiver in direct-sampling
|
||||
mode — the menu says which is which rather than leaving you to find out by
|
||||
listening to silence.
|
||||
|
||||
### What comes out
|
||||
|
||||
`--grid IO91` is what turns decodes into geography: every station calling CQ
|
||||
says where it is, so with your own square filled in the report gives each one
|
||||
a distance and a bearing, and names the furthest heard. On shortwave that is
|
||||
the entire interest of the thing.
|
||||
|
||||
Three tables afterwards: **stations heard** with grid, distance, count and
|
||||
best and worst report; **calling CQ**, which is who is available; and **who
|
||||
was working whom**, which is the band as a social event rather than a list.
|
||||
|
||||
`--adif` writes the log again in the form every amateur logging program
|
||||
imports — marked as *heard*, not worked. Nothing here transmits, so nothing
|
||||
here is a contact, and an ADIF that let a logging program treat these as
|
||||
worked would put claims into somebody's log they cannot make.
|
||||
|
||||
### How well it works
|
||||
|
||||
Checked against eleven off-air recordings with published decodes, which is
|
||||
the only honest way to test a decoder — an encoder tested against its own
|
||||
decoder agrees with it about anything they are both wrong about, a lesson
|
||||
this project learned expensively on 433 MHz.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| messages decoded | 97 of 150 (65%), **no false decodes** |
|
||||
| timing | +0.00 s, spread 0.04 |
|
||||
| frequency | +0.1 Hz, spread 1.1 |
|
||||
| signal report | −0.4 dB, spread 4.8 |
|
||||
| weakest decoded | −24 dB on this program's own scale |
|
||||
|
||||
The 35% not decoded are the weakest signals in each slot. A mature decoder
|
||||
subtracts each signal it decodes and looks again in what is left, and does
|
||||
ordered-statistics decoding when belief propagation fails; neither is built
|
||||
here. What *is* here decodes nothing that other receivers did not also hear,
|
||||
which is the property that matters in a log.
|
||||
|
||||
## Meters on 900 MHz
|
||||
|
||||
A scan of 902–928 MHz that turns up a burst gets it named rather than reported
|
||||
|
|
@ -3439,3 +3526,21 @@ by the Free Software Foundation. It is distributed in the hope that it will be
|
|||
useful, but with no warranty whatsoever — not even the implied warranty of
|
||||
merchantability or fitness for a particular purpose. The full text is in
|
||||
[LICENSE](LICENSE), and at <https://www.gnu.org/licenses/>.
|
||||
|
||||
### Borrowed material
|
||||
|
||||
One file is not original work. `bandsaunter/ft8tables.py` holds the two fixed
|
||||
tables that *define* the FT8 error-correcting code — the generator and the
|
||||
sparse parity-check matrix — taken from
|
||||
[ft8_lib](https://github.com/kgoba/ft8_lib), MIT licensed, copyright © 2018
|
||||
Kārlis Goba, which took them in turn from WSJT-X. They are reproduced under
|
||||
the MIT terms, which permit it, and that file carries the attribution they
|
||||
ask for.
|
||||
|
||||
They are there because they cannot be derived. Everything else about FT8 in
|
||||
this program is worked out from first principles — the tones, the sync, the
|
||||
parity arithmetic, the belief propagation, the way a callsign is squeezed
|
||||
into twenty-eight bits — but those two tables are not derived from anything.
|
||||
They are the code itself, chosen once by its designers and published, and a
|
||||
receiver that guessed at them would be speaking a different protocol. No
|
||||
decoding logic was taken.
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ and transcribing speech.
|
|||
# 2026-08-21_02 is the second build made on the 21st. The revision is padded
|
||||
# to two digits so versions sort as text.
|
||||
VERSION_DATE = "2026-09-21"
|
||||
VERSION_REVISION = 3
|
||||
VERSION_REVISION = 4
|
||||
|
||||
__version__ = f"{VERSION_DATE}_{VERSION_REVISION:02d}"
|
||||
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ from .device import RtlSdrError, list_devices, set_driver_messages
|
|||
from .librtlsdr import load_error
|
||||
from . import settings as st
|
||||
from .ax25 import APRS_CHANNELS as _APRS_CHANNELS
|
||||
from .ft8 import BANDS as _FT8_BANDS
|
||||
from .ranges import (RangeError, ScanRange, build_plan, parse_range_list)
|
||||
from .scanner import Scanner, ScannerCallbacks
|
||||
from .tui import TUIAbort, first_run_setup, run_tui, settings_menu
|
||||
|
|
@ -639,6 +640,74 @@ examples:
|
|||
a.add_argument("--freq", type=float, default=0.0,
|
||||
help="centre frequency in Hz, for band-aware naming")
|
||||
a.add_argument("--morse", action="store_true", help="force a CW decode")
|
||||
|
||||
# -- ft8 ------------------------------------------------------------------
|
||||
f8 = sub.add_parser("ft8",
|
||||
help="listen to FT8: fifteen-second slots, forty "
|
||||
"stations at once, most of them under the noise")
|
||||
f8.add_argument("--device", type=int, default=None, help="which receiver")
|
||||
f8.add_argument("--gain", default=None, help="tuner gain in dB, or auto")
|
||||
f8.add_argument("--rate", type=float, default=None,
|
||||
help="sample rate in Hz; 96 kS/s is the least that holds "
|
||||
"three kilohertz of audio")
|
||||
f8.add_argument("--band", default=None,
|
||||
choices=[key for key, _hz, _n in _FT8_BANDS],
|
||||
help="which FT8 channel to listen on; sets the frequency")
|
||||
f8.add_argument("--frequency", "--freq", dest="frequency", type=float,
|
||||
default=None, metavar="HZ",
|
||||
help="the dial frequency, for an upconverter or a "
|
||||
"channel the band list does not have")
|
||||
f8.add_argument("--simulate", dest="simulate", action="store_true",
|
||||
default=None, help="invent a band instead of using a "
|
||||
"receiver")
|
||||
f8.add_argument("--no-simulate", dest="simulate", action="store_false",
|
||||
help="use the receiver (the default)")
|
||||
f8.add_argument("--seconds", type=float, default=None,
|
||||
help="stop after this long (default: until interrupted)")
|
||||
f8.add_argument("--slots", type=int, default=None, metavar="N",
|
||||
help="stop after this many fifteen-second slots")
|
||||
f8.add_argument("--log", dest="log", action="store_true", default=None,
|
||||
help="write a log of every decode (the default)")
|
||||
f8.add_argument("--no-log", dest="log", action="store_false",
|
||||
help="do not write a log")
|
||||
f8.add_argument("--decodes-seen", dest="decodes_seen",
|
||||
action="store_true", default=None,
|
||||
help="print a line per decode instead of a live table")
|
||||
f8.add_argument("--no-decodes-seen", dest="decodes_seen",
|
||||
action="store_false", help="show the live table")
|
||||
f8.add_argument("--hold", type=float, default=None, metavar="SECONDS",
|
||||
help="how long a station stays on the display")
|
||||
f8.add_argument("--lowest", type=float, default=None, metavar="HZ",
|
||||
help="the lowest audio frequency to search")
|
||||
f8.add_argument("--highest", type=float, default=None, metavar="HZ",
|
||||
help="the highest audio frequency to search")
|
||||
f8.add_argument("--most", type=int, default=None, metavar="N",
|
||||
help="how many candidate transmissions to try per slot")
|
||||
f8.add_argument("--rounds", type=int, default=None, metavar="N",
|
||||
help="how many error-correction passes before giving up")
|
||||
f8.add_argument("--grid", default=None, metavar="SQUARE",
|
||||
help="your own grid square, so distances can be worked "
|
||||
"out (four characters, like IO91)")
|
||||
f8.add_argument("--units", default=None, choices=("metric", "imperial"),
|
||||
help="which units to show distances in")
|
||||
f8.add_argument("--calls-only", dest="calls_only", action="store_true",
|
||||
default=None, help="leave out free text and telemetry")
|
||||
f8.add_argument("--no-calls-only", dest="calls_only",
|
||||
action="store_false", help="show everything decoded")
|
||||
f8.add_argument("--report", dest="report", action="store_true",
|
||||
default=None, help="print the tables at the end")
|
||||
f8.add_argument("--no-report", dest="report", action="store_false",
|
||||
help="do not print the tables")
|
||||
f8.add_argument("--csv", dest="csv", action="store_true", default=None,
|
||||
help="also write a spreadsheet of every decode")
|
||||
f8.add_argument("--no-csv", dest="csv", action="store_false",
|
||||
help="do not write a spreadsheet")
|
||||
f8.add_argument("--adif", dest="adif", action="store_true", default=None,
|
||||
help="also write an ADIF of stations heard, for a "
|
||||
"logging program")
|
||||
f8.add_argument("--no-adif", dest="adif", action="store_false",
|
||||
help="do not write an ADIF")
|
||||
|
||||
return p
|
||||
|
||||
|
||||
|
|
@ -2151,6 +2220,7 @@ def main(argv=None) -> int:
|
|||
"flights": cmd_flights, "weather": cmd_weather,
|
||||
"readings": cmd_readings, "sensors": cmd_sensors,
|
||||
"aprs": cmd_aprs, "packets": cmd_packets,
|
||||
"ft8": cmd_ft8,
|
||||
}
|
||||
try:
|
||||
return handlers[args.command](args)
|
||||
|
|
@ -2161,5 +2231,42 @@ def main(argv=None) -> int:
|
|||
return 0
|
||||
|
||||
|
||||
def cmd_ft8(args) -> int:
|
||||
"""Park on an FT8 channel and decode every slot.
|
||||
|
||||
Fifteen seconds of everybody at once: one dial frequency carries the
|
||||
whole band's worth of stations, fifty hertz apart across three
|
||||
kilohertz of audio, and most of them arrive below the noise.
|
||||
"""
|
||||
from . import ft8
|
||||
|
||||
cfg, _ = load_default()
|
||||
options = ft8.load_options()
|
||||
for flag, key in (("device", "device"), ("gain", "gain"),
|
||||
("rate", "rate"), ("band", "band"),
|
||||
("frequency", "frequency"), ("simulate", "simulate"),
|
||||
("seconds", "seconds"), ("slots", "slots"),
|
||||
("log", "log"), ("decodes_seen", "decodes_seen"),
|
||||
("hold", "hold"), ("lowest", "lowest"),
|
||||
("highest", "highest"), ("most", "most"),
|
||||
("rounds", "rounds"), ("grid", "grid"),
|
||||
("units", "units"), ("calls_only", "calls_only"),
|
||||
("report", "report"), ("csv", "csv"),
|
||||
("adif", "adif")):
|
||||
value = getattr(args, flag, None)
|
||||
if value is not None:
|
||||
setattr(options, key, value)
|
||||
# The band picks the frequency unless the frequency was given outright,
|
||||
# which is the one order that lets both flags mean what they say.
|
||||
if getattr(args, "band", None) and getattr(args, "frequency", None) is None:
|
||||
ft8.use_band(options, options.band)
|
||||
errs = options.validate()
|
||||
if errs:
|
||||
for e in errs:
|
||||
console.print(f"[red]{e}[/red]")
|
||||
return 2
|
||||
ft8.listen(console, options, cfg.output_dir)
|
||||
return 0
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
|
|
|||
919
bandsaunter/ft8.py
Normal file
919
bandsaunter/ft8.py
Normal file
|
|
@ -0,0 +1,919 @@
|
|||
"""Listening to FT8, from either front end.
|
||||
|
||||
Fifteen seconds of everybody at once. Every station on the band transmits
|
||||
in the same quarter-minute slots, on the same dial frequency, fifty hertz
|
||||
wide each, stacked across three kilohertz of audio -- so one receiver
|
||||
parked on one frequency hears the whole band's worth of stations at the
|
||||
same time, and hears most of them well below the noise.
|
||||
|
||||
That is why this is a section rather than something the scanner stops on.
|
||||
A sweep catches whatever was keyed up as it went past; FT8 is forty
|
||||
stations transmitting simultaneously for twelve and a half seconds out of
|
||||
every fifteen, and the interesting thing is all of them.
|
||||
|
||||
The band plan is mostly shortwave, which a plain receiver of this kind
|
||||
cannot reach without help -- so the default here is the two-metre channel
|
||||
at 144.174 MHz, which it can. The shortwave channels are all in the list
|
||||
and work perfectly well through an upconverter or a receiver in direct
|
||||
sampling mode; they are simply not what you get by not choosing.
|
||||
|
||||
Nothing here transmits. This listens, decodes and writes down.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
import time
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import yaml
|
||||
|
||||
from . import ft8code as code
|
||||
from . import ft8wave as wave
|
||||
from .settings import Setting, format_value
|
||||
|
||||
__all__ = ["Ft8Options", "OPTIONS", "OPTION_GROUPS", "defaults", "in_group",
|
||||
"by_key", "format_option", "describe", "summarise",
|
||||
"Station", "Band", "Heard", "listen", "open_device", "open_log",
|
||||
"pump", "finish", "report", "load_options", "save_options",
|
||||
"options_path", "logs_in", "BANDS", "band_named", "band_text",
|
||||
"use_band", "slot_start", "next_slot", "grid_at", "grid_away",
|
||||
"SLOT", "audio_from"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Where FT8 lives
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# name, hertz, what it needs. The dial frequency, in the usual convention:
|
||||
# signals sit from about 200 to 3000 Hz above it.
|
||||
BANDS: tuple[tuple[str, float, str], ...] = (
|
||||
("160m", 1_840_000.0, "shortwave"),
|
||||
("80m", 3_573_000.0, "shortwave"),
|
||||
("60m", 5_357_000.0, "shortwave"),
|
||||
("40m", 7_074_000.0, "shortwave"),
|
||||
("30m", 10_136_000.0, "shortwave"),
|
||||
("20m", 14_074_000.0, "shortwave"),
|
||||
("17m", 18_100_000.0, "shortwave"),
|
||||
("15m", 21_074_000.0, "shortwave"),
|
||||
("12m", 24_915_000.0, "shortwave"),
|
||||
("10m", 28_074_000.0, "shortwave"),
|
||||
("6m", 50_313_000.0, "direct"),
|
||||
("2m", 144_174_000.0, "direct"),
|
||||
("70cm", 432_174_000.0, "direct"),
|
||||
)
|
||||
|
||||
SLOT = code.SLOT_S # fifteen seconds, and everybody keeps to it
|
||||
|
||||
|
||||
def band_named(name: str) -> float:
|
||||
"""The dial frequency for a band, or the two-metre one."""
|
||||
wanted = (name or "").strip().lower()
|
||||
for key, hz, _needs in BANDS:
|
||||
if wanted == key:
|
||||
return hz
|
||||
return 144_174_000.0
|
||||
|
||||
|
||||
def band_needs(name: str) -> str:
|
||||
wanted = (name or "").strip().lower()
|
||||
for key, _hz, needs in BANDS:
|
||||
if wanted == key:
|
||||
return needs
|
||||
return "direct"
|
||||
|
||||
|
||||
def use_band(options: "Ft8Options", name: str) -> None:
|
||||
"""Point the receiver at a band, frequency and all."""
|
||||
options.band = (name or "").strip().lower()
|
||||
options.frequency = band_named(options.band)
|
||||
|
||||
|
||||
def band_text(options: "Ft8Options") -> str:
|
||||
where = f"{options.frequency / 1e6:g} MHz"
|
||||
needs = band_needs(options.band)
|
||||
if needs == "shortwave":
|
||||
return f"{options.band} — {where}, needs an upconverter"
|
||||
return f"{options.band} — {where}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Slots
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def slot_start(when: float | None = None) -> float:
|
||||
"""The top of the fifteen-second slot that ``when`` falls in.
|
||||
|
||||
Against UTC, not against the local clock and not against when this
|
||||
program happened to start: the whole protocol is built on every station
|
||||
on earth agreeing about which quarter-minute it is. A receiver whose
|
||||
clock is a second out still decodes -- the search covers a few seconds
|
||||
either way -- but one that is a slot out hears every transmission
|
||||
split across two captures and decodes none of them.
|
||||
"""
|
||||
when = time.time() if when is None else when
|
||||
return math.floor(when / SLOT) * SLOT
|
||||
|
||||
|
||||
def next_slot(when: float | None = None) -> float:
|
||||
return slot_start(when) + SLOT
|
||||
|
||||
|
||||
def slot_name(at: float) -> str:
|
||||
"""The slot, as the six digits every FT8 program labels it with."""
|
||||
return datetime.fromtimestamp(at, timezone.utc).strftime("%H%M%S")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Grid squares
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def grid_at(grid: str):
|
||||
"""The middle of a Maidenhead square, as latitude and longitude.
|
||||
|
||||
Four characters is about seventy miles by a hundred, which is all the
|
||||
position FT8 carries and quite enough to say which continent somebody
|
||||
is on. Six characters is finer and turns up in free text rather than
|
||||
in the position field.
|
||||
"""
|
||||
grid = (grid or "").strip().upper()
|
||||
if len(grid) < 4 or not grid[:2].isalpha() or not grid[2:4].isdigit():
|
||||
return None
|
||||
if not ("A" <= grid[0] <= "R" and "A" <= grid[1] <= "R"):
|
||||
return None
|
||||
lon = (ord(grid[0]) - 65) * 20.0 + int(grid[2]) * 2.0
|
||||
lat = (ord(grid[1]) - 65) * 10.0 + int(grid[3]) * 1.0
|
||||
if len(grid) >= 6 and grid[4].isalpha() and grid[5].isalpha():
|
||||
lon += (ord(grid[4]) - 65) * (2.0 / 24.0) + (1.0 / 24.0)
|
||||
lat += (ord(grid[5]) - 65) * (1.0 / 24.0) + (0.5 / 24.0)
|
||||
else:
|
||||
lon += 1.0
|
||||
lat += 0.5
|
||||
return lat - 90.0, lon - 180.0
|
||||
|
||||
|
||||
def grid_away(here: str, there: str):
|
||||
"""How far apart two grid squares are, in kilometres, and on what bearing.
|
||||
|
||||
Great-circle, from the middles of the squares -- so it is good to the
|
||||
size of a square and no better, which for FT8 is the honest answer and
|
||||
is why nothing here prints a distance to the nearest kilometre.
|
||||
"""
|
||||
a, b = grid_at(here), grid_at(there)
|
||||
if a is None or b is None:
|
||||
return None
|
||||
lat1, lon1 = math.radians(a[0]), math.radians(a[1])
|
||||
lat2, lon2 = math.radians(b[0]), math.radians(b[1])
|
||||
d = 2 * math.asin(math.sqrt(
|
||||
math.sin((lat2 - lat1) / 2) ** 2
|
||||
+ math.cos(lat1) * math.cos(lat2) * math.sin((lon2 - lon1) / 2) ** 2))
|
||||
y = math.sin(lon2 - lon1) * math.cos(lat2)
|
||||
x = (math.cos(lat1) * math.sin(lat2)
|
||||
- math.sin(lat1) * math.cos(lat2) * math.cos(lon2 - lon1))
|
||||
return 6371.0088 * d, (math.degrees(math.atan2(y, x)) + 360.0) % 360.0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The options
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class Ft8Options:
|
||||
"""Everything the FT8 mode can be told, in one place."""
|
||||
|
||||
# -- receiver -------------------------------------------------------
|
||||
device: int = 0
|
||||
gain: str = "auto"
|
||||
rate: float = 240_000.0
|
||||
band: str = "2m"
|
||||
frequency: float = 144_174_000.0
|
||||
simulate: bool = False
|
||||
|
||||
# -- listening ------------------------------------------------------
|
||||
seconds: float = 0.0
|
||||
slots: int = 0 # or stop after this many slots
|
||||
log: bool = True
|
||||
decodes_seen: bool = False # a line per decode rather than a table
|
||||
hold: float = 3600.0
|
||||
|
||||
# -- decoding -------------------------------------------------------
|
||||
lowest: float = 200.0
|
||||
highest: float = 3000.0
|
||||
most: int = 300 # candidates examined per slot
|
||||
rounds: int = 30 # how hard to try to repair one
|
||||
|
||||
# -- showing --------------------------------------------------------
|
||||
grid: str = "" # where the aerial is, as a grid square
|
||||
units: str = "metric"
|
||||
calls_only: bool = False # leave out free text and telemetry
|
||||
|
||||
# -- afterwards -----------------------------------------------------
|
||||
report: bool = True
|
||||
csv: bool = False
|
||||
adif: bool = False
|
||||
|
||||
@property
|
||||
def imperial(self) -> bool:
|
||||
return str(self.units).lower().startswith("imp")
|
||||
|
||||
def validate(self) -> list[str]:
|
||||
out = []
|
||||
if self.rate < 96_000:
|
||||
out.append("the sample rate must be at least 96 kS/s to hold "
|
||||
"three kilohertz of audio with room to filter it")
|
||||
if self.seconds < 0 or self.slots < 0:
|
||||
out.append("times cannot be negative")
|
||||
if self.hold <= 0:
|
||||
out.append("a station must stay on the display for some time")
|
||||
if not 1e5 < self.frequency < 2e9:
|
||||
out.append("that is not a frequency a receiver can be tuned to")
|
||||
if self.highest <= self.lowest:
|
||||
out.append("the top of the search must be above the bottom")
|
||||
if self.highest - self.lowest < 100.0:
|
||||
out.append("a search narrower than 100 Hz would miss almost "
|
||||
"everything: one signal is 50 Hz wide and they are "
|
||||
"spread across three kilohertz")
|
||||
if self.most < 1:
|
||||
out.append("at least one candidate has to be looked at")
|
||||
if self.rounds < 1:
|
||||
out.append("the error correction needs at least one pass")
|
||||
if self.grid and grid_at(self.grid) is None:
|
||||
out.append(f"{self.grid!r} is not a grid square — four "
|
||||
"characters like IO91 or FN31")
|
||||
return out
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return asdict(self)
|
||||
|
||||
|
||||
def defaults() -> Ft8Options:
|
||||
return Ft8Options()
|
||||
|
||||
|
||||
O = Setting
|
||||
_BANDS = tuple(key for key, _hz, _needs in BANDS)
|
||||
|
||||
OPTIONS: tuple[Setting, ...] = (
|
||||
# -- receiver -------------------------------------------------------
|
||||
O("device", "Receiver", "Receiver", "int",
|
||||
"which receiver to use, when more than one is plugged in",
|
||||
"The index shown by `bandsaunter devices`. Zero unless you have "
|
||||
"several dongles.",
|
||||
minimum=0, flags=("--device",), example="0"),
|
||||
O("gain", "Gain", "Receiver", "gain",
|
||||
"tuner gain in dB, or automatic",
|
||||
"FT8 is forty signals at once spanning fifty decibels of strength, "
|
||||
"and what matters is that none of them clips rather than that any of "
|
||||
"them is loud. Automatic copes. A fixed gain makes the signal "
|
||||
"reports comparable between one evening and the next, which is the "
|
||||
"whole point of them if you are judging an aerial or a band opening.",
|
||||
flags=("--gain",), example="auto"),
|
||||
O("rate", "Sample rate", "Receiver", "float",
|
||||
"how fast to sample; 96 kS/s is the least that holds the channel",
|
||||
"Only three kilohertz of it is used, so this is about having room to "
|
||||
"filter cleanly rather than about bandwidth. The default costs "
|
||||
"little and decimates to the twelve kilohertz the decoding wants by "
|
||||
"a whole number.",
|
||||
unit="Hz", minimum=96_000.0, flags=("--rate",), example="240000"),
|
||||
O("band", "Band", "Receiver", "choice",
|
||||
"which FT8 channel to listen on",
|
||||
"Every band has one agreed dial frequency and everybody uses it. "
|
||||
"Two metres is the default because a plain receiver of this kind can "
|
||||
"reach it; the shortwave bands, which is where almost all the "
|
||||
"activity is, need an upconverter or a receiver in direct sampling "
|
||||
"mode. Choosing a band sets the frequency with it.",
|
||||
choices=_BANDS, flags=("--band",), example="20m"),
|
||||
O("frequency", "Dial frequency", "Receiver", "float",
|
||||
"the dial frequency, if you want one the band list does not have",
|
||||
"Signals sit from about two hundred to three thousand hertz above "
|
||||
"this, which is the upper-sideband convention the whole mode is "
|
||||
"built on. Setting a band sets this; setting this directly leaves "
|
||||
"the band label alone, so use it for an upconverter's offset or a "
|
||||
"channel that is not in the list.",
|
||||
unit="Hz", minimum=1e5, maximum=2e9, flags=("--frequency", "--freq"),
|
||||
example="14074000"),
|
||||
O("simulate", "Simulate", "Receiver", "bool",
|
||||
"invent a band instead of using a receiver",
|
||||
"A made-up band of stations, on a real clock, for trying the display "
|
||||
"and the report without an aerial. What it cannot tell you is "
|
||||
"whether your receiver hears anything.",
|
||||
flags=("--simulate",), off_flags=("--no-simulate",)),
|
||||
|
||||
# -- listening ------------------------------------------------------
|
||||
O("seconds", "Listen for", "Listening", "float",
|
||||
"how long to listen; zero means until stopped",
|
||||
"Rounded up to a whole slot, because half a transmission decodes to "
|
||||
"nothing at all.",
|
||||
unit="s", minimum=0.0, flags=("--seconds",), example="600"),
|
||||
O("slots", "Or this many slots", "Listening", "int",
|
||||
"stop after this many fifteen-second slots; zero means no limit",
|
||||
"The same thing as the time limit, counted the way the band counts "
|
||||
"it. Whichever of the two is reached first stops the run.",
|
||||
minimum=0, flags=("--slots",), example="40"),
|
||||
O("log", "Write a log", "Listening", "bool",
|
||||
"keep every decode in a file",
|
||||
"One line per decode, with the slot, the frequency, how late it was "
|
||||
"and how strongly it came in. The report afterwards is built from "
|
||||
"this, and so is anything you want to do with it later.",
|
||||
flags=("--log",), off_flags=("--no-log",)),
|
||||
O("decodes_seen", "Print each decode", "Listening", "bool",
|
||||
"a line per decode instead of a table that refreshes",
|
||||
"What to use when the output is going into a pipe or a file. The "
|
||||
"table is easier to watch; the lines are easier to grep.",
|
||||
flags=("--decodes-seen",), off_flags=("--no-decodes-seen",)),
|
||||
O("hold", "Keep on display for", "Listening", "float",
|
||||
"how long a station stays on the table after its last decode",
|
||||
"A station transmits every other slot at most, and many call once "
|
||||
"and go away. Long enough that the table is a picture of the "
|
||||
"evening rather than of the last thirty seconds.",
|
||||
unit="s", minimum=1.0, flags=("--hold",), example="3600"),
|
||||
|
||||
# -- decoding -------------------------------------------------------
|
||||
O("lowest", "Search from", "Decoding", "float",
|
||||
"the lowest audio frequency to look for signals at",
|
||||
"Below a couple of hundred hertz there is nothing but the "
|
||||
"receiver's own rumble, and searching it costs time for nothing.",
|
||||
unit="Hz", minimum=0.0, flags=("--lowest",), example="200"),
|
||||
O("highest", "Search to", "Decoding", "float",
|
||||
"the highest audio frequency to look for signals at",
|
||||
"Three kilohertz is where a normal sideband receiver stops. Going "
|
||||
"higher finds nothing unless your receiver is wider than that, and "
|
||||
"costs time in proportion.",
|
||||
unit="Hz", minimum=100.0, flags=("--highest",), example="3000"),
|
||||
O("most", "Candidates per slot", "Decoding", "int",
|
||||
"how many possible transmissions to try to decode each slot",
|
||||
"The sync search ranks every place a transmission could be and this "
|
||||
"is how far down the list to go. A busy shortwave band puts forty "
|
||||
"real signals in a slot and several hundred plausible-looking "
|
||||
"places; more than a few hundred has been measured to add nothing "
|
||||
"but time.",
|
||||
minimum=1, flags=("--most",), example="300"),
|
||||
O("rounds", "Repair passes", "Decoding", "int",
|
||||
"how many passes of error correction to make before giving up",
|
||||
"The code either converges in twenty or so passes or it does not "
|
||||
"converge at all -- raising this has been measured to find nothing "
|
||||
"further, and it is here so that can be checked rather than taken "
|
||||
"on trust.",
|
||||
minimum=1, flags=("--rounds",), example="30"),
|
||||
|
||||
# -- showing --------------------------------------------------------
|
||||
O("grid", "Aerial at", "Showing", "text",
|
||||
"your own grid square, so distances can be worked out",
|
||||
"Four characters, like IO91 or FN31. Every FT8 station that calls "
|
||||
"CQ says where it is this way, so with your own square filled in "
|
||||
"the report can say how far each of them is and on what bearing — "
|
||||
"which on shortwave is the whole interest of the thing. Left blank "
|
||||
"the decodes are still recorded; there is simply nothing to measure "
|
||||
"them from.",
|
||||
flags=("--grid",), example="IO91", metavar="SQUARE"),
|
||||
O("units", "Show readings in", "Showing", "choice",
|
||||
"metric or imperial, for the display and the export",
|
||||
"Distances only. Signal reports are in decibels either way, that "
|
||||
"being what the mode measures in and what gets sent back on the "
|
||||
"air.",
|
||||
choices=("metric", "imperial"), flags=("--units",), example="metric"),
|
||||
O("calls_only", "Callsigns only", "Showing", "bool",
|
||||
"leave out free text and telemetry",
|
||||
"Most of what passes on FT8 is two callsigns and a report. The rest "
|
||||
"is free text -- thirteen characters, no callsign structure, so "
|
||||
"nothing can be said about who sent it -- and telemetry, which is "
|
||||
"eighteen hex digits. Turn this on to see only the exchanges.",
|
||||
flags=("--calls-only",), off_flags=("--no-calls-only",)),
|
||||
|
||||
# -- afterwards -----------------------------------------------------
|
||||
O("report", "Report at the end", "Afterwards", "bool",
|
||||
"print the tables when the listening stops",
|
||||
"Who was heard, how far off, and what was exchanged.",
|
||||
flags=("--report",), off_flags=("--no-report",)),
|
||||
O("csv", "Also write a CSV", "Afterwards", "bool",
|
||||
"a spreadsheet of every decode beside the log",
|
||||
"The same rows as the log, in the form a spreadsheet opens without "
|
||||
"being told anything.",
|
||||
flags=("--csv",), off_flags=("--no-csv",)),
|
||||
O("adif", "Also write an ADIF", "Afterwards", "bool",
|
||||
"the log again, in the form logging programs read",
|
||||
"ADIF is what every amateur logging program imports. What is "
|
||||
"written is what was heard rather than a contact made -- nothing "
|
||||
"here transmits, so nothing here is a QSO — so the records are "
|
||||
"marked as received-only and are for feeding a propagation map or a "
|
||||
"spotting tool rather than for claiming anything.",
|
||||
flags=("--adif",), off_flags=("--no-adif",)),
|
||||
)
|
||||
|
||||
OPTION_GROUPS = ("Receiver", "Listening", "Decoding", "Showing", "Afterwards")
|
||||
|
||||
|
||||
def in_group(group: str) -> tuple[Setting, ...]:
|
||||
return tuple(o for o in OPTIONS if o.group == group)
|
||||
|
||||
|
||||
def by_key(key: str):
|
||||
for option in OPTIONS:
|
||||
if option.key == key:
|
||||
return option
|
||||
return None
|
||||
|
||||
|
||||
# Options where nothing set means something, rather than nothing.
|
||||
_ZERO_MEANS = {
|
||||
"seconds": "until stopped",
|
||||
"slots": "no limit",
|
||||
"grid": "not set, so no distances",
|
||||
}
|
||||
|
||||
|
||||
def format_option(option: Setting, value) -> str:
|
||||
"""Render an option the way the menu and the manual should show it."""
|
||||
if not value and option.key in _ZERO_MEANS:
|
||||
return _ZERO_MEANS[option.key]
|
||||
return format_value(option, value)
|
||||
|
||||
|
||||
def describe(options: Ft8Options) -> str:
|
||||
"""The one-line summary the menu shows beside Listen."""
|
||||
bits = [band_text(options)]
|
||||
if options.seconds:
|
||||
bits.append(f"{options.seconds:g} s")
|
||||
elif options.slots:
|
||||
bits.append(f"{options.slots} slots")
|
||||
if options.grid:
|
||||
bits.append(f"from {options.grid.upper()}")
|
||||
if options.simulate:
|
||||
bits.append("simulated")
|
||||
return ", ".join(bits)
|
||||
|
||||
|
||||
def summarise(options: Ft8Options) -> str:
|
||||
return describe(options)
|
||||
|
||||
|
||||
def options_path(directory=None) -> Path:
|
||||
from .config import DEFAULT_CONFIG_DIR
|
||||
|
||||
return Path(directory or DEFAULT_CONFIG_DIR) / "ft8.yaml"
|
||||
|
||||
|
||||
def load_options(directory=None) -> Ft8Options:
|
||||
"""The saved options, or the defaults. A broken file is not an error."""
|
||||
options = Ft8Options()
|
||||
try:
|
||||
body = yaml.safe_load(options_path(directory).read_text()) or {}
|
||||
except (OSError, ValueError, yaml.YAMLError):
|
||||
return options
|
||||
if not isinstance(body, dict):
|
||||
return options
|
||||
known = set(options.__dict__)
|
||||
for key, value in body.items():
|
||||
if key in known and value is not None:
|
||||
try:
|
||||
setattr(options, key, type(getattr(options, key))(value))
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
return options
|
||||
|
||||
|
||||
def save_options(options: Ft8Options, directory=None) -> Path:
|
||||
path = options_path(directory)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(path, "w") as fh:
|
||||
yaml.safe_dump(options.to_dict(), fh, sort_keys=False,
|
||||
default_flow_style=False)
|
||||
return path
|
||||
|
||||
|
||||
def logs_in(directory) -> list[Path]:
|
||||
try:
|
||||
return sorted(Path(directory).glob("ft8-*.txt"))
|
||||
except OSError:
|
||||
return []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Who was heard
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class Station:
|
||||
"""One callsign, and everything heard from it."""
|
||||
|
||||
call: str = ""
|
||||
grid: str = ""
|
||||
first: float = 0.0
|
||||
last: float = 0.0
|
||||
decodes: int = 0
|
||||
best_snr: float = -99.0
|
||||
worst_snr: float = 99.0
|
||||
hertz: float = 0.0
|
||||
calling: int = 0 # how many times it called CQ
|
||||
worked: set = field(default_factory=set) # who it was talking to
|
||||
said: list = field(default_factory=list) # the lines, newest last
|
||||
|
||||
def away(self, here: str):
|
||||
"""How far off and on what bearing, if both squares are known."""
|
||||
if not here or not self.grid:
|
||||
return None
|
||||
return grid_away(here, self.grid)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Band:
|
||||
"""Every station heard, and every exchange between them.
|
||||
|
||||
Kept apart the way the APRS section keeps messages apart from stations,
|
||||
and for the same reason: a decode is about a moment and a station is
|
||||
about an evening, and a table that tried to be both would be neither.
|
||||
"""
|
||||
|
||||
stations: dict = field(default_factory=dict)
|
||||
decodes: list = field(default_factory=list)
|
||||
book: code.CallBook = field(default_factory=code.CallBook)
|
||||
slots: int = 0
|
||||
|
||||
def add(self, found) -> None:
|
||||
self.decodes.append(found)
|
||||
# The caller is the second callsign in a standard exchange -- the
|
||||
# first is who is being answered. A CQ has no first, so the one
|
||||
# station named is the one transmitting.
|
||||
sender = found.calls[1] if len(found.calls) > 1 else (
|
||||
found.calls[0] if found.calls else "")
|
||||
if not sender or sender == "<...>":
|
||||
return
|
||||
station = self.stations.get(sender)
|
||||
if station is None:
|
||||
station = Station(call=sender, first=found.at)
|
||||
self.stations[sender] = station
|
||||
station.last = found.at
|
||||
station.decodes += 1
|
||||
station.hertz = found.hertz
|
||||
station.best_snr = max(station.best_snr, found.snr_db)
|
||||
station.worst_snr = min(station.worst_snr, found.snr_db)
|
||||
if found.grid:
|
||||
station.grid = found.grid
|
||||
if found.calling:
|
||||
station.calling += 1
|
||||
elif len(found.calls) > 1 and found.calls[0] != "<...>":
|
||||
station.worked.add(found.calls[0])
|
||||
station.said.append(found.text)
|
||||
if len(station.said) > 32:
|
||||
del station.said[0]
|
||||
|
||||
def all(self) -> list:
|
||||
return list(self.stations.values())
|
||||
|
||||
|
||||
@dataclass
|
||||
class Heard:
|
||||
"""What the front ends share: the running totals of one run."""
|
||||
|
||||
band: Band = field(default_factory=Band)
|
||||
decodes: int = 0
|
||||
slots: int = 0
|
||||
started: float = 0.0
|
||||
latest: list = field(default_factory=list)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The receiver
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
AUDIO_RATE = 12_000 # what the decoding wants, and a whole division
|
||||
|
||||
|
||||
def open_device(console, options: Ft8Options):
|
||||
"""The receiver, or a made-up band, or None if neither can be had."""
|
||||
from rich.panel import Panel
|
||||
from rich.text import Text
|
||||
|
||||
from .device import RtlSdrDevice, RtlSdrError
|
||||
|
||||
if options.simulate:
|
||||
console.print("[yellow]simulated: these stations are not there."
|
||||
"[/yellow]")
|
||||
from .ft8sim import SimulatedBand
|
||||
|
||||
return SimulatedBand(rate=options.rate, realtime=True)
|
||||
try:
|
||||
device = RtlSdrDevice(index=options.device,
|
||||
sample_rate=int(options.rate),
|
||||
gain=options.gain,
|
||||
agc=options.gain == "auto",
|
||||
# Which is what makes the shortwave bands
|
||||
# reachable at all on a dongle that supports
|
||||
# it: below 24 MHz the tuner is bypassed and
|
||||
# the sampler is fed straight off the aerial.
|
||||
direct_sampling="auto")
|
||||
device.open()
|
||||
except RtlSdrError as exc:
|
||||
console.print(Panel(Text(str(exc)),
|
||||
title="[red]cannot open the receiver",
|
||||
border_style="red"))
|
||||
return None
|
||||
return device
|
||||
|
||||
|
||||
def audio_from(samples, options: Ft8Options, demod=None):
|
||||
"""Upper-sideband audio at twelve kilohertz, from the receiver's samples.
|
||||
|
||||
Upper sideband because that is the convention the whole mode is built
|
||||
on: the dial frequency is the bottom edge and every signal sits above
|
||||
it, which is why two stations three hundred hertz apart are two
|
||||
different signals rather than the same one heard twice.
|
||||
"""
|
||||
from .demod import make_demodulator
|
||||
|
||||
if demod is None:
|
||||
demod = make_demodulator("usb", sample_rate=int(options.rate),
|
||||
bandwidth=3200.0, audio_rate=AUDIO_RATE)
|
||||
audio, _iq = demod.step(samples)
|
||||
return audio, demod
|
||||
|
||||
|
||||
def pump(device, options: Ft8Options, heard: Heard, log, started: float,
|
||||
on_slot=None, on_decode=None, stopping=None, clock=time.time) -> int:
|
||||
"""Read the receiver, a slot at a time, until it stops or is told to.
|
||||
|
||||
The one loop both front ends are driven from, so that what is written
|
||||
to the log cannot depend on which one you happened to be looking at.
|
||||
|
||||
Audio is collected continuously and cut on the slot boundaries rather
|
||||
than being captured slot by slot. A receiver that started reading when
|
||||
the slot began would miss however long it took to start reading, and
|
||||
everything on the band begins in the first half-second.
|
||||
"""
|
||||
demod = None
|
||||
audio = np.zeros(0, dtype=np.float32)
|
||||
audio_at = 0.0 # wall time of audio[0]
|
||||
want = next_slot(clock()) # the first whole slot we can get
|
||||
total = 0
|
||||
device.tune(options.frequency)
|
||||
block = int(options.rate) # a second of samples at a time
|
||||
|
||||
while stopping is None or not stopping():
|
||||
at = clock()
|
||||
samples = device.read_samples(block)
|
||||
if samples is None or samples.size == 0:
|
||||
break
|
||||
fresh, demod = audio_from(samples, options, demod)
|
||||
if fresh.size:
|
||||
if audio.size == 0:
|
||||
# The clock was read before asking for the samples, and the
|
||||
# read returns once they have arrived -- so they cover the
|
||||
# block that *starts* here. Dating them a block earlier
|
||||
# instead slides every slice a second late, which cuts the
|
||||
# first half-second off every transmission: three symbols,
|
||||
# part of the opening Costas array, and most of the
|
||||
# decodes with it.
|
||||
audio_at = at
|
||||
audio = np.concatenate([audio, fresh])
|
||||
|
||||
# Every whole slot now sitting in the buffer.
|
||||
while audio.size and audio_at <= want and \
|
||||
audio_at + audio.size / AUDIO_RATE >= want + SLOT:
|
||||
first = int(round((want - audio_at) * AUDIO_RATE))
|
||||
piece = audio[first:first + int(round(SLOT * AUDIO_RATE))]
|
||||
found = wave.listen_to(piece, AUDIO_RATE, book=heard.band.book,
|
||||
most=options.most, rounds=options.rounds,
|
||||
at=want)
|
||||
found = [d for d in found
|
||||
if options.lowest <= d.hertz <= options.highest]
|
||||
if options.calls_only:
|
||||
found = [d for d in found if d.kind in ("standard",
|
||||
"non-standard")]
|
||||
heard.slots += 1
|
||||
heard.band.slots += 1
|
||||
heard.latest = found
|
||||
for d in found:
|
||||
heard.band.add(d)
|
||||
heard.decodes += 1
|
||||
total += 1
|
||||
if log is not None:
|
||||
log.append(d)
|
||||
if on_decode is not None:
|
||||
on_decode(d)
|
||||
if on_slot is not None:
|
||||
on_slot(want, found)
|
||||
want += SLOT
|
||||
# Keep a little before the next slot: a transmission can start
|
||||
# before the slot it belongs to.
|
||||
keep = int(round(max(0.0, want - 3.0 - audio_at) * AUDIO_RATE))
|
||||
if keep > 0:
|
||||
audio = audio[keep:]
|
||||
audio_at += keep / AUDIO_RATE
|
||||
|
||||
if options.seconds and clock() - started >= options.seconds:
|
||||
break
|
||||
if options.slots and heard.slots >= options.slots:
|
||||
break
|
||||
return total
|
||||
|
||||
|
||||
def open_log(console, options: Ft8Options, output_dir, started: float):
|
||||
from . import ft8log
|
||||
|
||||
if not options.log:
|
||||
return None
|
||||
log = ft8log.open_log(output_dir, started, options.frequency,
|
||||
options.band)
|
||||
if log is None:
|
||||
console.print("[yellow]could not open a log; carrying on without "
|
||||
"one[/yellow]")
|
||||
return log
|
||||
|
||||
|
||||
def listen(console, options: Ft8Options, output_dir: str,
|
||||
heard: Heard | None = None, stopping=None) -> int:
|
||||
"""Listen on one channel and write down everything decoded."""
|
||||
from .ui import Ft8Display
|
||||
|
||||
started = time.time()
|
||||
heard = heard if heard is not None else Heard()
|
||||
heard.started = started
|
||||
# The receiver first. Announcing that it is listening and then failing
|
||||
# to open a dongle reads as though the listening went wrong, when what
|
||||
# went wrong happened before any of it started.
|
||||
device = open_device(console, options)
|
||||
if device is None:
|
||||
return 0
|
||||
log = open_log(console, options, output_dir, started)
|
||||
_say_where(console, options)
|
||||
console.print(f"[grey62]listening on {band_text(options)} at "
|
||||
f"{options.rate / 1e6:g} MS/s — control-C to stop"
|
||||
"[/grey62]")
|
||||
if not options.decodes_seen:
|
||||
console.print("[grey62]the first decodes appear when the slot after "
|
||||
"next ends, which is up to thirty seconds away"
|
||||
"[/grey62]")
|
||||
display = None
|
||||
live = None
|
||||
if not options.decodes_seen and getattr(console, "is_terminal", False):
|
||||
from rich.live import Live
|
||||
|
||||
display = Ft8Display(console, hold=options.hold,
|
||||
imperial=options.imperial, grid=options.grid,
|
||||
band=band_text(options))
|
||||
display.started = started
|
||||
live = Live(display.render(), console=console, refresh_per_second=2,
|
||||
screen=False, transient=False, vertical_overflow="crop")
|
||||
live.start()
|
||||
|
||||
def show_slot(at, found):
|
||||
if display is not None:
|
||||
display.update(heard, at)
|
||||
live.update(display.render())
|
||||
|
||||
def show_decode(d):
|
||||
if options.decodes_seen:
|
||||
console.print(f"{slot_name(d.at)} {d.snr_db:>4.0f} "
|
||||
f"{d.offset:>5.1f} {d.hertz:>7.1f} ~ {d.text}")
|
||||
|
||||
try:
|
||||
pump(device, options, heard, log, started, on_slot=show_slot,
|
||||
on_decode=show_decode, stopping=stopping)
|
||||
except KeyboardInterrupt:
|
||||
console.print("\n[grey62]stopped[/grey62]")
|
||||
finally:
|
||||
if live is not None:
|
||||
live.stop()
|
||||
try:
|
||||
device.close()
|
||||
except Exception:
|
||||
pass
|
||||
return finish(console, options, output_dir, heard, log)
|
||||
|
||||
|
||||
def _say_where(console, options: Ft8Options) -> None:
|
||||
"""Whether distances can be worked out, and what it costs if not."""
|
||||
if options.grid:
|
||||
where = grid_at(options.grid)
|
||||
console.print(f"[grey62]aerial at {options.grid.upper()} "
|
||||
f"({where[0]:.2f},{where[1]:.2f}) — distances and "
|
||||
f"bearings will be worked out[/grey62]")
|
||||
return
|
||||
console.print("[grey62]no grid square set, so no distances and no "
|
||||
"bearings — `--grid IO91` gives both[/grey62]")
|
||||
if band_needs(options.band) == "shortwave":
|
||||
console.print("[grey62]this band is shortwave: a plain receiver "
|
||||
"needs an upconverter or direct sampling to hear it"
|
||||
"[/grey62]")
|
||||
|
||||
|
||||
def finish(console, options: Ft8Options, output_dir: str, heard: Heard,
|
||||
log=None) -> int:
|
||||
"""Close the log, write the exports, and print the report."""
|
||||
from . import ft8log
|
||||
|
||||
if log is not None:
|
||||
log.close()
|
||||
console.print(f"[grey62]{log.lines} decodes written to "
|
||||
f"{log.path}[/grey62]")
|
||||
if options.csv:
|
||||
where = ft8log.write_csv(log.path.with_suffix(".csv"),
|
||||
heard.band.decodes)
|
||||
if where is not None:
|
||||
console.print(f"[grey62]and as a spreadsheet: {where}"
|
||||
"[/grey62]")
|
||||
if options.adif:
|
||||
where = ft8log.write_adif(log.path.with_suffix(".adi"),
|
||||
heard.band.decodes, band=options.band,
|
||||
frequency=options.frequency,
|
||||
my_grid=options.grid)
|
||||
if where is not None:
|
||||
console.print(f"[grey62]and as ADIF, marked as heard rather "
|
||||
f"than worked: {where}[/grey62]")
|
||||
if options.report:
|
||||
report(console, heard.band, options)
|
||||
return heard.decodes
|
||||
|
||||
|
||||
def _away_text(station: Station, options: Ft8Options) -> str:
|
||||
away = station.away(options.grid)
|
||||
if away is None:
|
||||
return ""
|
||||
km, bearing = away
|
||||
if options.imperial:
|
||||
return f"{km * 0.621371:.0f} mi {bearing:.0f}°"
|
||||
return f"{km:.0f} km {bearing:.0f}°"
|
||||
|
||||
|
||||
def report(console, band: Band, options: Ft8Options) -> None:
|
||||
"""Three tables, because they answer three different questions."""
|
||||
from rich.table import Table
|
||||
|
||||
stations = sorted(band.all(), key=lambda s: (-s.decodes, s.call))
|
||||
if not stations and not band.decodes:
|
||||
console.print("[yellow]nothing decoded[/yellow]")
|
||||
if band.slots:
|
||||
console.print("[grey62]the band was quiet, the aerial is not "
|
||||
"hearing it, or the clock is out: FT8 needs the "
|
||||
"clock right to a second or two[/grey62]")
|
||||
return
|
||||
|
||||
console.print()
|
||||
console.print(f"[bold]Stations heard[/bold] — {len(stations)} in "
|
||||
f"{band.slots} slots")
|
||||
t = Table(box=None, header_style="bold", pad_edge=False)
|
||||
t.add_column("station")
|
||||
t.add_column("grid")
|
||||
if options.grid:
|
||||
t.add_column("away")
|
||||
t.add_column("decodes", justify="right")
|
||||
t.add_column("best", justify="right")
|
||||
t.add_column("worst", justify="right")
|
||||
t.add_column("audio", justify="right")
|
||||
t.add_column("CQ", justify="right")
|
||||
for s in stations[:60]:
|
||||
row = [s.call, s.grid or "—"]
|
||||
if options.grid:
|
||||
row.append(_away_text(s, options) or "—")
|
||||
row += [str(s.decodes), f"{s.best_snr:.0f}", f"{s.worst_snr:.0f}",
|
||||
f"{s.hertz:.0f} Hz", str(s.calling) if s.calling else ""]
|
||||
t.add_row(*row)
|
||||
console.print(t)
|
||||
if len(stations) > 60:
|
||||
console.print(f"[grey62]…and {len(stations) - 60} more[/grey62]")
|
||||
|
||||
calling = [s for s in stations if s.calling]
|
||||
if calling:
|
||||
console.print()
|
||||
console.print(f"[bold]Calling CQ[/bold] — {len(calling)} stations "
|
||||
"looking for a contact")
|
||||
t = Table(box=None, header_style="bold", pad_edge=False)
|
||||
t.add_column("station")
|
||||
t.add_column("grid")
|
||||
if options.grid:
|
||||
t.add_column("away")
|
||||
t.add_column("times", justify="right")
|
||||
t.add_column("best", justify="right")
|
||||
for s in sorted(calling, key=lambda s: -s.calling)[:30]:
|
||||
row = [s.call, s.grid or "—"]
|
||||
if options.grid:
|
||||
row.append(_away_text(s, options) or "—")
|
||||
row += [str(s.calling), f"{s.best_snr:.0f}"]
|
||||
t.add_row(*row)
|
||||
console.print(t)
|
||||
|
||||
worked = [s for s in stations if s.worked]
|
||||
if worked:
|
||||
console.print()
|
||||
console.print("[bold]Who was working whom[/bold] — the exchanges "
|
||||
"that passed while this was listening")
|
||||
t = Table(box=None, header_style="bold", pad_edge=False)
|
||||
t.add_column("station")
|
||||
t.add_column("answered")
|
||||
for s in sorted(worked, key=lambda s: -len(s.worked))[:30]:
|
||||
t.add_row(s.call, ", ".join(sorted(s.worked)[:8]))
|
||||
console.print(t)
|
||||
|
||||
if options.grid:
|
||||
far = [(s, s.away(options.grid)) for s in stations if s.grid]
|
||||
far = [(s, a) for s, a in far if a is not None]
|
||||
if far:
|
||||
s, (km, bearing) = max(far, key=lambda x: x[1][0])
|
||||
unit = (f"{km * 0.621371:.0f} miles" if options.imperial
|
||||
else f"{km:.0f} km")
|
||||
console.print()
|
||||
console.print(f"[grey62]furthest heard: {s.call} in {s.grid}, "
|
||||
f"{unit} on {bearing:.0f}°[/grey62]")
|
||||
712
bandsaunter/ft8code.py
Normal file
712
bandsaunter/ft8code.py
Normal file
|
|
@ -0,0 +1,712 @@
|
|||
"""What an FT8 transmission carries, and how it is wrapped.
|
||||
|
||||
No radio in here at all. This is the part between a string of one hundred
|
||||
and seventy-four bits and a line somebody can read: the checksum, the
|
||||
error-correcting code that makes FT8 work at signal levels where the
|
||||
operator hears nothing, the Gray mapping onto eight tones, and the
|
||||
seventy-seven bits that hold two callsigns and a grid square.
|
||||
|
||||
The shape of it, outward from the message:
|
||||
|
||||
77 bits what was said: two callsigns and a report, or free text
|
||||
+ 14 bits a CRC, so a wrong answer is caught rather than printed
|
||||
= 91 bits
|
||||
+ 83 bits LDPC parity, so a wrong answer is usually *repaired*
|
||||
= 174 bits
|
||||
/ 3 three bits to a tone, in Gray order
|
||||
= 58 tones
|
||||
+ 21 tones three Costas arrays, at the start, the middle and the end
|
||||
= 79 tones at 6.25 Hz apart and 6.25 to the second: 12.64 seconds
|
||||
|
||||
The error correction is the whole trick. Fifty-eight tones of payload
|
||||
carried in one hundred and seventy-four bits is a rate of about one half:
|
||||
half of what is transmitted is redundancy, and that is what buys a protocol
|
||||
that decodes at twenty-odd decibels below the noise in the same bandwidth.
|
||||
A receiver that only took the strongest tone in each symbol and hoped would
|
||||
decode almost nothing -- which is why the tone detector below reports how
|
||||
confident it is, bit by bit, rather than just what it thinks it heard.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
import numpy as np
|
||||
|
||||
from .ft8tables import (BITS, CHECKS, COSTAS, CRC_BITS, CRC_POLYNOMIAL,
|
||||
GENERATOR, GRAY, PARITY, PAYLOAD, TONES)
|
||||
|
||||
__all__ = ["crc14", "with_crc", "crc_holds", "encode", "repair",
|
||||
"parity_holds", "tones_of",
|
||||
"bits_of", "unpack", "pack", "CallBook", "Message", "MESSAGE_BITS",
|
||||
"SYMBOL_HZ", "SYMBOL_S", "SLOT_S", "SENDING_S", "COSTAS", "TONES",
|
||||
"hash_of", "free_text", "is_standard_call"]
|
||||
|
||||
MESSAGE_BITS = 77 # what is actually said, before the checksum
|
||||
|
||||
# The channel, in numbers. Six and a quarter of everything: the tones are
|
||||
# 6.25 Hz apart and there are 6.25 of them a second, which makes each tone
|
||||
# exactly one cycle of separation long and the whole thing orthogonal.
|
||||
SYMBOL_HZ = 6.25
|
||||
SYMBOL_S = 1.0 / SYMBOL_HZ
|
||||
SENDING_S = TONES * SYMBOL_S # 12.64 s of transmission
|
||||
SLOT_S = 15.0 # in a slot of fifteen
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The checksum
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def crc14(bits) -> int:
|
||||
"""CRC-14 over a sequence of bits, most significant first.
|
||||
|
||||
The one thing standing between a repaired codeword and a confident lie.
|
||||
The error correction below will happily converge on *a* valid codeword
|
||||
from noise; whether it is the codeword that was sent is what this
|
||||
answers, and one in sixteen thousand wrong answers gets through, which
|
||||
over an evening is a handful of lines nobody should trust. Every
|
||||
decoder prints them anyway, and so does this one -- but only after the
|
||||
CRC has agreed, which is the difference between a handful and a flood.
|
||||
"""
|
||||
top = 1 << (CRC_BITS - 1)
|
||||
mask = (top << 1) - 1
|
||||
remainder = 0
|
||||
for i, bit in enumerate(bits):
|
||||
if i % 8 == 0:
|
||||
# A byte at a time into the top of the register, exactly as the
|
||||
# specification frames it: the message is a byte sequence and
|
||||
# the odd bits at the end are zeros.
|
||||
byte = 0
|
||||
for k in range(8):
|
||||
byte = (byte << 1) | (bits[i + k] if i + k < len(bits) else 0)
|
||||
remainder ^= byte << (CRC_BITS - 8)
|
||||
remainder = ((remainder << 1) ^ CRC_POLYNOMIAL if remainder & top
|
||||
else remainder << 1)
|
||||
return remainder & mask
|
||||
|
||||
|
||||
def with_crc(payload) -> np.ndarray:
|
||||
"""The seventy-seven bits said, plus the fourteen that check them.
|
||||
|
||||
The checksum covers eighty-two bits rather than seventy-seven: the
|
||||
message zero-extended by five, which is what the specification says and
|
||||
is not the same answer as checksumming seventy-seven.
|
||||
"""
|
||||
payload = np.asarray(payload, dtype=np.uint8)
|
||||
if payload.size != MESSAGE_BITS:
|
||||
raise ValueError(f"a message is {MESSAGE_BITS} bits, not "
|
||||
f"{payload.size}")
|
||||
extended = np.concatenate([payload, np.zeros(5, dtype=np.uint8)])
|
||||
check = crc14(extended.tolist())
|
||||
tail = np.array([(check >> (CRC_BITS - 1 - k)) & 1
|
||||
for k in range(CRC_BITS)], dtype=np.uint8)
|
||||
return np.concatenate([payload, tail])
|
||||
|
||||
|
||||
def crc_holds(bits91) -> bool:
|
||||
"""Whether the checksum on a decoded block agrees with its message."""
|
||||
bits91 = np.asarray(bits91, dtype=np.uint8)
|
||||
if bits91.size != PAYLOAD:
|
||||
return False
|
||||
return bool((with_crc(bits91[:MESSAGE_BITS]) == bits91).all())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The error-correcting code
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _generator() -> np.ndarray:
|
||||
"""The parity half of the generator, unpacked to bits."""
|
||||
rows = np.zeros((PARITY, PAYLOAD), dtype=np.uint8)
|
||||
for m, row in enumerate(GENERATOR):
|
||||
packed = bytes.fromhex(row)
|
||||
for k in range(PAYLOAD):
|
||||
rows[m, k] = (packed[k // 8] >> (7 - k % 8)) & 1
|
||||
return rows
|
||||
|
||||
|
||||
GEN = _generator()
|
||||
|
||||
|
||||
def encode(bits91) -> np.ndarray:
|
||||
"""Ninety-one bits in, one hundred and seventy-four out.
|
||||
|
||||
Systematic: the message comes out unchanged at the front and the parity
|
||||
is appended, which is why the decoder can read the message straight off
|
||||
a repaired codeword without undoing anything.
|
||||
"""
|
||||
bits91 = np.asarray(bits91, dtype=np.uint8)
|
||||
if bits91.size != PAYLOAD:
|
||||
raise ValueError(f"expected {PAYLOAD} bits, got {bits91.size}")
|
||||
parity = (GEN @ bits91) & 1
|
||||
return np.concatenate([bits91, parity.astype(np.uint8)])
|
||||
|
||||
|
||||
def _sparse():
|
||||
"""The parity-check matrix as flat slots, for the message passing.
|
||||
|
||||
Every codeword bit sits in exactly three checks and every check covers
|
||||
six or seven bits, so the whole graph fits in two small index arrays and
|
||||
the decoding never needs a scatter-add. The short checks are padded to
|
||||
seven with a slot pointing at a dummy bit that is held at no opinion.
|
||||
"""
|
||||
width = max(len(c) for c in CHECKS)
|
||||
bit_of_slot = np.full(PARITY * width, BITS, dtype=np.int64) # BITS = dummy
|
||||
live = np.zeros(PARITY * width, dtype=bool)
|
||||
for m, check in enumerate(CHECKS):
|
||||
for j, one_based in enumerate(check):
|
||||
bit_of_slot[m * width + j] = one_based - 1
|
||||
live[m * width + j] = True
|
||||
# And the other way: for each bit, the slots that talk about it.
|
||||
per_bit = [[] for _ in range(BITS)]
|
||||
for slot in np.flatnonzero(live):
|
||||
per_bit[bit_of_slot[slot]].append(slot)
|
||||
degree = max(len(s) for s in per_bit)
|
||||
slots_of_bit = np.zeros((BITS, degree), dtype=np.int64)
|
||||
for n, slots in enumerate(per_bit):
|
||||
slots_of_bit[n] = slots + [slots[-1]] * (degree - len(slots))
|
||||
return width, bit_of_slot, live, slots_of_bit
|
||||
|
||||
|
||||
WIDTH, BIT_OF_SLOT, LIVE, SLOTS_OF_BIT = _sparse()
|
||||
|
||||
|
||||
def parity_holds(bits174) -> bool:
|
||||
"""Whether every one of the eighty-three checks comes out even.
|
||||
|
||||
Here rather than left to callers because the padding convention -- short
|
||||
checks point their spare slots at a dummy bit past the end -- is an
|
||||
implementation detail of the message passing, and anything outside this
|
||||
module that had to know about it would be a bug waiting to be written.
|
||||
"""
|
||||
bits174 = np.asarray(bits174, dtype=np.uint8)
|
||||
if bits174.size != BITS:
|
||||
return False
|
||||
padded = np.concatenate([bits174, np.zeros(1, dtype=np.uint8)])
|
||||
sums = padded[BIT_OF_SLOT].copy()
|
||||
sums[~LIVE] = 0
|
||||
return bool((sums.reshape(PARITY, WIDTH).sum(axis=1) % 2 == 0).all())
|
||||
|
||||
|
||||
def repair(llr, rounds: int = 30, alpha: float = 0.75):
|
||||
"""Belief propagation over one or many candidate codewords.
|
||||
|
||||
``llr`` is how strongly each bit is believed to be a zero: positive for
|
||||
zero, negative for one, and the size of it is the confidence. Shape
|
||||
(174,) for one candidate or (n, 174) for a batch, which is how it is
|
||||
actually used -- a busy slot throws up hundreds of candidates and doing
|
||||
them one at a time is most of the decoding time.
|
||||
|
||||
Normalised min-sum rather than the exact sum-product: it is within a
|
||||
few tenths of a decibel of it, and it is multiplication-free in the
|
||||
inner loop, which at this size matters more than the tenths.
|
||||
|
||||
Returns the repaired bits, or None where the checks never came out
|
||||
even. An answer here still has to pass the CRC before it is believed:
|
||||
this finds *a* codeword, and noise has valid codewords in it too.
|
||||
"""
|
||||
single = np.ndim(llr) == 1
|
||||
belief = np.atleast_2d(np.asarray(llr, dtype=np.float32))
|
||||
n = belief.shape[0]
|
||||
if belief.shape[1] != BITS:
|
||||
raise ValueError(f"a codeword is {BITS} bits, not {belief.shape[1]}")
|
||||
|
||||
messages = np.zeros((n, PARITY * WIDTH), dtype=np.float32)
|
||||
dead = ~LIVE
|
||||
dead_grid = dead.reshape(PARITY, WIDTH)
|
||||
out = np.zeros((n, BITS), dtype=np.uint8)
|
||||
done = np.zeros(n, dtype=bool)
|
||||
|
||||
def totals(msgs):
|
||||
"""What every bit is believed to be, once its three checks are in.
|
||||
|
||||
A gather and a sum rather than a scatter-add: every bit sits in
|
||||
exactly three checks, so the three places to look are known in
|
||||
advance and np.add.at -- which is where an earlier version of this
|
||||
spent most of its time -- is not needed at all.
|
||||
"""
|
||||
return belief + msgs[:, SLOTS_OF_BIT].sum(axis=2)
|
||||
|
||||
for _ in range(rounds):
|
||||
total = totals(messages)
|
||||
# What each check hears about a bit, less what it said itself.
|
||||
wide = np.concatenate([total, np.zeros((n, 1), dtype=np.float32)],
|
||||
axis=1)
|
||||
heard = (wide[:, BIT_OF_SLOT] - messages).reshape(n, PARITY, WIDTH)
|
||||
|
||||
size = np.abs(heard)
|
||||
size[:, dead_grid] = np.inf
|
||||
# The smallest and the next smallest in each check, so "the
|
||||
# smallest of the others" is a lookup rather than a loop.
|
||||
order = np.argsort(size, axis=2)
|
||||
smallest = np.take_along_axis(size, order[:, :, :1], axis=2)
|
||||
next_up = np.take_along_axis(size, order[:, :, 1:2], axis=2)
|
||||
others = np.where(size == smallest, next_up, smallest)
|
||||
|
||||
odd = (heard < 0).sum(axis=2, keepdims=True) % 2 == 1
|
||||
mine = np.where(heard < 0, -1.0, 1.0).astype(np.float32)
|
||||
whole = np.where(odd, -1.0, 1.0).astype(np.float32)
|
||||
messages = (alpha * whole * mine
|
||||
* np.minimum(others, 1e4)).astype(np.float32)
|
||||
messages = messages.reshape(n, -1)
|
||||
messages[:, dead] = 0.0
|
||||
|
||||
hard = (totals(messages) < 0).astype(np.uint8)
|
||||
wide_hard = np.concatenate([hard, np.zeros((n, 1), dtype=np.uint8)],
|
||||
axis=1)
|
||||
sums = wide_hard[:, BIT_OF_SLOT].reshape(n, PARITY, WIDTH).copy()
|
||||
sums[:, dead_grid] = 0
|
||||
even = (sums.sum(axis=2) % 2 == 0).all(axis=1)
|
||||
fresh = even & ~done
|
||||
if fresh.any():
|
||||
out[fresh] = hard[fresh]
|
||||
done |= fresh
|
||||
if done.all():
|
||||
break
|
||||
|
||||
if single:
|
||||
return out[0] if done[0] else None
|
||||
return out, done
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tones
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
UNGRAY = np.zeros(8, dtype=np.uint8)
|
||||
for _value, _tone in enumerate(GRAY):
|
||||
UNGRAY[_tone] = _value
|
||||
|
||||
SYNC_AT = (0, 36, 72) # where the three Costas arrays sit
|
||||
|
||||
|
||||
def tones_of(bits174) -> np.ndarray:
|
||||
"""The seventy-nine tones that carry these bits, sync included."""
|
||||
bits174 = np.asarray(bits174, dtype=np.uint8)
|
||||
if bits174.size != BITS:
|
||||
raise ValueError(f"a codeword is {BITS} bits, not {bits174.size}")
|
||||
trips = bits174.reshape(-1, 3)
|
||||
values = trips[:, 0] * 4 + trips[:, 1] * 2 + trips[:, 2]
|
||||
data = np.array([GRAY[v] for v in values], dtype=np.uint8)
|
||||
out = np.zeros(TONES, dtype=np.uint8)
|
||||
costas = np.array(COSTAS, dtype=np.uint8)
|
||||
taken = 0
|
||||
for i in range(TONES):
|
||||
which = i // 36 if i % 36 < 7 else -1
|
||||
if i in range(0, 7) or i in range(36, 43) or i in range(72, 79):
|
||||
out[i] = costas[(i - (0 if i < 7 else 36 if i < 43 else 72))]
|
||||
else:
|
||||
out[i] = data[taken]
|
||||
taken += 1
|
||||
return out
|
||||
|
||||
|
||||
def bits_of(tones) -> np.ndarray:
|
||||
"""The bits those tones carry, sync thrown away."""
|
||||
tones = np.asarray(tones, dtype=np.uint8)
|
||||
if tones.size != TONES:
|
||||
raise ValueError(f"a transmission is {TONES} tones, not {tones.size}")
|
||||
data = [tones[i] for i in range(TONES)
|
||||
if not (i < 7 or 36 <= i < 43 or 72 <= i)]
|
||||
out = np.zeros(BITS, dtype=np.uint8)
|
||||
for k, tone in enumerate(data):
|
||||
value = UNGRAY[tone]
|
||||
out[3 * k] = (value >> 2) & 1
|
||||
out[3 * k + 1] = (value >> 1) & 1
|
||||
out[3 * k + 2] = value & 1
|
||||
return out
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Callsigns
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
ALPHANUM_SPACE = " 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ" # 37
|
||||
ALPHANUM = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ" # 36
|
||||
NUMERIC = "0123456789" # 10
|
||||
LETTERS_SPACE = " ABCDEFGHIJKLMNOPQRSTUVWXYZ" # 27
|
||||
FULL = " 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ+-./?" # 42
|
||||
WITH_SLASH = " 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ/" # 38
|
||||
|
||||
TOKENS = 2_063_592 # DE, QRZ, CQ and the CQ-with-an-argument forms
|
||||
MAX22 = 4_194_304 # room for a hashed callsign
|
||||
MAX_GRID = 32_400 # four-character grids, before the report forms
|
||||
|
||||
|
||||
def hash_of(call: str, width: int = 22) -> int:
|
||||
"""The short hash a compound callsign is carried as.
|
||||
|
||||
A callsign that will not fit in twenty-eight bits -- anything with a
|
||||
slash in it, and anything long -- travels as a hash, and the receiver
|
||||
is expected to have heard the full call earlier in the exchange and to
|
||||
remember it. That is why a fresh receiver prints <...>: not a decoding
|
||||
failure, but a station whose full name has not been said yet in
|
||||
anything this receiver heard.
|
||||
"""
|
||||
packed = 0
|
||||
padded = (call.upper() + " ")[:11]
|
||||
for ch in padded:
|
||||
packed = packed * 38 + (WITH_SLASH.index(ch) if ch in WITH_SLASH else 0)
|
||||
return int((47_055_833_459 * packed) >> (64 - width)) & ((1 << width) - 1)
|
||||
|
||||
|
||||
def is_standard_call(call: str) -> bool:
|
||||
"""Whether a callsign fits the twenty-eight bit form.
|
||||
|
||||
One or two characters of prefix, a digit, and up to three letters. Most
|
||||
callsigns on earth; not the ones with a slash in them.
|
||||
"""
|
||||
return _pack_standard(call) is not None
|
||||
|
||||
|
||||
def _pack_standard(call: str):
|
||||
"""A plain callsign as its number, or None if it is not a plain one.
|
||||
|
||||
The form is one or two characters of prefix, exactly one digit, and up
|
||||
to three letters -- and the twenty-eight bit packing wants the digit in
|
||||
the third place. Which character is *the* digit cannot be found by
|
||||
looking at the second one: Z33Z has a digit there and it belongs to the
|
||||
prefix. It is the last character that is not one of the trailing
|
||||
letters, so that is how it is found.
|
||||
"""
|
||||
call = (call or "").strip().upper()
|
||||
if not call or "/" in call or len(call) > 6:
|
||||
return None
|
||||
at = len(call) - 1
|
||||
while at >= 0 and call[at].isalpha():
|
||||
at -= 1
|
||||
if at < 0 or at > 2 or not call[at].isdigit():
|
||||
return None
|
||||
call = " " * (2 - at) + call
|
||||
if len(call) > 6:
|
||||
return None
|
||||
call = (call + " ")[:6]
|
||||
if (call[0] not in ALPHANUM_SPACE or call[1] not in ALPHANUM
|
||||
or call[2] not in NUMERIC):
|
||||
return None
|
||||
if any(c not in LETTERS_SPACE for c in call[3:6]):
|
||||
return None
|
||||
n = ALPHANUM_SPACE.index(call[0])
|
||||
n = n * 36 + ALPHANUM.index(call[1])
|
||||
n = n * 10 + NUMERIC.index(call[2])
|
||||
n = n * 27 + LETTERS_SPACE.index(call[3])
|
||||
n = n * 27 + LETTERS_SPACE.index(call[4])
|
||||
n = n * 27 + LETTERS_SPACE.index(call[5])
|
||||
return n
|
||||
|
||||
|
||||
def _unpack_standard(n: int) -> str:
|
||||
out = [""] * 6
|
||||
out[5] = LETTERS_SPACE[n % 27]
|
||||
n //= 27
|
||||
out[4] = LETTERS_SPACE[n % 27]
|
||||
n //= 27
|
||||
out[3] = LETTERS_SPACE[n % 27]
|
||||
n //= 27
|
||||
out[2] = NUMERIC[n % 10]
|
||||
n //= 10
|
||||
out[1] = ALPHANUM[n % 36]
|
||||
n //= 36
|
||||
out[0] = ALPHANUM_SPACE[n % 37]
|
||||
return "".join(out).strip()
|
||||
|
||||
|
||||
@dataclass
|
||||
class CallBook:
|
||||
"""Full callsigns heard earlier, so a hash can be turned back into one.
|
||||
|
||||
A compound callsign travels as a hash and is expected to be remembered
|
||||
from earlier in the same exchange. This is that memory. It is only
|
||||
ever added to by callsigns that arrived in full and passed their CRC,
|
||||
so a station that was never heard properly stays <...> rather than being
|
||||
guessed at -- a wrong callsign in a log is worse than a missing one.
|
||||
"""
|
||||
|
||||
by_hash: dict = field(default_factory=dict)
|
||||
|
||||
def remember(self, call: str) -> None:
|
||||
call = (call or "").strip().upper()
|
||||
if len(call) < 3 or call in ("CQ", "DE", "QRZ"):
|
||||
return
|
||||
for width in (10, 12, 22):
|
||||
self.by_hash[(width, hash_of(call, width))] = call
|
||||
|
||||
def look_up(self, value: int, width: int = 22) -> str:
|
||||
return self.by_hash.get((width, value), "<...>")
|
||||
|
||||
|
||||
def _unpack28(n: int, book: CallBook) -> str:
|
||||
"""One of the two callsign fields of a standard message."""
|
||||
if n < TOKENS:
|
||||
if n == 0:
|
||||
return "DE"
|
||||
if n == 1:
|
||||
return "QRZ"
|
||||
if n == 2:
|
||||
return "CQ"
|
||||
if n <= 1002:
|
||||
return f"CQ {n - 3:03d}"
|
||||
if n <= 532_443:
|
||||
rest = n - 1003
|
||||
letters = [""] * 4
|
||||
for i in (3, 2, 1, 0):
|
||||
letters[i] = LETTERS_SPACE[rest % 27]
|
||||
rest //= 27
|
||||
return "CQ " + "".join(letters).strip()
|
||||
return "<...>"
|
||||
n -= TOKENS
|
||||
if n < MAX22:
|
||||
return book.look_up(n, 22)
|
||||
return _unpack_standard(n - MAX22)
|
||||
|
||||
|
||||
def _unpack_grid(value: int, rover: int) -> str:
|
||||
"""The fifteen-bit field: a grid square, a signal report, or a sign-off."""
|
||||
if value <= MAX_GRID:
|
||||
n = value
|
||||
out = ["", "", "", ""]
|
||||
out[3] = NUMERIC[n % 10]
|
||||
n //= 10
|
||||
out[2] = NUMERIC[n % 10]
|
||||
n //= 10
|
||||
out[1] = chr(ord("A") + n % 18)
|
||||
n //= 18
|
||||
out[0] = chr(ord("A") + n % 18)
|
||||
grid = "".join(out)
|
||||
return ("R " + grid) if rover else grid
|
||||
report = value - MAX_GRID
|
||||
if report == 1:
|
||||
return ""
|
||||
if report == 2:
|
||||
return "RRR"
|
||||
if report == 3:
|
||||
return "RR73"
|
||||
if report == 4:
|
||||
return "73"
|
||||
decibels = report - 35
|
||||
return f"{'R' if rover else ''}{decibels:+03d}"
|
||||
|
||||
|
||||
def _pack_grid(text: str):
|
||||
"""The inverse: a grid, a report or a token as fifteen bits and a flag."""
|
||||
text = (text or "").strip().upper()
|
||||
if not text:
|
||||
return MAX_GRID + 1, 0
|
||||
if text == "RRR":
|
||||
return MAX_GRID + 2, 0
|
||||
if text == "RR73":
|
||||
return MAX_GRID + 3, 0
|
||||
if text == "73":
|
||||
return MAX_GRID + 4, 0
|
||||
rover = 0
|
||||
if text.startswith("R ") or (text.startswith("R") and len(text) > 1
|
||||
and text[1] in "+-"):
|
||||
rover, text = 1, text[2:].strip() if text[1] == " " else text[1:]
|
||||
if (len(text) == 4 and "A" <= text[0] <= "R" and "A" <= text[1] <= "R"
|
||||
and text[2].isdigit() and text[3].isdigit()):
|
||||
value = ((ord(text[0]) - 65) * 18 + (ord(text[1]) - 65)) * 10
|
||||
value = (value + int(text[2])) * 10 + int(text[3])
|
||||
return value, rover
|
||||
try:
|
||||
return MAX_GRID + 35 + int(text), rover
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Messages
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class Message:
|
||||
"""One decoded line, and what could be picked out of it."""
|
||||
|
||||
text: str = ""
|
||||
kind: str = "" # standard, free text, telemetry, ...
|
||||
calls: tuple = () # the callsigns in it, in order
|
||||
grid: str = "" # a four-character grid, where there was one
|
||||
report: str = "" # a signal report, where there was one
|
||||
calling: bool = False # whether this is a CQ
|
||||
|
||||
|
||||
def _number(bits, start: int, count: int) -> int:
|
||||
value = 0
|
||||
for k in range(count):
|
||||
value = (value << 1) | int(bits[start + k])
|
||||
return value
|
||||
|
||||
|
||||
def _put(bits, start: int, count: int, value: int) -> None:
|
||||
for k in range(count):
|
||||
bits[start + k] = (value >> (count - 1 - k)) & 1
|
||||
|
||||
|
||||
def free_text(bits77) -> str:
|
||||
"""The thirteen-character form: anything at all, at a price.
|
||||
|
||||
Base forty-two over seventy-one bits, which is thirteen characters and
|
||||
no callsign structure -- so a receiver cannot tell who sent it from the
|
||||
message alone, and nor can the network that reports these things.
|
||||
"""
|
||||
n = _number(bits77, 0, 71)
|
||||
out = [""] * 13
|
||||
for i in range(12, -1, -1):
|
||||
out[i] = FULL[n % 42]
|
||||
n //= 42
|
||||
return "".join(out).strip()
|
||||
|
||||
|
||||
def unpack(bits77, book: CallBook | None = None) -> Message:
|
||||
"""Seventy-seven bits as something to read.
|
||||
|
||||
The type is the last three bits, which is an odd place for it until you
|
||||
remember that everything before it is a different length in each form.
|
||||
"""
|
||||
bits77 = np.asarray(bits77, dtype=np.uint8)
|
||||
book = book or CallBook()
|
||||
i3 = _number(bits77, 74, 3)
|
||||
|
||||
if i3 == 0:
|
||||
n3 = _number(bits77, 71, 3)
|
||||
if n3 == 0:
|
||||
return Message(text=free_text(bits77), kind="free text")
|
||||
if n3 == 5:
|
||||
value = _number(bits77, 0, 71)
|
||||
return Message(text=f"{value:018X}", kind="telemetry")
|
||||
return Message(text=free_text(bits77), kind="free text")
|
||||
|
||||
if i3 in (1, 2):
|
||||
a = _number(bits77, 0, 28)
|
||||
a_rover = _number(bits77, 28, 1)
|
||||
b = _number(bits77, 29, 28)
|
||||
b_rover = _number(bits77, 57, 1)
|
||||
rover = _number(bits77, 58, 1)
|
||||
grid = _number(bits77, 59, 15)
|
||||
one = _unpack28(a, book)
|
||||
two = _unpack28(b, book)
|
||||
if a_rover:
|
||||
one += "/R"
|
||||
if b_rover:
|
||||
two += "/R"
|
||||
extra = _unpack_grid(grid, rover)
|
||||
text = " ".join(x for x in (one, two, extra) if x)
|
||||
looks_like_grid = (len(extra) == 4 and extra[:2].isalpha()
|
||||
and extra[2:].isdigit())
|
||||
return Message(text=text, kind="standard",
|
||||
calls=tuple(c for c in (one, two)
|
||||
if c not in ("CQ", "DE", "QRZ")
|
||||
and not c.startswith("CQ ")),
|
||||
grid=extra if looks_like_grid else "",
|
||||
report="" if looks_like_grid else extra,
|
||||
calling=one.startswith("CQ") or one == "QRZ")
|
||||
|
||||
if i3 == 4:
|
||||
# A compound callsign: one hashed, one carried in full as eleven
|
||||
# characters of base thirty-eight.
|
||||
short = _number(bits77, 0, 12)
|
||||
n58 = _number(bits77, 12, 58)
|
||||
flip = _number(bits77, 70, 1)
|
||||
kind = _number(bits77, 71, 2)
|
||||
rest = n58
|
||||
letters = [""] * 11
|
||||
for i in range(10, -1, -1):
|
||||
letters[i] = WITH_SLASH[rest % 38]
|
||||
rest //= 38
|
||||
full = "".join(letters).strip()
|
||||
other = book.look_up(short, 12)
|
||||
one, two = (full, other) if flip else (other, full)
|
||||
tail = {1: "RRR", 2: "RR73", 3: "73"}.get(kind, "")
|
||||
text = " ".join(x for x in (one, two, tail) if x)
|
||||
return Message(text=text, kind="non-standard",
|
||||
calls=tuple(c for c in (one, two) if c != "<...>"),
|
||||
report=tail, calling=one.startswith("CQ"))
|
||||
|
||||
return Message(text=free_text(bits77), kind=f"type {i3}")
|
||||
|
||||
|
||||
def pack(text: str, book: CallBook | None = None):
|
||||
"""The inverse, for the standard and free-text forms.
|
||||
|
||||
Here so the decoder can be tested against something that is not itself.
|
||||
A receiver never needs to build a message -- this one does not transmit
|
||||
-- but a test that encodes with the same misunderstanding it decodes
|
||||
with agrees with itself about anything they are both wrong about.
|
||||
"""
|
||||
text = (text or "").strip().upper()
|
||||
bits = np.zeros(MESSAGE_BITS, dtype=np.uint8)
|
||||
words = text.split()
|
||||
|
||||
if len(words) in (2, 3) or (len(words) == 4 and words[0] == "CQ"):
|
||||
made = _pack_standard_message(words, bits)
|
||||
if made is not None:
|
||||
return made
|
||||
return _pack_free_text(text, bits)
|
||||
|
||||
|
||||
def _pack_standard_message(words, bits):
|
||||
if words[0] == "CQ" and len(words) >= 3 and len(words[1]) <= 4 \
|
||||
and not _is_call(words[1]):
|
||||
one, rest = f"CQ {words[1]}", words[2:]
|
||||
else:
|
||||
one, rest = words[0], words[1:]
|
||||
if not rest:
|
||||
return None
|
||||
two, extra = rest[0], (rest[1] if len(rest) > 1 else "")
|
||||
a = _pack28(one)
|
||||
b = _pack28(two)
|
||||
if a is None or b is None:
|
||||
return None
|
||||
grid = _pack_grid(extra)
|
||||
if grid is None:
|
||||
return None
|
||||
value, rover = grid
|
||||
_put(bits, 0, 28, a)
|
||||
_put(bits, 28, 1, 0)
|
||||
_put(bits, 29, 28, b)
|
||||
_put(bits, 57, 1, 0)
|
||||
_put(bits, 58, 1, rover)
|
||||
_put(bits, 59, 15, value)
|
||||
_put(bits, 74, 3, 1)
|
||||
return bits
|
||||
|
||||
|
||||
def _is_call(word: str) -> bool:
|
||||
return any(c.isdigit() for c in word) and _pack_standard(word) is not None
|
||||
|
||||
|
||||
def _pack28(call: str):
|
||||
call = call.strip().upper()
|
||||
if call == "DE":
|
||||
return 0
|
||||
if call == "QRZ":
|
||||
return 1
|
||||
if call == "CQ":
|
||||
return 2
|
||||
if call.startswith("CQ "):
|
||||
rest = call[3:].strip()
|
||||
if rest.isdigit() and len(rest) == 3:
|
||||
return 3 + int(rest)
|
||||
if 1 <= len(rest) <= 4 and all(c in LETTERS_SPACE for c in rest):
|
||||
n = 0
|
||||
for ch in (rest + " ")[:4]:
|
||||
n = n * 27 + LETTERS_SPACE.index(ch)
|
||||
return 1003 + n
|
||||
return None
|
||||
plain = _pack_standard(call)
|
||||
if plain is not None:
|
||||
return TOKENS + MAX22 + plain
|
||||
# Deliberately not a hash. A hash always succeeds, and a packer that
|
||||
# always succeeds would send "HELLO WORLD" as two hashed callsigns
|
||||
# rather than as the free text it plainly is.
|
||||
return None
|
||||
|
||||
|
||||
def _pack_free_text(text: str, bits):
|
||||
text = "".join(c if c in FULL else " " for c in text)[:13]
|
||||
text = text.rjust(13)
|
||||
n = 0
|
||||
for ch in text:
|
||||
n = n * 42 + FULL.index(ch)
|
||||
_put(bits, 0, 71, n)
|
||||
_put(bits, 71, 3, 0)
|
||||
_put(bits, 74, 3, 0)
|
||||
return bits
|
||||
177
bandsaunter/ft8log.py
Normal file
177
bandsaunter/ft8log.py
Normal file
|
|
@ -0,0 +1,177 @@
|
|||
"""Writing FT8 decodes down.
|
||||
|
||||
Three forms, because three different things want to read them. The log is
|
||||
for a person and for this program's own report; the CSV is for a
|
||||
spreadsheet; the ADIF is for the logging and spotting programs every
|
||||
amateur already has.
|
||||
|
||||
The ADIF is the one worth a note. It is the format for recording contacts,
|
||||
and nothing here makes a contact -- this receiver does not transmit. So
|
||||
what is written is a record of a station *heard*, marked as such, which is
|
||||
what a propagation map or a reverse-beacon feed wants. Passing it off as a
|
||||
worked contact would put claims into somebody's log that they cannot make.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from . import __version__
|
||||
|
||||
__all__ = ["Ft8Log", "open_log", "write_csv", "write_adif", "read_log",
|
||||
"LOG_VERSION"]
|
||||
|
||||
LOG_VERSION = 1
|
||||
|
||||
|
||||
@dataclass
|
||||
class Ft8Log:
|
||||
"""A file of decodes, one to a line, written as they arrive."""
|
||||
|
||||
path: Path
|
||||
handle: object = None
|
||||
lines: int = 0
|
||||
|
||||
def append(self, found) -> None:
|
||||
if self.handle is None:
|
||||
return
|
||||
when = datetime.fromtimestamp(found.at, timezone.utc)
|
||||
self.handle.write(
|
||||
f"{when.strftime('%Y-%m-%d %H:%M:%S')} "
|
||||
f"{found.snr_db:>4.0f} {found.offset:>5.1f} "
|
||||
f"{found.hertz:>7.1f} ~ {found.text}\n")
|
||||
self.handle.flush()
|
||||
self.lines += 1
|
||||
|
||||
def close(self) -> None:
|
||||
if self.handle is not None:
|
||||
self.handle.close()
|
||||
self.handle = None
|
||||
|
||||
|
||||
def open_log(directory, started: float, frequency: float,
|
||||
band: str = "") -> Ft8Log | None:
|
||||
"""A new log for this run, or nothing if it cannot be opened."""
|
||||
when = datetime.fromtimestamp(started, timezone.utc)
|
||||
path = Path(directory) / f"ft8-{when.strftime('%Y%m%d-%H%M%S')}.txt"
|
||||
try:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
handle = open(path, "w")
|
||||
except OSError:
|
||||
return None
|
||||
handle.write(f"# bandsaunter {__version__} FT8 log v{LOG_VERSION}\n")
|
||||
handle.write(f"# {frequency / 1e6:g} MHz"
|
||||
f"{(' ' + band) if band else ''}, times are UTC\n")
|
||||
handle.write("# when snr dt hz ~ message\n")
|
||||
handle.flush()
|
||||
return Ft8Log(path=path, handle=handle)
|
||||
|
||||
|
||||
def read_log(path) -> list[dict]:
|
||||
"""A log back off the disk, for reading an evening again.
|
||||
|
||||
Deliberately forgiving: a line that cannot be parsed is skipped rather
|
||||
than raising, because a log is often read while it is still being
|
||||
written and the last line may be half there.
|
||||
"""
|
||||
out = []
|
||||
try:
|
||||
text = Path(path).read_text()
|
||||
except OSError:
|
||||
return out
|
||||
for line in text.splitlines():
|
||||
if line.startswith("#") or "~" not in line:
|
||||
continue
|
||||
head, message = line.split("~", 1)
|
||||
parts = head.split()
|
||||
if len(parts) < 5:
|
||||
continue
|
||||
try:
|
||||
when = datetime.strptime(f"{parts[0]} {parts[1]}",
|
||||
"%Y-%m-%d %H:%M:%S")
|
||||
out.append({"at": when.replace(tzinfo=timezone.utc).timestamp(),
|
||||
"snr_db": float(parts[2]), "offset": float(parts[3]),
|
||||
"hertz": float(parts[4]), "text": message.strip()})
|
||||
except ValueError:
|
||||
continue
|
||||
return out
|
||||
|
||||
|
||||
def write_csv(path, decodes) -> Path | None:
|
||||
try:
|
||||
with open(path, "w", newline="") as fh:
|
||||
writer = csv.writer(fh)
|
||||
writer.writerow(["utc", "snr_db", "offset_s", "hertz", "kind",
|
||||
"calls", "grid", "report", "cq", "message"])
|
||||
for d in decodes:
|
||||
when = datetime.fromtimestamp(d.at, timezone.utc)
|
||||
writer.writerow([when.strftime("%Y-%m-%d %H:%M:%S"),
|
||||
f"{d.snr_db:.0f}", f"{d.offset:.1f}",
|
||||
f"{d.hertz:.1f}", d.kind,
|
||||
" ".join(d.calls), d.grid, d.report,
|
||||
"yes" if d.calling else "", d.text])
|
||||
except OSError:
|
||||
return None
|
||||
return Path(path)
|
||||
|
||||
|
||||
def _adif(field: str, value: str) -> str:
|
||||
value = str(value)
|
||||
return f"<{field}:{len(value)}>{value} "
|
||||
|
||||
|
||||
def write_adif(path, decodes, band: str = "", frequency: float = 0.0,
|
||||
my_grid: str = "") -> Path | None:
|
||||
"""The decodes as ADIF records of stations heard.
|
||||
|
||||
One record per station rather than per decode: a logging program that
|
||||
was handed forty identical records for one evening's CQ calls would be
|
||||
right to complain. The strongest report is the one kept, that being
|
||||
the one worth passing on.
|
||||
"""
|
||||
best: dict = {}
|
||||
for d in decodes:
|
||||
sender = d.calls[1] if len(d.calls) > 1 else (
|
||||
d.calls[0] if d.calls else "")
|
||||
if not sender or sender == "<...>":
|
||||
continue
|
||||
was = best.get(sender)
|
||||
if was is None or d.snr_db > was.snr_db:
|
||||
best[sender] = d
|
||||
if d.grid and not getattr(best[sender], "grid", ""):
|
||||
best[sender].grid = d.grid
|
||||
try:
|
||||
with open(path, "w") as fh:
|
||||
fh.write(f"bandsaunter {__version__} — FT8 stations heard.\n")
|
||||
fh.write("These are receptions, not contacts: this program "
|
||||
"does not transmit.\n")
|
||||
fh.write(_adif("ADIF_VER", "3.1.4") + "\n")
|
||||
fh.write(_adif("PROGRAMID", "bandsaunter") + "\n")
|
||||
fh.write(_adif("PROGRAMVERSION", __version__) + "\n")
|
||||
fh.write("<EOH>\n")
|
||||
for call, d in sorted(best.items()):
|
||||
when = datetime.fromtimestamp(d.at, timezone.utc)
|
||||
row = (_adif("CALL", call)
|
||||
+ _adif("QSO_DATE", when.strftime("%Y%m%d"))
|
||||
+ _adif("TIME_ON", when.strftime("%H%M%S"))
|
||||
+ _adif("MODE", "FT8"))
|
||||
if band:
|
||||
row += _adif("BAND", band)
|
||||
if frequency:
|
||||
row += _adif("FREQ", f"{frequency / 1e6:.6f}")
|
||||
if d.grid:
|
||||
row += _adif("GRIDSQUARE", d.grid)
|
||||
if my_grid:
|
||||
row += _adif("MY_GRIDSQUARE", my_grid.upper())
|
||||
row += _adif("RST_RCVD", f"{d.snr_db:.0f}")
|
||||
# Said plainly in the record itself, so that a log this is
|
||||
# imported into cannot quietly become a claim.
|
||||
row += _adif("COMMENT", "heard only, not worked")
|
||||
row += _adif("QSO_RANDOM", "Y")
|
||||
fh.write(row + "<EOR>\n")
|
||||
except OSError:
|
||||
return None
|
||||
return Path(path)
|
||||
172
bandsaunter/ft8sim.py
Normal file
172
bandsaunter/ft8sim.py
Normal file
|
|
@ -0,0 +1,172 @@
|
|||
"""A made-up FT8 band, on a real clock.
|
||||
|
||||
For trying the display, the report and the exports without an aerial, and
|
||||
for testing the whole path from samples to tables. It stands in for a
|
||||
receiver: same `tune`, `read_samples` and `close`, same sample rate, same
|
||||
complex samples.
|
||||
|
||||
What it cannot tell you is whether your receiver hears anything, which is
|
||||
the one question a simulator is never allowed to answer. It is also the
|
||||
lesson this project learned expensively on another band: an encoder tested
|
||||
against its own decoder agrees with it about anything they are both wrong
|
||||
about. So the decoding is tested against real off-air recordings; this is
|
||||
here for the parts above the decoding.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
import time
|
||||
|
||||
import numpy as np
|
||||
|
||||
from . import ft8code as code
|
||||
from . import ft8wave as wave
|
||||
|
||||
__all__ = ["SimulatedBand", "CALLS", "GRIDS"]
|
||||
|
||||
# A plausible spread: a few continents, a few strengths, a few habits.
|
||||
CALLS = ("G4UJS", "SP4FCA", "IK4LZH", "EA1ABT", "DL1UDO", "OH8GDU",
|
||||
"JA2GQT", "VK4BLE", "RA6ABO", "F4FSY", "K1ABC", "W2XYZ",
|
||||
"PA3EPP", "SM0ABC", "LZ1LZ", "9A9TT", "ZS6ABC", "LU1AAA")
|
||||
GRIDS = ("IO83", "KO03", "JN54", "IN73", "JO31", "KP24",
|
||||
"PM85", "QG62", "KN96", "JN25", "FN42", "FN20",
|
||||
"JO21", "JO99", "KN12", "JN75", "KG33", "GF05")
|
||||
|
||||
|
||||
class SimulatedBand:
|
||||
"""A receiver that invents a band of FT8 stations.
|
||||
|
||||
Paced against the real clock, because everything about FT8 is: the
|
||||
slots are quarter-minutes of UTC, and a simulator that produced fifteen
|
||||
seconds of audio instantly would make the section look as though it
|
||||
worked while testing none of the timing that makes it work.
|
||||
"""
|
||||
|
||||
def __init__(self, rate: float = 240_000.0, seed: int = 0,
|
||||
stations: int = 12, realtime: bool = True,
|
||||
clock=time.time, sleep=time.sleep):
|
||||
self.rate = float(rate)
|
||||
self.frequency = 0.0
|
||||
self.rng = np.random.default_rng(seed)
|
||||
self.stations = max(1, int(stations))
|
||||
self.realtime = realtime
|
||||
self.clock = clock
|
||||
self.sleep = sleep
|
||||
self.closed = False
|
||||
self._audio_rate = 12_000
|
||||
self._made: dict[float, np.ndarray] = {}
|
||||
self._one_sided: dict[float, np.ndarray] = {}
|
||||
self._at = None
|
||||
|
||||
# -- the receiver's shape -------------------------------------------
|
||||
def tune(self, hz: float) -> None:
|
||||
self.frequency = float(hz)
|
||||
|
||||
def close(self) -> None:
|
||||
self.closed = True
|
||||
|
||||
def read_samples(self, count: int):
|
||||
"""A block of samples, paced to take as long as it would on the air."""
|
||||
if self.closed:
|
||||
return None
|
||||
now = self.clock()
|
||||
if self._at is None:
|
||||
self._at = now
|
||||
wanted = count / self.rate
|
||||
if self.realtime:
|
||||
behind = self._at + wanted - self.clock()
|
||||
if behind > 0:
|
||||
self.sleep(behind)
|
||||
block = self._samples_for(self._at, count)
|
||||
self._at += wanted
|
||||
return block
|
||||
|
||||
# -- making the band ------------------------------------------------
|
||||
def _slot_audio(self, slot_at: float) -> np.ndarray:
|
||||
"""One slot of the band, at twelve kilohertz, made once and kept."""
|
||||
have = self._made.get(slot_at)
|
||||
if have is not None:
|
||||
return have
|
||||
rng = np.random.default_rng(int(slot_at) & 0xFFFFFFFF)
|
||||
total = int(self._audio_rate * code.SLOT_S)
|
||||
audio = rng.standard_normal(total).astype(np.float32) * 0.02
|
||||
# Stations pick a frequency and keep it, mostly, and take turns.
|
||||
even = int(slot_at / code.SLOT_S) % 2 == 0
|
||||
for i in range(self.stations):
|
||||
if (i % 2 == 0) != even:
|
||||
continue
|
||||
call = CALLS[(i + int(slot_at / code.SLOT_S) // 2) % len(CALLS)]
|
||||
grid = GRIDS[CALLS.index(call)]
|
||||
other = CALLS[(i * 7 + 3) % len(CALLS)]
|
||||
pick = rng.integers(0, 3)
|
||||
if pick == 0:
|
||||
text = f"CQ {call} {grid}"
|
||||
elif pick == 1:
|
||||
text = f"{other} {call} {grid}"
|
||||
else:
|
||||
text = f"{other} {call} {rng.integers(-24, 5):+03d}"
|
||||
hertz = 300.0 + i * 190.0 + float(rng.integers(-30, 30))
|
||||
strength = float(rng.uniform(-18.0, 10.0))
|
||||
one = wave.transmit(text, rate=float(self._audio_rate),
|
||||
hertz=hertz,
|
||||
offset=0.5 + float(rng.uniform(-0.3, 0.6)),
|
||||
seconds=code.SLOT_S)
|
||||
scale = 10.0 ** (strength / 20.0) * 0.05
|
||||
audio[:one.size] += (one * scale).astype(np.float32)
|
||||
if len(self._made) > 2:
|
||||
self._made.clear()
|
||||
self._made[slot_at] = audio
|
||||
return audio
|
||||
|
||||
def _samples_for(self, at: float, count: int):
|
||||
"""Complex samples covering ``count`` samples from ``at``.
|
||||
|
||||
The band is made as audio and then put back on a carrier, which is
|
||||
the reverse of what the section does to it. One-sided, so that
|
||||
what comes out of the demodulator is what went in: a real audio
|
||||
tone at 1000 Hz has to arrive as a complex tone at +1000 Hz, or
|
||||
upper sideband and lower sideband would be the same thing.
|
||||
|
||||
Held at the audio rate and stepped through, rather than properly
|
||||
resampled. The images that leaves sit at multiples of twelve
|
||||
kilohertz and the demodulator's own decimation filter removes them
|
||||
-- which is worth saying out loud, because it means this simulator
|
||||
leans on the thing it is being used to test. That is tolerable for
|
||||
the display and the report, and is exactly why the decoding itself
|
||||
is tested against real recordings instead.
|
||||
"""
|
||||
out = np.zeros(count, dtype=np.complex64)
|
||||
step = 1.0 / self.rate
|
||||
ratio = self._audio_rate / self.rate
|
||||
for k in range(0, count, 4096):
|
||||
piece = min(4096, count - k)
|
||||
when = at + k * step
|
||||
slot_at = math.floor(when / code.SLOT_S) * code.SLOT_S
|
||||
audio = self._analytic_slot(slot_at)
|
||||
into = (when - slot_at) * self._audio_rate
|
||||
idx = np.clip((into + np.arange(piece) * ratio).astype(np.int64),
|
||||
0, audio.size - 1)
|
||||
out[k:k + piece] = audio[idx].astype(np.complex64)
|
||||
return out
|
||||
|
||||
def _analytic_slot(self, slot_at: float) -> np.ndarray:
|
||||
have = self._one_sided.get(slot_at)
|
||||
if have is None:
|
||||
have = _analytic(self._slot_audio(slot_at))
|
||||
# Two slots kept: a block of samples commonly straddles a
|
||||
# boundary, and rebuilding a slot for every such block is most
|
||||
# of the simulator's time.
|
||||
if len(self._one_sided) > 2:
|
||||
self._one_sided.clear()
|
||||
self._one_sided[slot_at] = have
|
||||
return have
|
||||
|
||||
|
||||
def _analytic(real_audio: np.ndarray) -> np.ndarray:
|
||||
"""The one-sided version of a real signal: no energy below zero."""
|
||||
spectrum = np.fft.fft(real_audio)
|
||||
n = spectrum.size
|
||||
spectrum[n // 2 + 1:] = 0.0
|
||||
spectrum[1:(n + 1) // 2] *= 2.0
|
||||
return np.fft.ifft(spectrum)
|
||||
219
bandsaunter/ft8tables.py
Normal file
219
bandsaunter/ft8tables.py
Normal file
|
|
@ -0,0 +1,219 @@
|
|||
"""The two fixed tables the FT8 code is defined by.
|
||||
|
||||
Everything else in this section is worked out from first principles -- the
|
||||
tones, the sync, the parity arithmetic, the way a callsign is squeezed into
|
||||
twenty-eight bits. These two cannot be, because they are not derived from
|
||||
anything. They are the code, chosen once by its designers and published, and
|
||||
a receiver that guessed at them would be speaking a different protocol.
|
||||
|
||||
Taken from ft8_lib (https://github.com/kgoba/ft8_lib), MIT licensed,
|
||||
Copyright (c) 2018 K\u0101rlis Goba, which took them in turn from WSJT-X.
|
||||
Reproduced here under that licence; the MIT terms permit it and this file
|
||||
carries the attribution they ask for. Nothing else in bandsaunter comes from
|
||||
there: the decoder below is its own.
|
||||
|
||||
GENERATOR is the parity half of the systematic generator: row m gives the
|
||||
message bits that are exclusive-ored to make parity bit m, as ninety-one bits
|
||||
in twelve bytes, most significant bit first.
|
||||
|
||||
CHECKS is the sparse parity-check matrix, as the bit positions each of the
|
||||
eighty-three checks covers, one-based as published. It is *not* derivable
|
||||
from GENERATOR even though both describe the same code: the generator's
|
||||
parity half runs to a weight of fifty-odd bits per row, and belief
|
||||
propagation on a matrix that dense says nothing useful. The sparse one is a
|
||||
different basis for the same dual space, six or seven bits to a check, and it
|
||||
is the one the decoding works on. That they agree is checked by a test
|
||||
rather than assumed -- encode at random, and every sparse check must come out
|
||||
even.
|
||||
"""
|
||||
|
||||
__all__ = ["GENERATOR", "CHECKS", "COSTAS", "GRAY", "BITS", "PAYLOAD",
|
||||
"PARITY", "TONES", "CRC_POLYNOMIAL", "CRC_BITS"]
|
||||
|
||||
# The 7x7 Costas array that starts, middles and ends every transmission.
|
||||
COSTAS = (3, 1, 4, 0, 6, 5, 2)
|
||||
|
||||
# Three bits to a tone, in Gray order, so that mistaking a tone for its
|
||||
# neighbour costs one bit rather than three.
|
||||
GRAY = (0, 1, 3, 2, 5, 6, 4, 7)
|
||||
|
||||
BITS = 174 # bits in a codeword
|
||||
PAYLOAD = 91 # of which are message and checksum
|
||||
PARITY = BITS - PAYLOAD
|
||||
TONES = 79 # channel symbols, sync included
|
||||
|
||||
# CRC-14, without the leading bit.
|
||||
CRC_POLYNOMIAL = 0x2757
|
||||
CRC_BITS = 14
|
||||
|
||||
GENERATOR: tuple[str, ...] = (
|
||||
"8329CE11BF31EAF509F27FC0",
|
||||
"761C264E25C2593354931320",
|
||||
"DC265902FB277C6410A1BDC0",
|
||||
"1B3F417858CD2DD33EC7F620",
|
||||
"09FDA4FEE04195FD034783A0",
|
||||
"077CCCC11B8873ED5C3D48A0",
|
||||
"29B62AFE3CA036F4FE1A9DA0",
|
||||
"6054FAF5F35D96D3B0C8C3E0",
|
||||
"E20798E4310EED27884AE900",
|
||||
"775C9C08E80E26DDAE563180",
|
||||
"B0B811028C2BF997213487C0",
|
||||
"18A0C9231FC60ADF5C5EA320",
|
||||
"76471E8302A0721E01B12B80",
|
||||
"FFBCCB80CA8341FAFB47B2E0",
|
||||
"66A72A158F9325A2BF671700",
|
||||
"C4243689FE85B1C51363A180",
|
||||
"0DFF739414D1A1B34B1C2700",
|
||||
"15B48830636C8B99894972E0",
|
||||
"29A89C0D3DE81D665489B0E0",
|
||||
"4F126F37FA51CBE61BD6B940",
|
||||
"99C47239D0D97D3C84E09400",
|
||||
"1919B75119765621BB4F1E80",
|
||||
"09DB12D731FAEE0B86DF6B80",
|
||||
"488FC33DF43FBDEEA4EAFB40",
|
||||
"827423EE40B675F756EB5FE0",
|
||||
"ABE197C484CB74757144A9A0",
|
||||
"2B500E4BC0EC5A6D2BDBDD00",
|
||||
"C474AA53D702187616693600",
|
||||
"8EBA1A13DB3390BD6718CEC0",
|
||||
"753844673A27782CC42012E0",
|
||||
"06FF83A145C37035A5C12680",
|
||||
"3B37417858CC2DD33EC3F620",
|
||||
"9A4A5A28EE17CA9C324842C0",
|
||||
"BC29F465309C977E89610A40",
|
||||
"2663AE6DDF8B5CE2BB294880",
|
||||
"46F231EFE457034C18144180",
|
||||
"3FB2CE85ABE9B0C72E06FBE0",
|
||||
"DE87481F282C153971A0A2E0",
|
||||
"FCD7CCF23C69FA99BBA14120",
|
||||
"F0261447E9490CA8E474CEC0",
|
||||
"4410115818196F95CDD70120",
|
||||
"088FC31DF4BFBDE2A4EAFB40",
|
||||
"B8FEF1B6307729FB0A078C00",
|
||||
"5AFEA7ACCCB77BBC9D99A900",
|
||||
"49A7016AC653F65ECDC90760",
|
||||
"1944D085BE4E7DA8D6CC7D00",
|
||||
"251F62ADC4032F0EE7140020",
|
||||
"56471F8702A0721E00B12B80",
|
||||
"2B8E4923F2DD51E2D537FA00",
|
||||
"6B550A40A66F4755DE95C260",
|
||||
"A18AD28D4E27FE92A4F6C840",
|
||||
"10C2E586388CB82A3D807580",
|
||||
"EF34A41817EE02133DB2EB00",
|
||||
"7E9C0C54325A9C15836E0000",
|
||||
"3693E572D1FDE4CDF079E860",
|
||||
"BFB2CEC5ABE1B0C72E07FBE0",
|
||||
"7EE18230C583CCCC57D4B080",
|
||||
"A066CB2FEDAFC9F526641260",
|
||||
"BB23725ABC47CC5F4CC4CD20",
|
||||
"DED9DBA3BEE40C59B5609B40",
|
||||
"D9A7016AC653E6DECDC90360",
|
||||
"9AD46AED5F707F280AB5FC40",
|
||||
"E5921C77822587316D7D3C20",
|
||||
"4F14DA8242A8B86DCA733520",
|
||||
"8B8B507AD467D4441DF770E0",
|
||||
"22831C9CF1169467AD04B680",
|
||||
"213B838FE2AE54C38EE71800",
|
||||
"5D926B6DD71F085181A4E120",
|
||||
"66AB79D4B29EE6E69509E560",
|
||||
"958148682D748A38DD68BAA0",
|
||||
"B8CE020CF069C32A723AB140",
|
||||
"F4331D6D461607E957527460",
|
||||
"6DA23BA424B9596133CF9C80",
|
||||
"A636BCBC7B30C5FBEAE67FE0",
|
||||
"5CB0D86A07DF654A9089A200",
|
||||
"F11F106848780FC9ECDD80A0",
|
||||
"1FBB5364FB8D2C9D730D5BA0",
|
||||
"FCB86BC70A50C9D02A5D0340",
|
||||
"A534433029EAC15F322E34C0",
|
||||
"C989D9C7C3D3B8C55D751300",
|
||||
"7BB38B2F0186D46643AE9620",
|
||||
"2644EBADEB44B9467D1F42C0",
|
||||
"608CC857594BFBB55D696000",
|
||||
)
|
||||
|
||||
CHECKS: tuple[tuple[int, ...], ...] = (
|
||||
(4, 31, 59, 91, 92, 96, 153),
|
||||
(5, 32, 60, 93, 115, 146),
|
||||
(6, 24, 61, 94, 122, 151),
|
||||
(7, 33, 62, 95, 96, 143),
|
||||
(8, 25, 63, 83, 93, 96, 148),
|
||||
(6, 32, 64, 97, 126, 138),
|
||||
(5, 34, 65, 78, 98, 107, 154),
|
||||
(9, 35, 66, 99, 139, 146),
|
||||
(10, 36, 67, 100, 107, 126),
|
||||
(11, 37, 67, 87, 101, 139, 158),
|
||||
(12, 38, 68, 102, 105, 155),
|
||||
(13, 39, 69, 103, 149, 162),
|
||||
(8, 40, 70, 82, 104, 114, 145),
|
||||
(14, 41, 71, 88, 102, 123, 156),
|
||||
(15, 42, 59, 106, 123, 159),
|
||||
(1, 33, 72, 106, 107, 157),
|
||||
(16, 43, 73, 108, 141, 160),
|
||||
(17, 37, 74, 81, 109, 131, 154),
|
||||
(11, 44, 75, 110, 121, 166),
|
||||
(45, 55, 64, 111, 130, 161, 173),
|
||||
(8, 46, 71, 112, 119, 166),
|
||||
(18, 36, 76, 89, 113, 114, 143),
|
||||
(19, 38, 77, 104, 116, 163),
|
||||
(20, 47, 70, 92, 138, 165),
|
||||
(2, 48, 74, 113, 128, 160),
|
||||
(21, 45, 78, 83, 117, 121, 151),
|
||||
(22, 47, 58, 118, 127, 164),
|
||||
(16, 39, 62, 112, 134, 158),
|
||||
(23, 43, 79, 120, 131, 145),
|
||||
(19, 35, 59, 73, 110, 125, 161),
|
||||
(20, 36, 63, 94, 136, 161),
|
||||
(14, 31, 79, 98, 132, 164),
|
||||
(3, 44, 80, 124, 127, 169),
|
||||
(19, 46, 81, 117, 135, 167),
|
||||
(7, 49, 58, 90, 100, 105, 168),
|
||||
(12, 50, 61, 118, 119, 144),
|
||||
(13, 51, 64, 114, 118, 157),
|
||||
(24, 52, 76, 129, 148, 149),
|
||||
(25, 53, 69, 90, 101, 130, 156),
|
||||
(20, 46, 65, 80, 120, 140, 170),
|
||||
(21, 54, 77, 100, 140, 171),
|
||||
(35, 82, 133, 142, 171, 174),
|
||||
(14, 30, 83, 113, 125, 170),
|
||||
(4, 29, 68, 120, 134, 173),
|
||||
(1, 4, 52, 57, 86, 136, 152),
|
||||
(26, 51, 56, 91, 122, 137, 168),
|
||||
(52, 84, 110, 115, 145, 168),
|
||||
(7, 50, 81, 99, 132, 173),
|
||||
(23, 55, 67, 95, 172, 174),
|
||||
(26, 41, 77, 109, 141, 148),
|
||||
(2, 27, 41, 61, 62, 115, 133),
|
||||
(27, 40, 56, 124, 125, 126),
|
||||
(18, 49, 55, 124, 141, 167),
|
||||
(6, 33, 85, 108, 116, 156),
|
||||
(28, 48, 70, 85, 105, 129, 158),
|
||||
(9, 54, 63, 131, 147, 155),
|
||||
(22, 53, 68, 109, 121, 174),
|
||||
(3, 13, 48, 78, 95, 123),
|
||||
(31, 69, 133, 150, 155, 169),
|
||||
(12, 43, 66, 89, 97, 135, 159),
|
||||
(5, 39, 75, 102, 136, 167),
|
||||
(2, 54, 86, 101, 135, 164),
|
||||
(15, 56, 87, 108, 119, 171),
|
||||
(10, 44, 82, 91, 111, 144, 149),
|
||||
(23, 34, 71, 94, 127, 153),
|
||||
(11, 49, 88, 92, 142, 157),
|
||||
(29, 34, 87, 97, 147, 162),
|
||||
(30, 50, 60, 86, 137, 142, 162),
|
||||
(10, 53, 66, 84, 112, 128, 165),
|
||||
(22, 57, 85, 93, 140, 159),
|
||||
(28, 32, 72, 103, 132, 166),
|
||||
(28, 29, 84, 88, 117, 143, 150),
|
||||
(1, 26, 45, 80, 128, 147),
|
||||
(17, 27, 89, 103, 116, 153),
|
||||
(51, 57, 98, 163, 165, 172),
|
||||
(21, 37, 73, 138, 152, 169),
|
||||
(16, 47, 76, 130, 137, 154),
|
||||
(3, 24, 30, 72, 104, 139),
|
||||
(9, 40, 90, 106, 134, 151),
|
||||
(15, 58, 60, 74, 111, 150, 163),
|
||||
(18, 42, 79, 144, 146, 152),
|
||||
(25, 38, 65, 99, 122, 160),
|
||||
(17, 42, 75, 129, 170, 172),
|
||||
)
|
||||
411
bandsaunter/ft8wave.py
Normal file
411
bandsaunter/ft8wave.py
Normal file
|
|
@ -0,0 +1,411 @@
|
|||
"""Finding FT8 in fifteen seconds of audio.
|
||||
|
||||
The coding is in ft8code; this is the part that has to deal with the air.
|
||||
A slot of audio three kilohertz wide holds anything from nothing to forty
|
||||
transmissions, each fifty hertz wide, each starting when its operator's
|
||||
clock said to rather than when ours did, and most of them below the noise.
|
||||
|
||||
The shape of the search:
|
||||
|
||||
1. a waterfall -- the whole slot as power against time and frequency, at
|
||||
half a symbol and half a tone, so nothing falls between two stools;
|
||||
2. the sync search -- every place a Costas array could be, scored;
|
||||
3. for each candidate, the eight tone powers of each of seventy-nine
|
||||
symbols, turned into how strongly each bit is believed;
|
||||
4. the error correction, which either converges or does not;
|
||||
5. the checksum, which decides whether to believe it.
|
||||
|
||||
Steps four and five are why this works at all. Nothing up to step three is
|
||||
clever enough to read a signal that is twenty decibels under the noise; what
|
||||
it produces is a wash of barely-tilted opinions, and the code turns a
|
||||
hundred and seventy-four of those into ninety-one bits of certainty -- or
|
||||
into nothing, which is the other important answer.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
|
||||
import numpy as np
|
||||
|
||||
from . import ft8code as code
|
||||
from .ft8code import COSTAS, SYMBOL_S, TONES
|
||||
|
||||
__all__ = ["Waterfall", "Candidate", "Decode", "listen_to", "waterfall",
|
||||
"candidates", "soft_bits", "TIME_STEPS", "FREQ_STEPS"]
|
||||
|
||||
# Half a symbol and half a tone. A transmission that started a quarter of a
|
||||
# symbol late, or sits a quarter of a tone off, is still caught: with steps
|
||||
# a whole symbol wide the worst case loses most of the signal to the gap
|
||||
# between two bins, and the worst case is common because nobody's clock and
|
||||
# nobody's dial agree with ours.
|
||||
TIME_STEPS = 2
|
||||
FREQ_STEPS = 2
|
||||
|
||||
SYNC_AT = (0, 36, 72) # where the three Costas arrays sit, in symbols
|
||||
|
||||
# A perfectly-timed station does not start at the top of the slot. The
|
||||
# transmission is 12.64 seconds long in a slot of fifteen, and the
|
||||
# convention every FT8 program follows is to begin half a second in and to
|
||||
# report lateness against that -- so a station whose clock is right reports
|
||||
# as dt 0.0 rather than dt 0.5. Followed here so the figures mean the same
|
||||
# thing as everybody else's.
|
||||
NOMINAL_START = 0.5
|
||||
|
||||
|
||||
@dataclass
|
||||
class Waterfall:
|
||||
"""The slot as power against time and frequency."""
|
||||
|
||||
power: np.ndarray # (times, bins), already in decibels
|
||||
rate: float
|
||||
hz_per_bin: float
|
||||
seconds_per_step: float
|
||||
|
||||
@property
|
||||
def times(self) -> int:
|
||||
return self.power.shape[0]
|
||||
|
||||
@property
|
||||
def bins(self) -> int:
|
||||
return self.power.shape[1]
|
||||
|
||||
|
||||
@dataclass
|
||||
class Candidate:
|
||||
"""Somewhere a transmission might start, and how much it looks like one."""
|
||||
|
||||
step: int # in half-symbols from the top of the slot
|
||||
bin: int # in half-tones up the waterfall
|
||||
score: float
|
||||
|
||||
def seconds(self, fall: Waterfall) -> float:
|
||||
"""How late this started, against the nominal start of sending."""
|
||||
return self.step * fall.seconds_per_step - NOMINAL_START
|
||||
|
||||
def hertz(self, fall: Waterfall) -> float:
|
||||
return self.bin * fall.hz_per_bin
|
||||
|
||||
|
||||
@dataclass
|
||||
class Decode:
|
||||
"""One transmission, read."""
|
||||
|
||||
text: str = ""
|
||||
kind: str = ""
|
||||
calls: tuple = ()
|
||||
grid: str = ""
|
||||
report: str = ""
|
||||
calling: bool = False
|
||||
hertz: float = 0.0
|
||||
offset: float = 0.0 # how late it started, in seconds
|
||||
snr_db: float = 0.0
|
||||
score: float = 0.0
|
||||
at: float = 0.0 # when the slot began, as a clock time
|
||||
|
||||
|
||||
def waterfall(audio, rate: float) -> Waterfall:
|
||||
"""The slot, as power against time and frequency.
|
||||
|
||||
One symbol of samples to a row, stepped half a symbol at a time, and
|
||||
zero-padded to twice its length before the transform so the bins come
|
||||
out half a tone apart. Padding rather than a longer window on purpose:
|
||||
a longer window would average two symbols together, and the thing being
|
||||
looked for changes every symbol.
|
||||
"""
|
||||
audio = np.asarray(audio, dtype=np.float32)
|
||||
per_symbol = int(round(rate * SYMBOL_S))
|
||||
if per_symbol < 8:
|
||||
raise ValueError("the audio rate is too low to hold FT8 tones")
|
||||
step = per_symbol // TIME_STEPS
|
||||
size = per_symbol * FREQ_STEPS
|
||||
window = np.hanning(per_symbol).astype(np.float32)
|
||||
|
||||
rows = 1 + max(0, (audio.size - per_symbol) // step)
|
||||
if rows < 1:
|
||||
raise ValueError("not enough audio for a single symbol")
|
||||
# One strided view of the whole slot, transformed in a single call:
|
||||
# a loop over two hundred rows in Python is most of the decode time.
|
||||
frames = np.lib.stride_tricks.sliding_window_view(
|
||||
audio, per_symbol)[::step][:rows]
|
||||
spectrum = np.fft.rfft(frames * window, n=size, axis=1)
|
||||
power = np.abs(spectrum).astype(np.float32)
|
||||
# Decibels, because everything downstream adds and subtracts these and
|
||||
# the useful comparisons are all ratios.
|
||||
floor = max(float(np.median(power)) * 1e-4, 1e-12)
|
||||
power = 20.0 * np.log10(np.maximum(power, floor))
|
||||
return Waterfall(power=power, rate=float(rate),
|
||||
hz_per_bin=rate / size,
|
||||
seconds_per_step=step / rate)
|
||||
|
||||
|
||||
def _sync_offsets():
|
||||
"""Where the twenty-one sync tones sit, relative to a candidate."""
|
||||
steps, bins = [], []
|
||||
for block in SYNC_AT:
|
||||
for k, tone in enumerate(COSTAS):
|
||||
steps.append((block + k) * TIME_STEPS)
|
||||
bins.append(tone * FREQ_STEPS)
|
||||
return np.array(steps), np.array(bins)
|
||||
|
||||
|
||||
SYNC_STEP, SYNC_BIN = _sync_offsets()
|
||||
|
||||
|
||||
def candidates(fall: Waterfall, most: int = 300, lowest: float = 200.0,
|
||||
highest: float = 3000.0, earliest: float = -2.5,
|
||||
latest: float = 3.5) -> list[Candidate]:
|
||||
"""Every place a transmission plausibly starts, best first.
|
||||
|
||||
Scored as how far the twenty-one sync tones stand above the other seven
|
||||
tones at the same instants. Not above the noise floor of the band --
|
||||
that would rank a strong signal's neighbours above a weak signal, and
|
||||
the weak ones are the whole point of the protocol.
|
||||
|
||||
Only local peaks are kept. A real transmission lights up every
|
||||
neighbouring offset as well, and without this one loud station would
|
||||
fill the list and crowd out forty quiet ones.
|
||||
"""
|
||||
power = fall.power
|
||||
span = TONES * TIME_STEPS
|
||||
first = int(math.floor(earliest / fall.seconds_per_step))
|
||||
last = int(math.ceil(latest / fall.seconds_per_step))
|
||||
starts = np.arange(first, last + 1)
|
||||
starts = starts[(starts >= 0) | (starts + SYNC_STEP.min() >= 0)]
|
||||
starts = starts[starts + span <= power.shape[0]]
|
||||
starts = starts[starts >= 0]
|
||||
low = max(0, int(lowest / fall.hz_per_bin))
|
||||
high = min(power.shape[1] - 8 * FREQ_STEPS,
|
||||
int(highest / fall.hz_per_bin))
|
||||
if starts.size == 0 or high <= low:
|
||||
return []
|
||||
tops = np.arange(low, high)
|
||||
|
||||
# The sync tones, and all eight tones at the same instants, for every
|
||||
# (start, frequency) at once.
|
||||
times = starts[:, None] + SYNC_STEP[None, :] # (S, 21)
|
||||
here = power[times] # (S, 21, bins)
|
||||
want = tops[None, None, :] + SYNC_BIN[None, :, None]
|
||||
sync = np.take_along_axis(here, want, axis=2).mean(axis=1) # (S, tops)
|
||||
whole = np.zeros_like(sync)
|
||||
for tone in range(8):
|
||||
idx = tops[None, None, :] + tone * FREQ_STEPS
|
||||
whole += np.take_along_axis(here, idx, axis=2).mean(axis=1)
|
||||
score = sync - whole / 8.0
|
||||
|
||||
# Local peaks only, in both directions.
|
||||
keep = np.ones_like(score, dtype=bool)
|
||||
keep[1:, :] &= score[1:, :] >= score[:-1, :]
|
||||
keep[:-1, :] &= score[:-1, :] >= score[1:, :]
|
||||
keep[:, 1:] &= score[:, 1:] >= score[:, :-1]
|
||||
keep[:, :-1] &= score[:, :-1] >= score[:, 1:]
|
||||
rows, cols = np.nonzero(keep)
|
||||
values = score[rows, cols]
|
||||
order = np.argsort(values)[::-1][:most]
|
||||
return [Candidate(step=int(starts[rows[i]]), bin=int(tops[cols[i]]),
|
||||
score=float(values[i])) for i in order]
|
||||
|
||||
|
||||
def soft_bits(fall: Waterfall, spot: Candidate) -> np.ndarray:
|
||||
"""How strongly each of the hundred and seventy-four bits is a zero.
|
||||
|
||||
Every symbol gives eight tone powers. A bit is a zero in four of the
|
||||
eight tones and a one in the other four, so its evidence is the best of
|
||||
the four against the best of the other four -- in decibels, which is
|
||||
already a log-likelihood up to a scale factor, and the scale does not
|
||||
matter to the min-sum decoding downstream.
|
||||
|
||||
Taking only the loudest tone and calling that three bits, which is the
|
||||
obvious thing to do, throws away exactly the information the error
|
||||
correction runs on, and decodes almost nothing.
|
||||
"""
|
||||
power = fall.power
|
||||
steps = spot.step + np.arange(TONES) * TIME_STEPS
|
||||
bins = spot.bin + np.arange(8) * FREQ_STEPS
|
||||
if steps[-1] >= power.shape[0] or bins[-1] >= power.shape[1]:
|
||||
return np.zeros(code.BITS, dtype=np.float32)
|
||||
grid = power[np.ix_(steps, bins)] # (79, 8)
|
||||
|
||||
# Sync symbols carry nothing; drop them and put the tones back in the
|
||||
# order the bits were in before the Gray mapping.
|
||||
data = np.array([i for i in range(TONES)
|
||||
if not (i < 7 or 36 <= i < 43 or 72 <= i)])
|
||||
values = grid[data][:, list(code.GRAY)] # (58, 8) by value
|
||||
out = np.zeros(code.BITS, dtype=np.float32)
|
||||
for b in range(3):
|
||||
mask = (np.arange(8) >> (2 - b)) & 1
|
||||
zero = values[:, mask == 0].max(axis=1)
|
||||
one = values[:, mask == 1].max(axis=1)
|
||||
out[b::3] = zero - one
|
||||
return out
|
||||
|
||||
|
||||
# Worked out by measuring against transmissions of known strength and
|
||||
# against off-air recordings with published readings, not derived. Turning
|
||||
# a ratio of bins into the decibels-in-2500-Hz everybody quotes depends on
|
||||
# the window, the padding and the shape of the noise, and calibrating it is
|
||||
# both easier and more honest than deriving it and hoping.
|
||||
SNR_TRIM = -37.0
|
||||
|
||||
# Making the transmitter above label its signals the way the reading below
|
||||
# reports them. The reading is the one tied to reality -- it matches the
|
||||
# published figures for the off-air recordings to within half a decibel
|
||||
# across a hundred of them -- and a bare ratio of signal power to noise in
|
||||
# 2500 Hz comes out eight decibels away from it, consistently enough that
|
||||
# the difference is a definition rather than an error. Whose definition is
|
||||
# "right" is not a question this can settle; what matters is that a signal
|
||||
# built here at -15 reads as -15, so a test of sensitivity means what it
|
||||
# says.
|
||||
SNR_MATCHES = 10.0 ** (8.0 / 10.0)
|
||||
SNR_NEAR_HZ = 200.0 # how far either side to look for the noise
|
||||
SNR_PERCENTILE = 30 # low enough to ignore the neighbours
|
||||
|
||||
|
||||
def _snr(fall: Waterfall, spot: Candidate, tones) -> float:
|
||||
"""A signal-to-noise in the usual 2500 Hz reference bandwidth.
|
||||
|
||||
Measured in the tone that was actually sent, which is known by the time
|
||||
this is called because the message decoded. Taking the loudest of the
|
||||
eight instead looks like the same thing and is not: the largest of
|
||||
eight noisy numbers is well above their mean even when there is no
|
||||
signal at all, so weak transmissions all read as though they were
|
||||
stronger than they are, and the scale compresses where it matters.
|
||||
|
||||
The noise is measured beside the signal rather than across the band. A
|
||||
receiver with a three-kilohertz passband has nothing above it but the
|
||||
noise floor of the sound card, and a median taken over the whole
|
||||
spectrum sits twelve to sixteen decibels below the real one -- which is
|
||||
a receiver flattering every station it hears by that much.
|
||||
|
||||
Good to a few decibels, honestly: the readings here track the published
|
||||
ones for the same recordings with a correlation of about 0.83 and no
|
||||
systematic bias between -25 and +10, which is the range that matters.
|
||||
Very strong signals read a little low, as they do everywhere.
|
||||
"""
|
||||
power = (10.0 ** (fall.power / 20.0)) ** 2
|
||||
steps = spot.step + np.arange(TONES) * TIME_STEPS
|
||||
live = (steps >= 0) & (steps < power.shape[0])
|
||||
steps, sent = steps[live], np.asarray(tones)[live]
|
||||
bins = spot.bin + np.arange(8) * FREQ_STEPS
|
||||
if steps.size == 0 or bins[-1] >= power.shape[1]:
|
||||
return -99.0
|
||||
near = int(SNR_NEAR_HZ / fall.hz_per_bin)
|
||||
lo = max(0, spot.bin - near)
|
||||
high = min(power.shape[1], bins[-1] + near)
|
||||
columns = np.arange(lo, high)
|
||||
# A guard either side of the signal: its own skirts are not noise.
|
||||
beside = ((columns < spot.bin - 2 * FREQ_STEPS)
|
||||
| (columns > bins[-1] + 2 * FREQ_STEPS))
|
||||
if beside.sum() < 8:
|
||||
return -99.0
|
||||
noise = float(np.percentile(power[np.ix_(steps, columns[beside])],
|
||||
SNR_PERCENTILE))
|
||||
grid = power[np.ix_(steps, bins)]
|
||||
here = float(grid[np.arange(grid.shape[0]), sent].mean())
|
||||
if noise <= 0 or here <= noise:
|
||||
return -99.0
|
||||
return round(10.0 * math.log10((here - noise) / noise) + SNR_TRIM, 0)
|
||||
|
||||
|
||||
def listen_to(audio, rate: float, book=None, most: int = 300,
|
||||
rounds: int = 30, at: float = 0.0) -> list[Decode]:
|
||||
"""Everything decodable in one slot of audio.
|
||||
|
||||
Candidates are decoded in one batch rather than one at a time: a busy
|
||||
slot throws up two or three hundred of them, the error correction is
|
||||
most of the work, and doing them together is the difference between a
|
||||
slot that decodes in a second and one that does not keep up with the
|
||||
air.
|
||||
"""
|
||||
book = book if book is not None else code.CallBook()
|
||||
fall = waterfall(audio, rate)
|
||||
spots = candidates(fall, most=most)
|
||||
if not spots:
|
||||
return []
|
||||
|
||||
beliefs = np.stack([soft_bits(fall, spot) for spot in spots])
|
||||
repaired, worked = code.repair(beliefs, rounds=rounds)
|
||||
|
||||
out: list[Decode] = []
|
||||
seen: dict[str, Decode] = {}
|
||||
for i, spot in enumerate(spots):
|
||||
if not worked[i]:
|
||||
continue
|
||||
bits = repaired[i]
|
||||
if not code.crc_holds(bits[:code.PAYLOAD]):
|
||||
continue
|
||||
message = code.unpack(bits[:code.MESSAGE_BITS], book)
|
||||
if not message.text:
|
||||
continue
|
||||
found = Decode(text=message.text, kind=message.kind,
|
||||
calls=message.calls, grid=message.grid,
|
||||
report=message.report, calling=message.calling,
|
||||
hertz=round(spot.hertz(fall), 1),
|
||||
offset=round(spot.seconds(fall), 2),
|
||||
snr_db=_snr(fall, spot, code.tones_of(bits)),
|
||||
score=spot.score, at=at)
|
||||
# The same transmission is found at several neighbouring offsets.
|
||||
# Keep the one with the best sync, which is the one nearest the
|
||||
# truth about where and when it actually was.
|
||||
was = seen.get(found.text)
|
||||
if was is None or found.score > was.score:
|
||||
seen[found.text] = found
|
||||
out = sorted(seen.values(), key=lambda d: d.hertz)
|
||||
for found in out:
|
||||
for call in found.calls:
|
||||
book.remember(call)
|
||||
return out
|
||||
|
||||
|
||||
def transmit(text: str, rate: float = 12000.0, hertz: float = 1000.0,
|
||||
offset: float = 0.5, seconds: float = code.SLOT_S,
|
||||
snr_db: float | None = None, seed: int = 0,
|
||||
book=None) -> np.ndarray:
|
||||
"""A slot of audio with one transmission in it.
|
||||
|
||||
Here rather than in a test because the tests need it, the simulator
|
||||
needs it, and something that generates the signal this module claims to
|
||||
understand is the only way to ask a question of the decoder whose answer
|
||||
is known in advance -- where a transmission started, to the sample.
|
||||
|
||||
It does not transmit anything anywhere. It builds samples.
|
||||
"""
|
||||
payload = code.pack(text, book)
|
||||
tones = code.tones_of(code.encode(code.with_crc(payload)))
|
||||
per_symbol = int(round(rate * SYMBOL_S))
|
||||
total = int(round(rate * seconds))
|
||||
audio = np.zeros(total, dtype=np.float64)
|
||||
|
||||
# Continuous phase across the whole transmission. A tone that restarted
|
||||
# its phase every symbol would splatter across the band, which is both
|
||||
# unneighbourly on the air and not what a decoder should be tested with.
|
||||
phase = 0.0
|
||||
start = int(round(offset * rate))
|
||||
for i, tone in enumerate(tones):
|
||||
freq = hertz + float(tone) * code.SYMBOL_HZ
|
||||
at = start + i * per_symbol
|
||||
if at >= total or at + per_symbol <= 0:
|
||||
phase += 2 * math.pi * freq * per_symbol / rate
|
||||
continue
|
||||
step = 2 * math.pi * freq / rate
|
||||
angles = phase + step * np.arange(per_symbol)
|
||||
phase = angles[-1] + step
|
||||
first = max(0, at)
|
||||
last = min(total, at + per_symbol)
|
||||
audio[first:last] += np.sin(angles[first - at:last - at])
|
||||
|
||||
if snr_db is not None:
|
||||
# The usual reference: the signal's own power against the noise in
|
||||
# 2500 Hz, which is how every FT8 program quotes a report. The
|
||||
# noise added here is white across the whole sampled band, so the
|
||||
# part of it that lands in the reference 2500 Hz is that fraction
|
||||
# of the total -- which is the step it is easy to get wrong, and
|
||||
# getting it wrong makes a decoder look deaf when it is the test
|
||||
# signal that was quietly attenuated.
|
||||
rng = np.random.default_rng(seed)
|
||||
signal = float(np.mean(audio[audio != 0] ** 2)) if audio.any() else 1.0
|
||||
bandwidth = rate / 2.0
|
||||
whole = (signal / (10.0 ** (snr_db / 10.0))
|
||||
* (bandwidth / 2500.0) / SNR_MATCHES)
|
||||
audio = audio + rng.standard_normal(total) * math.sqrt(whole)
|
||||
return audio.astype(np.float32)
|
||||
|
|
@ -24,7 +24,8 @@ from .config import (DEFAULT_CONFIG_DIR, DEFAULT_CONFIG_PATH,
|
|||
from .ranges import RangeError, ScanRange, parse_frequency
|
||||
|
||||
__all__ = ["run_tui", "show_ranges", "settings_menu", "help_screen",
|
||||
"aircraft_menu", "weather_menu", "aprs_menu", "first_run_setup",
|
||||
"aircraft_menu", "weather_menu", "aprs_menu", "ft8_menu",
|
||||
"first_run_setup",
|
||||
"TUIAbort"]
|
||||
|
||||
_BACK = ("", "b", "back", "q", "quit", "x")
|
||||
|
|
@ -1676,6 +1677,9 @@ def _main_loop(console: Console, cfg: ScanConfig) -> ScanConfig | None:
|
|||
f" [cyan]7[/cyan] APRS (144 MHz packet) "
|
||||
f"[grey62]positions, weather and messages from amateurs"
|
||||
f"[/grey62]\n"
|
||||
f" [cyan]8[/cyan] FT8 "
|
||||
f"[grey62]fifteen-second slots, whole bands at once, mostly "
|
||||
f"under the noise[/grey62]\n"
|
||||
f" [cyan]h[/cyan] Help\n"
|
||||
f" [cyan]s[/cyan] [bold green]Start scanning[/bold green]\n"
|
||||
f" [cyan]q[/cyan] Quit\n")
|
||||
|
|
@ -1695,6 +1699,8 @@ def _main_loop(console: Console, cfg: ScanConfig) -> ScanConfig | None:
|
|||
weather_menu(console, cfg)
|
||||
elif choice == "7":
|
||||
aprs_menu(console, cfg)
|
||||
elif choice == "8":
|
||||
ft8_menu(console, cfg)
|
||||
elif choice in ("h", "?", "help"):
|
||||
help_screen(console)
|
||||
elif choice in ("s", "start", "go"):
|
||||
|
|
@ -1706,3 +1712,195 @@ def _main_loop(console: Console, cfg: ScanConfig) -> ScanConfig | None:
|
|||
return cfg
|
||||
elif choice in ("q", "quit", "exit"):
|
||||
return None
|
||||
|
||||
|
||||
_FT8_INTRO = (
|
||||
"[bold]FT8[/bold] is fifteen seconds of everybody at once. Every station "
|
||||
"on the band transmits in the same quarter-minute slots, on the same "
|
||||
"dial frequency, fifty hertz wide each, stacked across three kilohertz "
|
||||
"of audio — so one receiver parked on one frequency hears the whole "
|
||||
"band's worth of stations at the same time.\n\n"
|
||||
"Most of them arrive [bold]below the noise[/bold], and decode anyway: "
|
||||
"half of what is sent is error-correcting code, which is what buys a "
|
||||
"mode that works twenty decibels under what an operator can hear.\n\n"
|
||||
"Two things it needs. The [bold]clock[/bold] has to be right to a second "
|
||||
"or two, because the slots are quarter-minutes of UTC and every station "
|
||||
"on earth agrees about which one it is. And almost all the activity is "
|
||||
"on [bold]shortwave[/bold], which a plain receiver of this kind cannot "
|
||||
"reach without an upconverter or direct sampling — so the default here "
|
||||
"is the two-metre channel, which it can.\n\n"
|
||||
"Nothing here transmits. It listens, decodes and writes down."
|
||||
)
|
||||
|
||||
|
||||
def ft8_menu(console: Console, cfg: ScanConfig) -> None:
|
||||
"""Listen to FT8, without a command line."""
|
||||
from . import ft8
|
||||
|
||||
options = ft8.load_options()
|
||||
while True:
|
||||
_rule(console, "FT8")
|
||||
console.print(Panel(Text.from_markup(_FT8_INTRO),
|
||||
border_style="blue", padding=(0, 1)))
|
||||
_option_groups(console, options, ft8)
|
||||
logs = ft8.logs_in(cfg.output_dir)
|
||||
kept = "no logs yet" if not logs else \
|
||||
f"{len(logs)} log{'s' if len(logs) != 1 else ''}"
|
||||
console.print(
|
||||
f"\n [cyan]l[/cyan] [bold green]Listen[/bold green]"
|
||||
f" [grey62]{ft8.describe(options)}[/grey62]\n"
|
||||
# The band is the setting that decides whether anything is heard
|
||||
# at all, and whether the receiver can reach it, so it sits here
|
||||
# rather than a level down among the gain and the sample rate.
|
||||
f" [cyan]c[/cyan] Band / channel "
|
||||
f"[bold]{ft8.band_text(options)}[/bold]\n"
|
||||
f" [cyan]g[/cyan] Where you are "
|
||||
f"[grey62]{options.grid.upper() or 'not set — no distances'}"
|
||||
f"[/grey62]\n"
|
||||
f" [cyan]r[/cyan] Read a log back [grey62]{kept} in "
|
||||
f"{cfg.output_dir}[/grey62]\n"
|
||||
f" [cyan]N[/cyan] open group N "
|
||||
f"[grey62]or type part of an option's name to find it[/grey62]\n"
|
||||
f" [cyan]s[/cyan] Save these as default "
|
||||
f"[grey62]kept in {ft8.options_path()}[/grey62]\n"
|
||||
f" [cyan]d[/cyan] Reset them\n"
|
||||
f" [cyan]b[/cyan] Back\n")
|
||||
answer = _ask(console, " choice", "l").strip().lower()
|
||||
|
||||
if answer in _BACK:
|
||||
return
|
||||
if answer in ("l", "listen", "p"):
|
||||
_ft8_listen(console, cfg, options)
|
||||
elif answer in ("c", "channel", "band"):
|
||||
_ft8_band(console, options)
|
||||
elif answer in ("g", "grid", "where"):
|
||||
_ft8_grid(console, options)
|
||||
elif answer in ("r", "read", "m"):
|
||||
_ft8_read(console, cfg, options, logs)
|
||||
elif answer == "s":
|
||||
try:
|
||||
where = ft8.save_options(options)
|
||||
console.print(f" [green]saved to {where}[/green]")
|
||||
except OSError as exc:
|
||||
console.print(f" [red]could not save: {exc}[/red]")
|
||||
elif answer == "d":
|
||||
if _confirm(" reset every FT8 option"):
|
||||
options = ft8.Ft8Options()
|
||||
console.print(" [green]reset[/green]")
|
||||
elif answer.isdigit() and 1 <= int(answer) <= len(ft8.OPTION_GROUPS):
|
||||
_option_group_menu(console, options,
|
||||
ft8.OPTION_GROUPS[int(answer) - 1], ft8)
|
||||
elif answer.lstrip("?").strip().isdigit():
|
||||
_edit_option(console, options, answer, ft8)
|
||||
elif answer:
|
||||
found = _find_options(answer, ft8)
|
||||
if not found:
|
||||
console.print(f" [yellow]nothing matches {answer!r} — "
|
||||
f"enter a group number, or l, c, g, r, s, d "
|
||||
f"or b[/yellow]")
|
||||
elif len(found) == 1:
|
||||
_edit_option(console, options,
|
||||
str(ft8.OPTIONS.index(found[0]) + 1), ft8)
|
||||
else:
|
||||
_option_list(console, options, found, f"matching {answer!r}",
|
||||
ft8)
|
||||
_pick_option(console, options, ft8)
|
||||
|
||||
|
||||
def _ft8_listen(console: Console, cfg: ScanConfig, options) -> None:
|
||||
from . import ft8
|
||||
|
||||
errs = options.validate()
|
||||
if errs:
|
||||
for e in errs:
|
||||
console.print(f" [red]{e}[/red]")
|
||||
return
|
||||
try:
|
||||
ft8.listen(console, options, cfg.output_dir)
|
||||
except Exception as exc: # a menu must survive it
|
||||
console.print(f" [red]{exc}[/red]")
|
||||
|
||||
|
||||
def _ft8_band(console: Console, options) -> None:
|
||||
"""Choose the band, which sets the dial frequency with it.
|
||||
|
||||
Which band is also the question of whether the receiver can hear it at
|
||||
all, so the list says so rather than leaving somebody to find out by
|
||||
listening to silence for ten minutes.
|
||||
"""
|
||||
from . import ft8
|
||||
|
||||
console.print("\n [bold]Which band[/bold]\n")
|
||||
for i, (name, hz, needs) in enumerate(ft8.BANDS, 1):
|
||||
note = ("[yellow]needs an upconverter or direct sampling[/yellow]"
|
||||
if needs == "shortwave"
|
||||
else "[green]a plain receiver reaches this[/green]")
|
||||
here = " [green]<- now[/green]" if options.band == name else ""
|
||||
console.print(f" [cyan]{i:>2}[/cyan] {name:<5} "
|
||||
f"{hz / 1e6:>10.3f} MHz {note}{here}")
|
||||
console.print("\n [grey62]Almost all the activity is on shortwave, and "
|
||||
"20m is the busiest band on earth. Two metres is quiet by "
|
||||
"comparison and is what a plain dongle can reach.[/grey62]")
|
||||
answer = _ask(console, "\n band (number, name, or blank to keep)",
|
||||
"").strip().lower()
|
||||
if not answer:
|
||||
return
|
||||
if answer.isdigit() and 1 <= int(answer) <= len(ft8.BANDS):
|
||||
ft8.use_band(options, ft8.BANDS[int(answer) - 1][0])
|
||||
elif any(answer == name for name, _hz, _n in ft8.BANDS):
|
||||
ft8.use_band(options, answer)
|
||||
else:
|
||||
console.print(f" [yellow]no band called {answer!r}[/yellow]")
|
||||
return
|
||||
console.print(f" [green]{ft8.band_text(options)}[/green]")
|
||||
|
||||
|
||||
def _ft8_grid(console: Console, options) -> None:
|
||||
"""Set the grid square, which is what distances are measured from."""
|
||||
from . import ft8
|
||||
|
||||
console.print("\n [grey62]Your Maidenhead grid square: four characters "
|
||||
"like IO91 or FN31. Every station calling CQ says where it "
|
||||
"is this way, so with yours filled in the report can say "
|
||||
"how far each of them is and on what bearing — which on "
|
||||
"shortwave is the whole interest of the thing.[/grey62]")
|
||||
answer = _ask(console, "\n grid square (blank to clear)",
|
||||
options.grid).strip()
|
||||
if not answer:
|
||||
options.grid = ""
|
||||
console.print(" [green]cleared — no distances[/green]")
|
||||
return
|
||||
if ft8.grid_at(answer) is None:
|
||||
console.print(f" [yellow]{answer!r} is not a grid square[/yellow]")
|
||||
return
|
||||
options.grid = answer.upper()
|
||||
lat, lon = ft8.grid_at(options.grid)
|
||||
console.print(f" [green]{options.grid} — {lat:.2f},{lon:.2f}[/green]")
|
||||
|
||||
|
||||
def _ft8_read(console: Console, cfg: ScanConfig, options, logs) -> None:
|
||||
"""Read an evening's decodes back off the disk."""
|
||||
from . import ft8, ft8log
|
||||
|
||||
if not logs:
|
||||
console.print(" [yellow]no FT8 logs yet — listen first[/yellow]")
|
||||
return
|
||||
console.print("\n [bold]Which log[/bold]\n")
|
||||
for i, path in enumerate(logs[-20:], 1):
|
||||
console.print(f" [cyan]{i:>2}[/cyan] {path.name}")
|
||||
answer = _ask(console, "\n log (number, or blank to go back)",
|
||||
"").strip()
|
||||
if not answer.isdigit():
|
||||
return
|
||||
picked = logs[-20:][int(answer) - 1] if 1 <= int(answer) <= len(logs[-20:]) \
|
||||
else None
|
||||
if picked is None:
|
||||
return
|
||||
rows = ft8log.read_log(picked)
|
||||
console.print(f"\n [grey62]{len(rows)} decodes in {picked.name}"
|
||||
f"[/grey62]\n")
|
||||
for row in rows[:40]:
|
||||
console.print(f" {row['snr_db']:>4.0f} {row['offset']:>5.1f} "
|
||||
f"{row['hertz']:>7.1f} ~ {row['text']}")
|
||||
if len(rows) > 40:
|
||||
console.print(f" [grey62]…and {len(rows) - 40} more[/grey62]")
|
||||
|
|
|
|||
|
|
@ -1071,3 +1071,118 @@ class AprsDisplay:
|
|||
if not out.plain:
|
||||
out.append(station.kind, style="grey62")
|
||||
return out
|
||||
|
||||
|
||||
class Ft8Display:
|
||||
"""The last slot's decodes, above a table of who has been heard.
|
||||
|
||||
Two things at once because FT8 is two things at once. A slot is an
|
||||
event -- forty stations transmitted, here is what they said -- and the
|
||||
evening is an accumulation. A display that showed only the latest slot
|
||||
would throw away the band; one that showed only the running table would
|
||||
never show the thing that just happened.
|
||||
"""
|
||||
|
||||
def __init__(self, console: Console, hold: float = 3600.0,
|
||||
imperial: bool = False, grid: str = "", band: str = ""):
|
||||
self.console = console
|
||||
self.hold = hold
|
||||
self.imperial = imperial
|
||||
self.grid = grid
|
||||
self.band = band
|
||||
self.started = time.time()
|
||||
self.heard = None
|
||||
self.slot_at = 0.0
|
||||
|
||||
def update(self, heard, slot_at: float = 0.0) -> None:
|
||||
self.heard = heard
|
||||
if slot_at:
|
||||
self.slot_at = slot_at
|
||||
|
||||
def showing(self, now: float | None = None) -> list:
|
||||
if self.heard is None:
|
||||
return []
|
||||
now = time.time() if now is None else now
|
||||
return [s for s in self.heard.band.all()
|
||||
if now - s.last <= self.hold]
|
||||
|
||||
def render(self, now: float | None = None, width: int | None = None):
|
||||
now = time.time() if now is None else now
|
||||
parts = [self._header(now)]
|
||||
latest = list(getattr(self.heard, "latest", []) or [])
|
||||
if latest:
|
||||
parts.append(self._slot(latest))
|
||||
here = self.showing(now)
|
||||
if here:
|
||||
parts.append(self._table(here, now))
|
||||
elif not latest:
|
||||
parts.append(Panel(_one_line(
|
||||
"[grey62]nothing decoded yet — the first decodes arrive "
|
||||
"when the slot after next ends, up to thirty seconds away. "
|
||||
"If nothing comes at all, check the clock: FT8 needs it "
|
||||
"right to a second or two[/grey62]"),
|
||||
border_style="grey37", padding=(0, 1)))
|
||||
return Group(*parts)
|
||||
|
||||
def _header(self, now: float):
|
||||
heard = self.heard
|
||||
slots = getattr(heard, "slots", 0)
|
||||
decodes = getattr(heard, "decodes", 0)
|
||||
stations = len(getattr(heard, "band", None).stations) if heard else 0
|
||||
elapsed = max(1e-9, now - self.started)
|
||||
bits = [f"[bold]FT8[/bold] {self.band}",
|
||||
f"{slots} slots",
|
||||
f"{decodes} decodes",
|
||||
f"{stations} stations"]
|
||||
if slots:
|
||||
bits.append(f"{decodes / max(1, slots):.1f} a slot")
|
||||
bits.append(_dur(elapsed))
|
||||
if self.grid:
|
||||
bits.append(f"from {self.grid.upper()}")
|
||||
return Text.from_markup(" ".join(f"[grey62]{b}[/grey62]"
|
||||
if i else b
|
||||
for i, b in enumerate(bits)))
|
||||
|
||||
def _slot(self, latest):
|
||||
table = Table(box=None, header_style="bold", pad_edge=False,
|
||||
title=f"last slot — {len(latest)} decoded",
|
||||
title_justify="left", title_style="bold")
|
||||
table.add_column("snr", justify="right")
|
||||
table.add_column("dt", justify="right")
|
||||
table.add_column("hz", justify="right")
|
||||
table.add_column("message")
|
||||
for d in latest[:24]:
|
||||
table.add_row(f"{d.snr_db:.0f}", f"{d.offset:+.1f}",
|
||||
f"{d.hertz:.0f}",
|
||||
f"[bold]{d.text}[/bold]" if d.calling else d.text)
|
||||
return table
|
||||
|
||||
def _table(self, here, now: float):
|
||||
from .ft8 import grid_away
|
||||
|
||||
here = sorted(here, key=lambda s: (-s.last, s.call))
|
||||
table = Table(box=None, header_style="bold", pad_edge=False,
|
||||
title=f"heard so far — {len(here)} stations",
|
||||
title_justify="left", title_style="bold")
|
||||
table.add_column("station")
|
||||
table.add_column("grid")
|
||||
if self.grid:
|
||||
table.add_column("away", justify="right")
|
||||
table.add_column("n", justify="right")
|
||||
table.add_column("best", justify="right")
|
||||
table.add_column("hz", justify="right")
|
||||
table.add_column("last", justify="right")
|
||||
for s in here[:20]:
|
||||
row = [s.call, s.grid or "—"]
|
||||
if self.grid:
|
||||
away = grid_away(self.grid, s.grid) if s.grid else None
|
||||
if away is None:
|
||||
row.append("—")
|
||||
elif self.imperial:
|
||||
row.append(f"{away[0] * 0.621371:.0f} mi")
|
||||
else:
|
||||
row.append(f"{away[0]:.0f} km")
|
||||
row += [str(s.decodes), f"{s.best_snr:.0f}", f"{s.hertz:.0f}",
|
||||
f"{now - s.last:.0f}s"]
|
||||
table.add_row(*row)
|
||||
return table
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
.\" Generated by packaging/make-man.py -- do not edit by hand.
|
||||
.TH BANDSAUNTER 1 "2026-09-21" "bandsaunter 2026-09-21_03" "User Commands"
|
||||
.TH BANDSAUNTER 1 "2026-09-21" "bandsaunter 2026-09-21_04" "User Commands"
|
||||
.SH NAME
|
||||
bandsaunter \- scan, record and identify radio signals with an RTL-SDR
|
||||
.SH SYNOPSIS
|
||||
|
|
@ -2958,6 +2958,196 @@ Also write a map \[em] write the stations and their tracks for Google Earth.
|
|||
.br
|
||||
Setting name \fBkml\fR, default \fBno\fR.
|
||||
.PP
|
||||
.SH FT8
|
||||
Fifteen seconds of everybody at once. Every station on the band transmits in
|
||||
the same quarter-minute slots, on the same dial frequency, fifty hertz wide
|
||||
each, stacked across three kilohertz of audio \[em] so one receiver parked on
|
||||
one frequency hears the whole band's worth of stations at the same time, and
|
||||
hears most of them well below the noise.
|
||||
.PP
|
||||
Half of what is transmitted is error-correcting code, and that is the trick:
|
||||
it is what buys a mode that decodes twenty-odd decibels under what an operator
|
||||
can hear. A receiver that took the loudest tone of each symbol and hoped would
|
||||
decode almost nothing, which is why the tone detector reports how confident it
|
||||
is bit by bit rather than what it thinks it heard.
|
||||
.SS What it needs
|
||||
The clock has to be right to a second or two. The slots are quarter-minutes of
|
||||
UTC and every station on earth agrees about which one it is. A receiver a
|
||||
second out still decodes; one a slot out hears every transmission split across
|
||||
two captures and decodes none of them. This is the one failure that looks
|
||||
exactly like a dead band, so the display and the report both say so when
|
||||
nothing arrives.
|
||||
.PP
|
||||
Almost all the activity is on shortwave, which a plain receiver of this kind
|
||||
cannot reach. The default is therefore the two-metre channel at 144.174 MHz,
|
||||
which it can. All thirteen channels are in the list and the shortwave ones
|
||||
work through an upconverter or a receiver in direct sampling mode; the menu
|
||||
says which is which rather than leaving somebody to find out by listening to
|
||||
silence.
|
||||
.SS What comes out
|
||||
.BI \-\-grid " SQUARE"
|
||||
is what turns decodes into geography: every station calling CQ says where it
|
||||
is, so with your own square filled in each gets a distance and a bearing and
|
||||
the furthest heard is named. Three tables afterwards \[em] stations heard,
|
||||
calling CQ, and who was working whom.
|
||||
.PP
|
||||
.B \-\-adif
|
||||
writes the log again in the form every amateur logging program imports,
|
||||
marked as heard rather than worked. Nothing here transmits, so nothing here is
|
||||
a contact, and an ADIF that let a logging program treat these as worked would
|
||||
put claims into somebody's log that they cannot make.
|
||||
.SS How well it works
|
||||
Checked against eleven off-air recordings with published decodes, which is the
|
||||
only honest way to test a decoder: an encoder tested against its own decoder
|
||||
agrees with it about anything they are both wrong about. Ninety-seven of a
|
||||
hundred and fifty messages, with no false decodes; timing within a hundredth
|
||||
of a second, frequency within a hertz, signal reports within half a decibel on
|
||||
average. The third not decoded are the weakest in each slot: a mature decoder
|
||||
subtracts what it has decoded and looks again in the remainder, which is not
|
||||
built here.
|
||||
.SH FT8 OPTIONS
|
||||
Every option the FT8 side takes, in the five groups the menu shows them in.
|
||||
Each is a flag here and a line in the menu, and both come from one table in
|
||||
the program, so they cannot disagree.
|
||||
.SS Receiver
|
||||
.TP
|
||||
.B --device
|
||||
Receiver \[em] which receiver to use, when more than one is plugged in.
|
||||
.br
|
||||
Setting name \fBdevice\fR, default \fB0\fR.
|
||||
.br
|
||||
Accepts: at least 0.
|
||||
.TP
|
||||
.B --gain
|
||||
Gain \[em] tuner gain in dB, or automatic.
|
||||
.br
|
||||
Setting name \fBgain\fR, default \fBauto\fR.
|
||||
.TP
|
||||
.B --rate
|
||||
Sample rate \[em] how fast to sample; 96 kS/s is the least that holds the channel (Hz).
|
||||
.br
|
||||
Setting name \fBrate\fR, default \fB240 kHz\fR.
|
||||
.br
|
||||
Accepts: at least 96000.
|
||||
.TP
|
||||
.B --band
|
||||
Band \[em] which FT8 channel to listen on.
|
||||
.br
|
||||
Setting name \fBband\fR, default \fB2m\fR.
|
||||
.br
|
||||
Accepts: one of: 160m, 80m, 60m, 40m, 30m, 20m, 17m, 15m, 12m, 10m, 6m, 2m, 70cm.
|
||||
.TP
|
||||
.B --frequency --freq
|
||||
Dial frequency \[em] the dial frequency, if you want one the band list does not have (Hz).
|
||||
.br
|
||||
Setting name \fBfrequency\fR, default \fB144.174 MHz\fR.
|
||||
.br
|
||||
Accepts: at least 100000, at most 2e+09.
|
||||
.TP
|
||||
.B --simulate / --no-simulate
|
||||
Simulate \[em] invent a band instead of using a receiver.
|
||||
.br
|
||||
Setting name \fBsimulate\fR, default \fBno\fR.
|
||||
.PP
|
||||
.SS Listening
|
||||
.TP
|
||||
.B --seconds
|
||||
Listen for \[em] how long to listen; zero means until stopped (s).
|
||||
.br
|
||||
Setting name \fBseconds\fR, default \fBuntil stopped\fR.
|
||||
.br
|
||||
Accepts: at least 0.
|
||||
.TP
|
||||
.B --slots
|
||||
Or this many slots \[em] stop after this many fifteen-second slots; zero means no limit.
|
||||
.br
|
||||
Setting name \fBslots\fR, default \fBno limit\fR.
|
||||
.br
|
||||
Accepts: at least 0.
|
||||
.TP
|
||||
.B --log / --no-log
|
||||
Write a log \[em] keep every decode in a file.
|
||||
.br
|
||||
Setting name \fBlog\fR, default \fByes\fR.
|
||||
.TP
|
||||
.B --decodes-seen / --no-decodes-seen
|
||||
Print each decode \[em] a line per decode instead of a table that refreshes.
|
||||
.br
|
||||
Setting name \fBdecodes_seen\fR, default \fBno\fR.
|
||||
.TP
|
||||
.B --hold
|
||||
Keep on display for \[em] how long a station stays on the table after its last decode (s).
|
||||
.br
|
||||
Setting name \fBhold\fR, default \fB3600 s\fR.
|
||||
.br
|
||||
Accepts: at least 1.
|
||||
.PP
|
||||
.SS Decoding
|
||||
.TP
|
||||
.B --lowest
|
||||
Search from \[em] the lowest audio frequency to look for signals at (Hz).
|
||||
.br
|
||||
Setting name \fBlowest\fR, default \fB200 Hz\fR.
|
||||
.br
|
||||
Accepts: at least 0.
|
||||
.TP
|
||||
.B --highest
|
||||
Search to \[em] the highest audio frequency to look for signals at (Hz).
|
||||
.br
|
||||
Setting name \fBhighest\fR, default \fB3 kHz\fR.
|
||||
.br
|
||||
Accepts: at least 100.
|
||||
.TP
|
||||
.B --most
|
||||
Candidates per slot \[em] how many possible transmissions to try to decode each slot.
|
||||
.br
|
||||
Setting name \fBmost\fR, default \fB300\fR.
|
||||
.br
|
||||
Accepts: at least 1.
|
||||
.TP
|
||||
.B --rounds
|
||||
Repair passes \[em] how many passes of error correction to make before giving up.
|
||||
.br
|
||||
Setting name \fBrounds\fR, default \fB30\fR.
|
||||
.br
|
||||
Accepts: at least 1.
|
||||
.PP
|
||||
.SS Showing
|
||||
.TP
|
||||
.B --grid
|
||||
Aerial at \[em] your own grid square, so distances can be worked out.
|
||||
.br
|
||||
Setting name \fBgrid\fR, default \fBnot set, so no distances\fR.
|
||||
.TP
|
||||
.B --units
|
||||
Show readings in \[em] metric or imperial, for the display and the export.
|
||||
.br
|
||||
Setting name \fBunits\fR, default \fBmetric\fR.
|
||||
.br
|
||||
Accepts: one of: metric, imperial.
|
||||
.TP
|
||||
.B --calls-only / --no-calls-only
|
||||
Callsigns only \[em] leave out free text and telemetry.
|
||||
.br
|
||||
Setting name \fBcalls_only\fR, default \fBno\fR.
|
||||
.PP
|
||||
.SS Afterwards
|
||||
.TP
|
||||
.B --report / --no-report
|
||||
Report at the end \[em] print the tables when the listening stops.
|
||||
.br
|
||||
Setting name \fBreport\fR, default \fByes\fR.
|
||||
.TP
|
||||
.B --csv / --no-csv
|
||||
Also write a CSV \[em] a spreadsheet of every decode beside the log.
|
||||
.br
|
||||
Setting name \fBcsv\fR, default \fBno\fR.
|
||||
.TP
|
||||
.B --adif / --no-adif
|
||||
Also write an ADIF \[em] the log again, in the form logging programs read.
|
||||
.br
|
||||
Setting name \fBadif\fR, default \fBno\fR.
|
||||
.PP
|
||||
.SH FILES
|
||||
.TP
|
||||
.I ~/.config/bandsaunter/config.yaml
|
||||
|
|
@ -3090,6 +3280,16 @@ terms, in
|
|||
or at
|
||||
.UR https://www.gnu.org/licenses/
|
||||
.UE .
|
||||
.PP
|
||||
One file is not original work.
|
||||
.I ft8tables.py
|
||||
holds the two fixed tables that define the FT8 error-correcting code, taken
|
||||
from ft8_lib (https://github.com/kgoba/ft8_lib), MIT licensed, copyright 2018
|
||||
K\[u0101]rlis Goba, which took them in turn from WSJT\-X. They are reproduced
|
||||
under the MIT terms and that file carries the attribution they ask for. They
|
||||
are there because they cannot be derived: everything else about FT8 here is
|
||||
worked out from first principles, but those two tables are the code itself,
|
||||
chosen once by its designers and published. No decoding logic was taken.
|
||||
.SH BUGS
|
||||
The DVB-T television driver claims these dongles on sight. If the receiver
|
||||
cannot be opened, that is almost always why: the package blacklists the
|
||||
|
|
|
|||
|
|
@ -77,6 +77,13 @@ def aprs_section() -> list[str]:
|
|||
return options_section(ap)
|
||||
|
||||
|
||||
def ft8_section() -> list[str]:
|
||||
"""Every FT8 option, from the same table again."""
|
||||
from bandsaunter import ft8
|
||||
|
||||
return options_section(ft8)
|
||||
|
||||
|
||||
def options_section(air) -> list[str]:
|
||||
"""One section's options, written out from the table the program uses.
|
||||
|
||||
|
|
@ -1792,6 +1799,58 @@ Every option the APRS side takes, in the four groups the menu shows them in.
|
|||
Each is a flag here and a line in the menu, and both come from one table in
|
||||
the program, so they cannot disagree.
|
||||
.APRS_OPTIONS_HERE
|
||||
.SH FT8
|
||||
Fifteen seconds of everybody at once. Every station on the band transmits in
|
||||
the same quarter-minute slots, on the same dial frequency, fifty hertz wide
|
||||
each, stacked across three kilohertz of audio \[em] so one receiver parked on
|
||||
one frequency hears the whole band's worth of stations at the same time, and
|
||||
hears most of them well below the noise.
|
||||
.PP
|
||||
Half of what is transmitted is error-correcting code, and that is the trick:
|
||||
it is what buys a mode that decodes twenty-odd decibels under what an operator
|
||||
can hear. A receiver that took the loudest tone of each symbol and hoped would
|
||||
decode almost nothing, which is why the tone detector reports how confident it
|
||||
is bit by bit rather than what it thinks it heard.
|
||||
.SS What it needs
|
||||
The clock has to be right to a second or two. The slots are quarter-minutes of
|
||||
UTC and every station on earth agrees about which one it is. A receiver a
|
||||
second out still decodes; one a slot out hears every transmission split across
|
||||
two captures and decodes none of them. This is the one failure that looks
|
||||
exactly like a dead band, so the display and the report both say so when
|
||||
nothing arrives.
|
||||
.PP
|
||||
Almost all the activity is on shortwave, which a plain receiver of this kind
|
||||
cannot reach. The default is therefore the two-metre channel at 144.174 MHz,
|
||||
which it can. All thirteen channels are in the list and the shortwave ones
|
||||
work through an upconverter or a receiver in direct sampling mode; the menu
|
||||
says which is which rather than leaving somebody to find out by listening to
|
||||
silence.
|
||||
.SS What comes out
|
||||
.BI \-\-grid " SQUARE"
|
||||
is what turns decodes into geography: every station calling CQ says where it
|
||||
is, so with your own square filled in each gets a distance and a bearing and
|
||||
the furthest heard is named. Three tables afterwards \[em] stations heard,
|
||||
calling CQ, and who was working whom.
|
||||
.PP
|
||||
.B \-\-adif
|
||||
writes the log again in the form every amateur logging program imports,
|
||||
marked as heard rather than worked. Nothing here transmits, so nothing here is
|
||||
a contact, and an ADIF that let a logging program treat these as worked would
|
||||
put claims into somebody's log that they cannot make.
|
||||
.SS How well it works
|
||||
Checked against eleven off-air recordings with published decodes, which is the
|
||||
only honest way to test a decoder: an encoder tested against its own decoder
|
||||
agrees with it about anything they are both wrong about. Ninety-seven of a
|
||||
hundred and fifty messages, with no false decodes; timing within a hundredth
|
||||
of a second, frequency within a hertz, signal reports within half a decibel on
|
||||
average. The third not decoded are the weakest in each slot: a mature decoder
|
||||
subtracts what it has decoded and looks again in the remainder, which is not
|
||||
built here.
|
||||
.SH FT8 OPTIONS
|
||||
Every option the FT8 side takes, in the five groups the menu shows them in.
|
||||
Each is a flag here and a line in the menu, and both come from one table in
|
||||
the program, so they cannot disagree.
|
||||
.FT8_OPTIONS_HERE
|
||||
.SH FILES
|
||||
.TP
|
||||
.I ~/.config/bandsaunter/config.yaml
|
||||
|
|
@ -1924,6 +1983,16 @@ terms, in
|
|||
or at
|
||||
.UR https://www.gnu.org/licenses/
|
||||
.UE .
|
||||
.PP
|
||||
One file is not original work.
|
||||
.I ft8tables.py
|
||||
holds the two fixed tables that define the FT8 error-correcting code, taken
|
||||
from ft8_lib (https://github.com/kgoba/ft8_lib), MIT licensed, copyright 2018
|
||||
K\[u0101]rlis Goba, which took them in turn from WSJT\-X. They are reproduced
|
||||
under the MIT terms and that file carries the attribution they ask for. They
|
||||
are there because they cannot be derived: everything else about FT8 here is
|
||||
worked out from first principles, but those two tables are the code itself,
|
||||
chosen once by its designers and published. No decoding logic was taken.
|
||||
.SH BUGS
|
||||
The DVB-T television driver claims these dongles on sight. If the receiver
|
||||
cannot be opened, that is almost always why: the package blacklists the
|
||||
|
|
@ -1944,6 +2013,7 @@ def main() -> int:
|
|||
text = text.replace(".WEATHER_OPTIONS_HERE",
|
||||
"\n".join(weather_section()))
|
||||
text = text.replace(".APRS_OPTIONS_HERE", "\n".join(aprs_section()))
|
||||
text = text.replace(".FT8_OPTIONS_HERE", "\n".join(ft8_section()))
|
||||
text = text.replace("\n\n", "\n") # troff dislikes blank lines
|
||||
target = Path(sys.argv[1] if len(sys.argv) > 1
|
||||
else Path(__file__).parent / "bandsaunter.1")
|
||||
|
|
|
|||
839
tests/test_ft8.py
Normal file
839
tests/test_ft8.py
Normal file
|
|
@ -0,0 +1,839 @@
|
|||
"""The FT8 section: the code, the radio, the options and the front ends.
|
||||
|
||||
The important thing about this file is where its truth comes from. An
|
||||
encoder tested against its own decoder agrees with it about anything they
|
||||
are both wrong about -- this project learned that expensively on 433 MHz --
|
||||
so the decoding here is checked against signals that came off the air and
|
||||
were decoded by somebody else's program, not against anything written here.
|
||||
|
||||
Those recordings live outside the repository, and the tests that use them
|
||||
skip when they are absent. What does not skip is REAL: whole codewords
|
||||
lifted off the air, which pin the checksum, the parity and every field of
|
||||
the message format to reality in forty-four characters apiece.
|
||||
"""
|
||||
import math
|
||||
import time
|
||||
import wave
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from bandsaunter import ft8, ft8code as code, ft8log, ft8sim, ft8wave
|
||||
from bandsaunter import ft8tables as tables
|
||||
|
||||
|
||||
REFERENCE = Path("/mnt/global/bandsaunter/ft8ref")
|
||||
needs_recordings = pytest.mark.skipif(
|
||||
not REFERENCE.is_dir(), reason="the off-air recordings are not here")
|
||||
|
||||
|
||||
# Whole codewords, off the air, with what they say. A hundred and
|
||||
# seventy-four bits each: seventy-seven of message, fourteen of checksum and
|
||||
# eighty-three of parity, so one of these exercises the lot.
|
||||
REAL = [
|
||||
("B79A7EA67655CA92DB4D342FDD55B071779A631CFB4C",
|
||||
"PA3EPP SP8NFO KN09", "standard"),
|
||||
("0000002046E853111D48153FD733DA62C110A5E053FC",
|
||||
"CQ F4FSY JN25", "standard"),
|
||||
("E20AFC7590F9DA9FA7C9EEEA8A1A96D06F6855C82B58",
|
||||
"VK4BLE OH1EDK -20", "standard"),
|
||||
("70D84EAC55E5FB9FA70CD7148B9180DD1CD36A7EFC6C",
|
||||
"ET3RFG/R IN3ADG -23", "standard"),
|
||||
("1E6191F629601212F10D6992973A9B5AEA0596877498",
|
||||
"2M0OGG RA6ABO KN96", "standard"),
|
||||
("70AFEC9338AD521FA50DE077F5C94CAD362B2CC3A364",
|
||||
"ES5GI DD3SF 73", "standard"),
|
||||
("00000024519BF311248987ED8D2C4AA596CE262E1E68",
|
||||
"CQ IK4LZH JN54", "standard"),
|
||||
("4B9ACCF4519BF31FAA4F0C50A7B6CEA7F9C51BED8E80",
|
||||
"9A9TT IK4LZH -10", "standard"),
|
||||
("0B13C7046826781FA50F539DFE6D9FAF63F7C21EC268",
|
||||
"R2EA IZ4OUL 73", "standard"),
|
||||
("CBEC04F45AC9D49FA509770E71FEDE65EE2EFE66E8A8",
|
||||
"SA5QED IQ5PJ 73", "standard"),
|
||||
]
|
||||
|
||||
|
||||
def bits_of_hex(text: str) -> np.ndarray:
|
||||
raw = np.unpackbits(np.frombuffer(bytes.fromhex(text), dtype=np.uint8))
|
||||
return raw[:code.BITS].astype(np.uint8)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The two tables the code is defined by
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_the_generator_and_the_parity_matrix_describe_the_same_code():
|
||||
"""The two published tables are not derivable from one another -- the
|
||||
generator's parity half runs to fifty-odd bits a row and the sparse one
|
||||
to six or seven -- so that they agree is checked rather than assumed.
|
||||
|
||||
If they disagreed, every codeword this builds would fail every check
|
||||
and nothing would ever decode, which is a long way to find out."""
|
||||
rng = np.random.default_rng(11)
|
||||
for _ in range(50):
|
||||
message = rng.integers(0, 2, code.PAYLOAD).astype(np.uint8)
|
||||
assert code.parity_holds(code.encode(message))
|
||||
|
||||
|
||||
def test_every_codeword_bit_is_checked_three_times():
|
||||
"""A regular column weight, which is what the message passing assumes
|
||||
when it gathers three slots per bit instead of scattering."""
|
||||
assert code.SLOTS_OF_BIT.shape == (code.BITS, 3)
|
||||
assert sorted({len(c) for c in tables.CHECKS}) == [6, 7]
|
||||
|
||||
|
||||
def test_a_broken_codeword_fails_its_parity():
|
||||
rng = np.random.default_rng(3)
|
||||
word = code.encode(rng.integers(0, 2, code.PAYLOAD).astype(np.uint8))
|
||||
for flip in (0, 40, 90, 173):
|
||||
bad = word.copy()
|
||||
bad[flip] ^= 1
|
||||
assert not code.parity_holds(bad)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Reality
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("packed,text,kind", REAL)
|
||||
def test_a_codeword_off_the_air_says_what_it_said(packed, text, kind):
|
||||
"""The whole chain against something nobody here made up."""
|
||||
word = bits_of_hex(packed)
|
||||
assert code.parity_holds(word), "the parity of a real transmission"
|
||||
assert code.crc_holds(word[:code.PAYLOAD]), "the checksum of a real one"
|
||||
message = code.unpack(word[:code.MESSAGE_BITS])
|
||||
assert message.text == text
|
||||
assert message.kind == kind
|
||||
|
||||
|
||||
def test_a_real_codeword_is_repaired_from_a_damaged_copy():
|
||||
"""What the error correction is for, on a real transmission rather than
|
||||
a made-up one.
|
||||
|
||||
Ten of the hundred and seventy-four bits, each of them *confidently*
|
||||
wrong, which is the hard case: a bit the detector is unsure about costs
|
||||
the decoding far less than one it is certain about and mistaken. Real
|
||||
errors arrive uncertain, which is why the same code reads signals off
|
||||
the air with far more than ten bits astray."""
|
||||
word = bits_of_hex(REAL[0][0])
|
||||
rng = np.random.default_rng(5)
|
||||
belief = np.where(word == 1, -4.0, 4.0).astype(np.float32)
|
||||
for flip in rng.choice(code.BITS, 10, replace=False):
|
||||
belief[flip] = -belief[flip]
|
||||
got = code.repair(belief)
|
||||
assert got is not None and (got == word).all()
|
||||
assert code.unpack(got[:code.MESSAGE_BITS]).text == REAL[0][1]
|
||||
|
||||
|
||||
def test_noise_does_not_decode_into_a_message():
|
||||
"""The checksum's job. The error correction will converge on *a*
|
||||
codeword given enough noise, and the fourteen bits are what stop that
|
||||
becoming a line in somebody's log."""
|
||||
rng = np.random.default_rng(17)
|
||||
lied = 0
|
||||
for _ in range(200):
|
||||
belief = rng.standard_normal(code.BITS).astype(np.float32) * 2.0
|
||||
got = code.repair(belief)
|
||||
if got is not None and code.crc_holds(got[:code.PAYLOAD]):
|
||||
lied += 1
|
||||
# One in sixteen thousand gets through by arithmetic; two hundred tries
|
||||
# should see none, and seeing several would mean the CRC is not being
|
||||
# checked at all.
|
||||
assert lied == 0, f"{lied} runs of noise passed the checksum"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The checksum and the coding
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_the_checksum_covers_the_message_zero_extended():
|
||||
"""Eighty-two bits, not seventy-seven -- which is not the same answer
|
||||
and is the sort of thing that decodes nothing at all."""
|
||||
payload = np.zeros(code.MESSAGE_BITS, dtype=np.uint8)
|
||||
payload[3] = payload[70] = 1
|
||||
block = code.with_crc(payload)
|
||||
assert block.size == code.PAYLOAD
|
||||
assert code.crc_holds(block)
|
||||
broken = block.copy()
|
||||
broken[10] ^= 1
|
||||
assert not code.crc_holds(broken)
|
||||
|
||||
|
||||
def test_the_tones_carry_the_bits_and_the_sync_is_where_it_should_be():
|
||||
rng = np.random.default_rng(2)
|
||||
word = code.encode(rng.integers(0, 2, code.PAYLOAD).astype(np.uint8))
|
||||
tones = code.tones_of(word)
|
||||
assert tones.size == code.TONES
|
||||
assert (code.bits_of(tones) == word).all()
|
||||
for start in (0, 36, 72):
|
||||
assert list(tones[start:start + 7]) == list(tables.COSTAS)
|
||||
|
||||
|
||||
def test_a_tone_mistaken_for_its_neighbour_costs_one_bit():
|
||||
"""Which is the whole reason for the Gray mapping, and is worth a test
|
||||
because getting the map backwards still round-trips perfectly."""
|
||||
for tone in range(7):
|
||||
a = code.UNGRAY[tone]
|
||||
b = code.UNGRAY[tone + 1]
|
||||
assert bin(int(a) ^ int(b)).count("1") == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Messages
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("text", [
|
||||
"CQ DL1UDO JO31", "CQ DX Z33Z KN11", "CQ JA OH1LWZ KP11",
|
||||
"VK4BLE OH8JK R-17", "JR5MJS OH8NW 73", "LZ1CWK DC8VA RR73",
|
||||
"G4CUS SP4FCA +10", "2M0OGG RA6ABO KN96", "YO7CGS A41ZZ -11",
|
||||
"R2EA IZ4OUL R-08", "W2XYZ K1ABC RRR",
|
||||
])
|
||||
def test_a_standard_message_survives_the_whole_chain(text):
|
||||
payload = code.pack(text)
|
||||
word = code.encode(code.with_crc(payload))
|
||||
assert code.parity_holds(word)
|
||||
back = code.bits_of(code.tones_of(word))
|
||||
assert (back == word).all()
|
||||
got = code.repair(np.where(word == 1, -4.0, 4.0).astype(np.float32))
|
||||
assert got is not None and (got == word).all()
|
||||
assert code.unpack(got[:code.MESSAGE_BITS]).text == text
|
||||
|
||||
|
||||
def test_the_digit_of_a_callsign_is_found_rather_than_guessed_at():
|
||||
"""Z33Z has a digit in the second place that belongs to the prefix, so
|
||||
a packer that assumes the second character is the separating digit
|
||||
turns it into a hash and loses the callsign."""
|
||||
assert code.is_standard_call("Z33Z")
|
||||
assert code.is_standard_call("K1ABC")
|
||||
assert code.is_standard_call("DL1UDO")
|
||||
assert code.is_standard_call("2M0OGG")
|
||||
assert not code.is_standard_call("ET3RFG/R")
|
||||
assert not code.is_standard_call("")
|
||||
|
||||
|
||||
def test_free_text_is_not_dressed_up_as_two_callsigns():
|
||||
"""A packer that falls back to hashing any word it does not recognise
|
||||
always succeeds, and would send HELLO WORLD as two hashed callsigns."""
|
||||
got = code.unpack(code.pack("HELLO WORLD"))
|
||||
assert got.kind == "free text"
|
||||
assert got.text == "HELLO WORLD"
|
||||
|
||||
|
||||
def test_free_text_is_thirteen_characters_and_says_so_by_truncating():
|
||||
got = code.unpack(code.pack("TNX FER QSO 73"))
|
||||
assert got.kind == "free text"
|
||||
assert len(got.text) <= 13
|
||||
|
||||
|
||||
@pytest.mark.parametrize("text,grid,report,calling", [
|
||||
("CQ DL1UDO JO31", "JO31", "", True),
|
||||
("VK4BLE OH8JK R-17", "", "R-17", False),
|
||||
("JR5MJS OH8NW 73", "", "73", False),
|
||||
("W2XYZ K1ABC RRR", "", "RRR", False),
|
||||
])
|
||||
def test_what_can_be_picked_out_of_a_message(text, grid, report, calling):
|
||||
got = code.unpack(code.pack(text))
|
||||
assert got.grid == grid and got.report == report
|
||||
assert got.calling is calling
|
||||
|
||||
|
||||
def test_a_hashed_callsign_stays_unknown_until_it_has_been_heard_in_full():
|
||||
"""A wrong callsign in a log is worse than a missing one, so a hash
|
||||
that has not been heard spelled out stays <...> rather than guessed."""
|
||||
book = code.CallBook()
|
||||
assert book.look_up(code.hash_of("GM4ABC", 22), 22) == "<...>"
|
||||
book.remember("GM4ABC")
|
||||
assert book.look_up(code.hash_of("GM4ABC", 22), 22) == "GM4ABC"
|
||||
assert book.look_up(code.hash_of("GM4ABD", 22), 22) == "<...>"
|
||||
|
||||
|
||||
def test_a_callsign_too_short_to_mean_anything_is_not_remembered():
|
||||
book = code.CallBook()
|
||||
for rubbish in ("", "A", "CQ", "DE", "QRZ"):
|
||||
book.remember(rubbish)
|
||||
assert book.by_hash == {}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The radio
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_a_transmission_is_found_where_it_was_put():
|
||||
"""The decoder's own clock and dial, against a signal whose time and
|
||||
frequency are known to the sample."""
|
||||
for hertz in (500.0, 1200.0, 2400.0):
|
||||
for offset in (0.2, 0.5, 1.4):
|
||||
audio = ft8wave.transmit("CQ DL1UDO JO31", hertz=hertz,
|
||||
offset=offset)
|
||||
got = ft8wave.listen_to(audio, 12000.0)
|
||||
assert len(got) == 1, (hertz, offset)
|
||||
assert got[0].text == "CQ DL1UDO JO31"
|
||||
assert abs(got[0].hertz - hertz) < 3.5
|
||||
# Reported against the nominal start of sending, half a second
|
||||
# into the slot, which is what every FT8 program reports.
|
||||
assert abs(got[0].offset - (offset - 0.5)) < 0.12
|
||||
|
||||
|
||||
def test_the_nominal_start_is_half_a_second_into_the_slot():
|
||||
"""A station whose clock is right reports as dt 0.0, not dt 0.5: the
|
||||
transmission is 12.64 seconds long in a slot of fifteen and everybody
|
||||
begins half a second in."""
|
||||
audio = ft8wave.transmit("CQ DL1UDO JO31", offset=ft8wave.NOMINAL_START)
|
||||
got = ft8wave.listen_to(audio, 12000.0)
|
||||
assert got and abs(got[0].offset) < 0.1
|
||||
|
||||
|
||||
def test_several_transmissions_in_one_slot_all_come_out():
|
||||
"""Which is the whole point of the mode: they overlap in time entirely
|
||||
and are told apart by frequency alone."""
|
||||
said = [("CQ DL1UDO JO31", 500.0), ("VK4BLE OH8JK R-17", 1000.0),
|
||||
("JR5MJS OH8NW 73", 1600.0), ("CQ DX Z33Z KN11", 2300.0)]
|
||||
audio = np.zeros(int(12000 * code.SLOT_S), dtype=np.float32)
|
||||
for text, hertz in said:
|
||||
audio += ft8wave.transmit(text, hertz=hertz, offset=0.5)
|
||||
got = {d.text for d in ft8wave.listen_to(audio, 12000.0)}
|
||||
assert got == {t for t, _hz in said}
|
||||
|
||||
|
||||
def test_a_slot_of_noise_decodes_to_nothing():
|
||||
rng = np.random.default_rng(9)
|
||||
noise = rng.standard_normal(int(12000 * code.SLOT_S)).astype(np.float32)
|
||||
assert ft8wave.listen_to(noise, 12000.0) == []
|
||||
|
||||
|
||||
def test_it_hears_a_signal_well_below_the_noise():
|
||||
"""The claim the mode is built on. Fifteen decibels under is a signal
|
||||
an operator hears nothing of at all."""
|
||||
heard = 0
|
||||
for seed in range(5):
|
||||
audio = ft8wave.transmit("CQ DL1UDO JO31", hertz=1200.0, offset=0.5,
|
||||
snr_db=-15.0, seed=seed)
|
||||
got = ft8wave.listen_to(audio, 12000.0)
|
||||
heard += any(d.text == "CQ DL1UDO JO31" for d in got)
|
||||
assert heard == 5
|
||||
|
||||
|
||||
def test_the_signal_report_tracks_the_signal():
|
||||
"""Not to a decibel -- it is an estimate -- but a stronger signal has
|
||||
to read stronger, or the number is worse than not printing one."""
|
||||
readings = []
|
||||
for want in (-15.0, -5.0, 5.0):
|
||||
audio = ft8wave.transmit("CQ DL1UDO JO31", hertz=1200.0, offset=0.5,
|
||||
snr_db=want, seed=1)
|
||||
got = ft8wave.listen_to(audio, 12000.0)
|
||||
assert got
|
||||
readings.append(got[0].snr_db)
|
||||
assert readings[0] < readings[1] < readings[2]
|
||||
assert all(abs(r - w) < 8.0
|
||||
for r, w in zip(readings, (-15.0, -5.0, 5.0)))
|
||||
|
||||
|
||||
def test_the_waterfall_is_half_a_symbol_and_half_a_tone():
|
||||
"""A transmission a quarter of a symbol late, or a quarter of a tone
|
||||
off, still has to be caught."""
|
||||
fall = ft8wave.waterfall(np.zeros(12000 * 15, dtype=np.float32), 12000.0)
|
||||
assert fall.hz_per_bin == pytest.approx(code.SYMBOL_HZ
|
||||
/ ft8wave.FREQ_STEPS)
|
||||
assert fall.seconds_per_step == pytest.approx(code.SYMBOL_S
|
||||
/ ft8wave.TIME_STEPS)
|
||||
|
||||
|
||||
def test_audio_too_short_to_hold_a_symbol_is_refused_rather_than_guessed():
|
||||
with pytest.raises(ValueError):
|
||||
ft8wave.waterfall(np.zeros(10, dtype=np.float32), 12000.0)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Against the air
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _recording(name):
|
||||
with wave.open(str(REFERENCE / f"{name}.wav")) as handle:
|
||||
raw = handle.readframes(handle.getnframes())
|
||||
rate = float(handle.getframerate())
|
||||
audio = np.frombuffer(raw, dtype=np.int16).astype(np.float32) / 32768.0
|
||||
want = {}
|
||||
for line in (REFERENCE / f"{name}.txt").read_text().splitlines():
|
||||
parts = line.split(None, 4)
|
||||
if len(parts) >= 5:
|
||||
want[" ".join(parts[4].lstrip("~ ").split())] = (
|
||||
float(parts[1]), float(parts[2]), float(parts[3]))
|
||||
return audio, rate, want
|
||||
|
||||
|
||||
@needs_recordings
|
||||
@pytest.mark.parametrize("name,least", [
|
||||
("191111_110145", 2), ("websdr_test3", 7), ("20m_busy_test_02", 14),
|
||||
])
|
||||
def test_it_decodes_signals_that_came_off_the_air(name, least):
|
||||
"""The only test here whose answers were not written by this program."""
|
||||
audio, rate, want = _recording(name)
|
||||
got = {d.text: d for d in ft8wave.listen_to(audio, rate)}
|
||||
matched = [w for w in want if any(w.startswith(t) for t in got)]
|
||||
assert len(matched) >= least, f"decoded {len(matched)} of {len(want)}"
|
||||
# And nothing it decoded may be something nobody else heard: a false
|
||||
# line in a log is worse than a missing one.
|
||||
unknown = [t for t in got if not any(w.startswith(t) for w in want)]
|
||||
assert len(unknown) <= 1, unknown
|
||||
|
||||
|
||||
@needs_recordings
|
||||
def test_the_time_and_frequency_agree_with_what_else_heard_them():
|
||||
audio, rate, want = _recording("websdr_test3")
|
||||
errors = []
|
||||
for d in ft8wave.listen_to(audio, rate):
|
||||
for text, (snr, dt, hz) in want.items():
|
||||
if text.startswith(d.text):
|
||||
errors.append((d.offset - dt, d.hertz - hz))
|
||||
assert len(errors) >= 6
|
||||
assert max(abs(dt) for dt, _hz in errors) < 0.2
|
||||
assert max(abs(hz) for _dt, hz in errors) < 4.0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Slots and grids
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_slots_are_quarter_minutes_of_utc():
|
||||
"""Not of the local clock and not of when this program started: the
|
||||
protocol is built on every station on earth agreeing which one it is."""
|
||||
assert ft8.slot_start(1_700_000_007.0) == 1_700_000_000.0 - \
|
||||
(1_700_000_000.0 % 15)
|
||||
for when in (0.0, 1_700_000_000.0, 1_700_000_014.9):
|
||||
assert ft8.slot_start(when) % ft8.SLOT == 0
|
||||
assert 0 <= when - ft8.slot_start(when) < ft8.SLOT
|
||||
assert ft8.next_slot(when) == ft8.slot_start(when) + ft8.SLOT
|
||||
|
||||
|
||||
@pytest.mark.parametrize("grid,lat,lon", [
|
||||
("IO91", 51.5, -1.0), ("FN31", 41.5, -73.0), ("JN54", 44.5, 11.0),
|
||||
("RE78", -41.5, 175.0),
|
||||
])
|
||||
def test_a_grid_square_is_where_it_says(grid, lat, lon):
|
||||
got = ft8.grid_at(grid)
|
||||
assert got == pytest.approx((lat, lon), abs=0.01)
|
||||
|
||||
|
||||
def test_a_grid_that_is_not_one_is_refused():
|
||||
for rubbish in ("", "IO", "9191", "ZZ99", "IO9", "ABCD"):
|
||||
assert ft8.grid_at(rubbish) is None
|
||||
|
||||
|
||||
def test_the_distance_between_two_squares_is_the_great_circle_one():
|
||||
km, bearing = ft8.grid_away("IO91", "FN31")
|
||||
assert km == pytest.approx(5390, abs=60)
|
||||
assert bearing == pytest.approx(288, abs=3)
|
||||
assert ft8.grid_away("IO91", "IO91")[0] < 1.0
|
||||
assert ft8.grid_away("IO91", "") is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The options
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_every_option_names_a_field_that_exists():
|
||||
fields = set(ft8.Ft8Options().__dict__)
|
||||
for option in ft8.OPTIONS:
|
||||
assert option.key in fields, f"{option.key} is not an option"
|
||||
|
||||
|
||||
def test_every_field_is_an_option_somebody_can_reach():
|
||||
assert set(ft8.Ft8Options().__dict__) == {o.key for o in ft8.OPTIONS}
|
||||
|
||||
|
||||
def test_every_option_is_in_a_group_and_says_what_it_does():
|
||||
for option in ft8.OPTIONS:
|
||||
assert option.group in ft8.OPTION_GROUPS
|
||||
assert option.help and option.detail and option.flags
|
||||
if option.kind == "bool":
|
||||
assert option.off_flags, f"{option.key} cannot be turned off"
|
||||
for group in ft8.OPTION_GROUPS:
|
||||
assert ft8.in_group(group)
|
||||
|
||||
|
||||
def test_every_option_flag_is_one_the_parser_accepts():
|
||||
from bandsaunter.cli import build_parser
|
||||
|
||||
known = set()
|
||||
for action in build_parser()._subparsers._group_actions[0].choices[
|
||||
"ft8"]._actions:
|
||||
known.update(action.option_strings)
|
||||
for option in ft8.OPTIONS:
|
||||
for flag in option.flags + option.off_flags:
|
||||
assert flag in known, f"{flag} is in the table and not the parser"
|
||||
|
||||
|
||||
def test_every_command_line_option_is_reachable_from_the_menu():
|
||||
"""Asked for explicitly, and worth a test rather than a promise: the
|
||||
menu builds itself from the groups, so an option in no group would be
|
||||
settable from the command line and invisible in the menu."""
|
||||
from bandsaunter import tui
|
||||
|
||||
text = Path(tui.__file__).read_text()
|
||||
assert "def ft8_menu(" in text
|
||||
assert "ft8_menu(console, cfg)" in text, "not reachable from the front page"
|
||||
reachable = set()
|
||||
for group in ft8.OPTION_GROUPS:
|
||||
reachable |= {o.key for o in ft8.in_group(group)}
|
||||
assert reachable == {o.key for o in ft8.OPTIONS}
|
||||
|
||||
|
||||
def test_the_options_survive_being_saved_and_read_back(tmp_path):
|
||||
options = ft8.Ft8Options(band="20m", frequency=14_074_000.0,
|
||||
grid="IO91", units="imperial", slots=12)
|
||||
ft8.save_options(options, tmp_path)
|
||||
assert ft8.load_options(tmp_path) == options
|
||||
|
||||
|
||||
def test_a_settings_file_with_a_mistake_in_it_costs_the_defaults(tmp_path):
|
||||
(tmp_path / "ft8.yaml").write_text("band: [not a band\n")
|
||||
assert ft8.load_options(tmp_path) == ft8.Ft8Options()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("options,broken", [
|
||||
(ft8.Ft8Options(rate=48_000.0), "sample rate"),
|
||||
(ft8.Ft8Options(seconds=-1.0), "negative"),
|
||||
(ft8.Ft8Options(hold=0.0), "display"),
|
||||
(ft8.Ft8Options(frequency=10.0), "frequency"),
|
||||
(ft8.Ft8Options(lowest=2000.0, highest=500.0), "above the bottom"),
|
||||
(ft8.Ft8Options(lowest=1000.0, highest=1050.0), "narrower"),
|
||||
(ft8.Ft8Options(most=0), "candidate"),
|
||||
(ft8.Ft8Options(rounds=0), "error correction"),
|
||||
(ft8.Ft8Options(grid="nowhere"), "grid square"),
|
||||
])
|
||||
def test_a_setting_that_cannot_work_is_refused(options, broken):
|
||||
assert any(broken in err for err in options.validate())
|
||||
|
||||
|
||||
def test_the_defaults_are_all_workable():
|
||||
assert ft8.Ft8Options().validate() == []
|
||||
|
||||
|
||||
def test_choosing_a_band_sets_the_frequency_with_it():
|
||||
options = ft8.defaults()
|
||||
ft8.use_band(options, "20m")
|
||||
assert options.frequency == 14_074_000.0 and options.band == "20m"
|
||||
ft8.use_band(options, "nonsense")
|
||||
assert options.frequency == 144_174_000.0
|
||||
|
||||
|
||||
def test_the_band_list_says_which_ones_a_plain_receiver_can_reach():
|
||||
"""Almost all the activity is on shortwave, which this kind of receiver
|
||||
cannot hear without help, and finding that out by listening to silence
|
||||
for ten minutes is the wrong way to learn it."""
|
||||
assert "upconverter" in ft8.band_text(ft8.Ft8Options(band="20m"))
|
||||
assert "upconverter" not in ft8.band_text(ft8.Ft8Options(band="2m"))
|
||||
assert ft8.defaults().band == "2m"
|
||||
|
||||
|
||||
def test_the_command_line_beats_the_saved_settings(tmp_path, monkeypatch):
|
||||
from bandsaunter.cli import build_parser
|
||||
|
||||
ft8.save_options(ft8.Ft8Options(band="2m", grid="AA00"), tmp_path)
|
||||
monkeypatch.setattr(ft8, "options_path",
|
||||
lambda directory=None: tmp_path / "ft8.yaml")
|
||||
args = build_parser().parse_args(["ft8", "--band", "20m",
|
||||
"--grid", "IO91"])
|
||||
assert args.band == "20m" and args.grid == "IO91"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# End to end
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class _Clock:
|
||||
"""A clock the test winds, so a run of slots takes no real time."""
|
||||
|
||||
def __init__(self, at=1_700_000_000.0):
|
||||
self.at = at
|
||||
|
||||
def now(self):
|
||||
return self.at
|
||||
|
||||
def sleep(self, seconds):
|
||||
self.at += seconds
|
||||
|
||||
|
||||
def _run(slots=2, **kw):
|
||||
clock = _Clock()
|
||||
band = ft8sim.SimulatedBand(rate=240_000.0, realtime=True, stations=12,
|
||||
clock=clock.now, sleep=clock.sleep)
|
||||
options = ft8.defaults()
|
||||
options.slots = slots
|
||||
for key, value in kw.items():
|
||||
setattr(options, key, value)
|
||||
heard = ft8.Heard()
|
||||
heard.started = clock.now()
|
||||
total = ft8.pump(band, options, heard, None, clock.now(),
|
||||
clock=clock.now)
|
||||
return heard, total
|
||||
|
||||
|
||||
def test_a_made_up_band_comes_out_the_other_end():
|
||||
"""The whole path: samples, sideband, slot boundaries, sync, decoding,
|
||||
and into the book of stations."""
|
||||
heard, total = _run(slots=2)
|
||||
assert heard.slots == 2
|
||||
assert total >= 8, f"only {total} decodes in two slots"
|
||||
assert len(heard.band.stations) >= 6
|
||||
for station in heard.band.all():
|
||||
assert station.decodes >= 1
|
||||
assert station.call and station.call != "<...>"
|
||||
|
||||
|
||||
def test_the_slot_boundaries_are_found_rather_than_assumed():
|
||||
"""A receiver that dated its samples a block out would slice every slot
|
||||
late and cut the first half-second -- three symbols and part of the
|
||||
opening sync -- off every transmission on the band."""
|
||||
heard, _total = _run(slots=2)
|
||||
assert heard.band.decodes
|
||||
# Everything on this band starts within half a second of the nominal
|
||||
# start, so everything should be reported within about that of zero.
|
||||
worst = max(abs(d.offset) for d in heard.band.decodes)
|
||||
assert worst < 1.0, f"the worst decode was {worst:.2f} s out"
|
||||
|
||||
|
||||
def test_callsigns_only_leaves_out_the_free_text():
|
||||
heard, _ = _run(slots=1, calls_only=True)
|
||||
assert all(d.kind in ("standard", "non-standard")
|
||||
for d in heard.band.decodes)
|
||||
|
||||
|
||||
def test_the_search_can_be_narrowed_and_the_narrowing_is_obeyed():
|
||||
heard, _ = _run(slots=1, lowest=800.0, highest=1600.0)
|
||||
assert heard.band.decodes
|
||||
assert all(800.0 <= d.hertz <= 1600.0 for d in heard.band.decodes)
|
||||
|
||||
|
||||
def test_a_station_that_calls_cq_is_counted_as_calling():
|
||||
heard, _ = _run(slots=2)
|
||||
calling = [s for s in heard.band.all() if s.calling]
|
||||
assert calling, "nobody called CQ on a band that is mostly CQ"
|
||||
assert all(s.grid for s in calling), "a CQ says where it is"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Writing it down
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _decode(text="CQ DL1UDO JO31", **kw):
|
||||
fields = dict(text=text, kind="standard", calls=("DL1UDO",),
|
||||
grid="JO31", report="", calling=True, hertz=1200.0,
|
||||
offset=0.2, snr_db=-7.0, at=1_700_000_000.0)
|
||||
fields.update(kw)
|
||||
return ft8wave.Decode(**fields)
|
||||
|
||||
|
||||
def test_a_log_is_written_and_read_back(tmp_path):
|
||||
log = ft8log.open_log(tmp_path, 1_700_000_000.0, 14_074_000.0, "20m")
|
||||
assert log is not None
|
||||
for i in range(3):
|
||||
log.append(_decode(hertz=1000.0 + i * 100))
|
||||
log.close()
|
||||
rows = ft8log.read_log(log.path)
|
||||
assert len(rows) == 3
|
||||
assert rows[0]["text"] == "CQ DL1UDO JO31"
|
||||
assert rows[1]["hertz"] == 1100.0
|
||||
|
||||
|
||||
def test_a_half_written_log_line_is_skipped_rather_than_raising(tmp_path):
|
||||
path = tmp_path / "ft8-part.txt"
|
||||
path.write_text("# header\n2023-11-14 22:13:20 -7 0.2 1200.0 ~ OK\n"
|
||||
"2023-11-14 22:13:35 -7 0.2\n")
|
||||
rows = ft8log.read_log(path)
|
||||
assert len(rows) == 1 and rows[0]["text"] == "OK"
|
||||
|
||||
|
||||
def test_a_csv_has_a_row_for_every_decode(tmp_path):
|
||||
where = ft8log.write_csv(tmp_path / "x.csv",
|
||||
[_decode(), _decode("W2XYZ K1ABC RRR")])
|
||||
assert where is not None
|
||||
body = where.read_text().splitlines()
|
||||
assert len(body) == 3 and body[0].startswith("utc,")
|
||||
|
||||
|
||||
def test_an_adif_says_these_were_heard_rather_than_worked(tmp_path):
|
||||
"""Nothing here transmits, so nothing here is a contact. An ADIF that
|
||||
let a logging program treat these as worked would put claims into
|
||||
somebody's log that they cannot make."""
|
||||
where = ft8log.write_adif(tmp_path / "x.adi",
|
||||
[_decode(), _decode("W2XYZ K1ABC RRR",
|
||||
calls=("W2XYZ", "K1ABC"))],
|
||||
band="20m", frequency=14_074_000.0,
|
||||
my_grid="IO91")
|
||||
body = where.read_text()
|
||||
assert "heard only, not worked" in body
|
||||
assert "not contacts" in body
|
||||
assert "<CALL:6>DL1UDO" in body
|
||||
assert "<MODE:3>FT8" in body
|
||||
assert "<MY_GRIDSQUARE:4>IO91" in body
|
||||
|
||||
|
||||
def test_an_adif_keeps_one_record_a_station_rather_than_one_a_decode(tmp_path):
|
||||
many = [_decode(snr_db=s) for s in (-20.0, -3.0, -11.0)]
|
||||
where = ft8log.write_adif(tmp_path / "y.adi", many)
|
||||
body = where.read_text()
|
||||
assert body.count("<EOR>") == 1
|
||||
# And the one kept is the best report, that being the one worth passing on.
|
||||
assert "<RST_RCVD:2>-3" in body
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The display
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_the_display_says_what_it_is_waiting_for_before_anything_arrives():
|
||||
from rich.console import Console
|
||||
from bandsaunter.ui import Ft8Display
|
||||
|
||||
console = Console(width=160, force_terminal=False)
|
||||
display = Ft8Display(console, band="20m — 14.074 MHz")
|
||||
display.update(ft8.Heard())
|
||||
with console.capture() as caught:
|
||||
console.print(display.render())
|
||||
shown = caught.get()
|
||||
assert "nothing decoded yet" in shown
|
||||
# And the one thing that silently stops FT8 working is worth saying.
|
||||
assert "clock" in shown
|
||||
|
||||
|
||||
def test_the_display_shows_the_last_slot_and_the_running_total():
|
||||
from rich.console import Console
|
||||
from bandsaunter.ui import Ft8Display
|
||||
|
||||
console = Console(width=110, force_terminal=False)
|
||||
display = Ft8Display(console, grid="IO91", band="20m — 14.074 MHz")
|
||||
heard = ft8.Heard()
|
||||
for text in ("CQ DL1UDO JO31", "W2XYZ K1ABC RRR"):
|
||||
found = _decode(text, calls=tuple(text.split()[1:2] or ["DL1UDO"]),
|
||||
grid="JO31" if "CQ" in text else "")
|
||||
heard.band.add(found)
|
||||
heard.decodes += 1
|
||||
heard.latest = heard.band.decodes
|
||||
heard.slots = 4
|
||||
display.update(heard, 1_700_000_000.0)
|
||||
with console.capture() as caught:
|
||||
console.print(display.render(now=1_700_000_005.0))
|
||||
shown = caught.get()
|
||||
assert "last slot" in shown and "heard so far" in shown
|
||||
assert "DL1UDO" in shown and "JO31" in shown
|
||||
assert "4 slots" in shown
|
||||
|
||||
|
||||
def test_the_report_says_nothing_rather_than_an_empty_table():
|
||||
from rich.console import Console
|
||||
|
||||
console = Console(width=100, force_terminal=False)
|
||||
band = ft8.Band()
|
||||
band.slots = 12
|
||||
with console.capture() as caught:
|
||||
ft8.report(console, band, ft8.defaults())
|
||||
shown = caught.get()
|
||||
assert "nothing decoded" in shown
|
||||
assert "clock" in shown, "the usual cause is worth naming"
|
||||
|
||||
|
||||
def test_the_report_names_the_furthest_station_when_it_can():
|
||||
from rich.console import Console
|
||||
|
||||
console = Console(width=120, force_terminal=False)
|
||||
band = ft8.Band()
|
||||
band.slots = 4
|
||||
band.add(_decode("CQ VK4BLE QG62", calls=("VK4BLE",), grid="QG62"))
|
||||
band.add(_decode("CQ DL1UDO JO31", calls=("DL1UDO",), grid="JO31"))
|
||||
options = ft8.defaults()
|
||||
options.grid = "IO91"
|
||||
with console.capture() as caught:
|
||||
ft8.report(console, band, options)
|
||||
shown = caught.get()
|
||||
assert "furthest heard" in shown and "VK4BLE" in shown
|
||||
|
||||
|
||||
def test_the_report_leaves_distances_out_when_it_has_nowhere_to_measure_from():
|
||||
from rich.console import Console
|
||||
|
||||
console = Console(width=120, force_terminal=False)
|
||||
band = ft8.Band()
|
||||
band.add(_decode("CQ DL1UDO JO31", calls=("DL1UDO",), grid="JO31"))
|
||||
with console.capture() as caught:
|
||||
ft8.report(console, band, ft8.defaults())
|
||||
shown = caught.get()
|
||||
assert "DL1UDO" in shown and "away" not in shown
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Opening a receiver
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_every_name_these_modules_import_actually_exists():
|
||||
"""Imports inside a function are not checked until that function runs.
|
||||
|
||||
Which is how `open_rtlsdr` -- a name nothing in this program has ever
|
||||
exported -- sat in the one line that opens the receiver, past every
|
||||
test, until somebody chose Listen and was told the section was broken.
|
||||
The tests reached that line only through the simulator, which takes the
|
||||
other branch.
|
||||
"""
|
||||
import ast
|
||||
import importlib
|
||||
|
||||
missing = []
|
||||
for name in ("ft8", "ft8code", "ft8wave", "ft8log", "ft8sim",
|
||||
"ft8tables"):
|
||||
module = importlib.import_module(f"bandsaunter.{name}")
|
||||
tree = ast.parse(Path(module.__file__).read_text())
|
||||
for node in ast.walk(tree):
|
||||
if not isinstance(node, ast.ImportFrom) or node.level != 1:
|
||||
continue
|
||||
if not node.module:
|
||||
continue
|
||||
target = importlib.import_module(f"bandsaunter.{node.module}")
|
||||
for alias in node.names:
|
||||
if not hasattr(target, alias.name):
|
||||
missing.append(f"{name}: {node.module}.{alias.name}")
|
||||
assert not missing, missing
|
||||
|
||||
|
||||
def test_the_made_up_band_needs_no_receiver_and_says_it_is_made_up():
|
||||
from rich.console import Console
|
||||
|
||||
console = Console(width=90, force_terminal=False)
|
||||
options = ft8.defaults()
|
||||
options.simulate = True
|
||||
with console.capture() as caught:
|
||||
device = ft8.open_device(console, options)
|
||||
assert isinstance(device, ft8sim.SimulatedBand)
|
||||
assert "not there" in caught.get(), "a simulation must say so"
|
||||
device.close()
|
||||
|
||||
|
||||
def test_a_receiver_that_cannot_be_opened_is_reported_rather_than_raising():
|
||||
"""A menu has to survive it, and the person has to be told which of the
|
||||
two things went wrong: the receiver, or the listening."""
|
||||
from rich.console import Console
|
||||
|
||||
console = Console(width=90, force_terminal=False)
|
||||
options = ft8.defaults()
|
||||
options.device = 99 # no such dongle
|
||||
with console.capture() as caught:
|
||||
assert ft8.open_device(console, options) is None
|
||||
assert "cannot open the receiver" in caught.get()
|
||||
|
||||
|
||||
def test_nothing_claims_to_be_listening_before_there_is_a_receiver(tmp_path):
|
||||
"""Announcing the frequency and then failing to open a dongle reads as
|
||||
though the listening went wrong, when what went wrong happened before
|
||||
any of it started."""
|
||||
from rich.console import Console
|
||||
|
||||
console = Console(width=90, force_terminal=False)
|
||||
options = ft8.defaults()
|
||||
options.device = 99
|
||||
options.log = False
|
||||
with console.capture() as caught:
|
||||
assert ft8.listen(console, options, str(tmp_path)) == 0
|
||||
shown = caught.get()
|
||||
assert "cannot open the receiver" in shown
|
||||
assert "listening on" not in shown
|
||||
Loading…
Add table
Add a link
Reference in a new issue