Two things, both found by measuring rather than by reading the code.
The voice-activity filter inside the recogniser is off. It was costing
words: across a night of land-mobile captures it dropped 5-15% of what
the same model finds without it -- 491 against 507, 339 against 384,
263 against 310 -- because a single-word over between two
transmissions looks to a VAD exactly like the noise it exists to
remove, and on a scanner those short replies are the ones worth
having.
Turning it off has a cost, and the cost is that Whisper hands back
"You" for five seconds of hiss as confidently as it hands back a
sentence. So the whole capture is now asked once whether anything in
it rises above its own noise. Digital silence measures 0.0 dB of
contrast and hiss at any level 0.7, while the quietest real capture of
that night measures 8.9 and most measure 10-27; the bar sits at 3, an
order of magnitude clear of both. It can veto a capture but never trim
one, which is the whole difference between it and the filter it
replaces.
The second thing: callsigns like WQVF960 were being missed entirely.
The shape being matched was the amateur one -- prefix, district digit,
suffix -- and everything else the FCC licenses is written the other
way round, the letters first and then the digits. On the GMRS and
business channels that is most of what is said: nine callsigns across
five transcripts of one evening went by unrecognised, and now do not.
The shape is written as the three allocations that exist rather than
as "letters then digits", which claims KN95, WD40 and KC135. Its
letters are checked against the word list even when they arrive as a
single token, which the amateur shape does not need -- no English word
has a digit in the middle of it, but "west 120" and "word 100" fit
this one exactly.
Lookups now fall back to hamdb.org when callook has nothing. Not a
spare copy: callook holds United States amateur licences only, so
DL1ABC and VE3ABC are INVALID there and resolve perfectly well from
the other. And a GMRS callsign is not looked up at all -- every
database reachable without an account is an amateur register, so
reporting WQVF960 as "unlisted" would blame the callsign for the
absence of a source.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PsWPTweCT6pwxKngvVxcg
Hexadecimal is a true answer to "what did that say" and not a useful one.
This is the work of turning the rest of what a receiver hears into
something a person can read, and most of it is pictures.
PICTURES
Three of the things on the air are images rather than sounds, and all three
arrive as the audio a scan already records:
SSTV 14.230 and 144.5 MHz Martin M1/M2, Scottie S1/S2/DX, Robot 36/72
APT 137-138 MHz the NOAA weather satellites
HF fax 2-20 MHz, sideband the marine weather charts
Each is written from its published specification, and the generators used to
test them are written from the same specification without reference to the
decoders -- so a picture that comes back matching the one that went in is
evidence about the format. Every SSTV mode reproduces its published line
time exactly, which is worth failing a test over: a line a few milliseconds
long walks the picture off the screen inside ten lines. Against synthetic
transmissions at 30 dB SNR, SSTV is 96-98% of pixels exact, APT correlates
at 0.97 and fax at 0.998; all three still read at 6-12 dB.
None of the three is guessed at, and that is what makes it safe to try them
on every recording. SSTV needs its VIS header, APT needs both line syncs at
the right distance from each other, fax needs the phasing signal. No false
pictures in 295 attempts over noise, tones, speech and swept whistles.
Two things had to be got right beyond the arithmetic. A band-pass does not
switch between two tones, it slides between them, so every edge is measured
at the midpoint of the slide rather than at the first sample past a
threshold -- the earlier version was reading the coarse search stride back
as the edge and shifting Martin M1 sideways by a whole colour bar. And a
picture now keeps its capture whatever the content check made of it: a
satellite is a steady tone with a wobble on it and SSTV is a whistle, so
both were being discarded as "no signal content" having already been
recognised.
PNG is written here rather than pulled in from Pillow. A scanner that
cannot start because an imaging library is missing is worse than one that
cannot draw.
saunterbrowse marks a picture in the list, gives its path in full -- wrapped
rather than cut off, because half a path opens nothing -- and moves or
deletes the PNGs with the recording. o prints the picture's path, not the
audio's.
GRIB is not a modulation and is not pretended to be one. It is the format
weather models are published in and it travels by satellite link and by
e-mail; where a decoded byte stream begins with its magic number it is
named, and that is all.
AIRCRAFT
`bandsaunter adsb` parks the receiver on 1090 MHz and reads Mode S extended
squitter: address, callsign, altitude, position, speed. A command of its
own because a megabit a second will not go through a channel twelve and a
half kilohertz wide. Every frame carries a 24-bit checksum so there is no
threshold anywhere in it -- with one trap, which is that a frame of all
zeros satisfies that checksum and silence is exactly that. Positions round
trip exactly through compact position reporting, and a pair straddling a
longitude-zone boundary is refused rather than resolved against two grids.
METERS AND SENSORS
Itron ERT utility meters on 900 MHz and AcuRite weather sensors on 433 MHz
are named rather than reported as hex, and neither is believed without its
own checksum -- BCH(255,239) for the meter, a checksum and four parity bits
for the sensor. Both are implemented from published descriptions and
checked against frames built from the same descriptions, which proves the
framing and the arithmetic and is not the same as having held a meter.
HEX INTO WORDS
Everything else that decodes to bits now gets its fields named where the
shape is standard, its text read out where there is text, and its bytes laid
out in groups with the printable characters beside them.
The text search is where the care went, because printability is not
evidence. Forty framings of each packet, and seven-bit values printable
three in four, meant a bar set on printability called 64% of random payloads
text. Real text is nearly all one case where random letters are half and
half, two fifths vowels where random is a fifth, and mostly alphanumeric
where random draws punctuation one time in four. Together: under 0.5%,
measured in the suite.
CALLSIGNS
The licensed address is recorded in full -- the street, not merely the town
-- and goes into the KML with everything else. US amateur records are
public by law and carry it; holding it and not saying so is worse than
either showing it or not asking, and --no-lookup asks for none of it.
Also here: Morse is decoded again from the whole recording where the capture
was made in cw mode. The first pass works from the classifier's buffer,
which holds a few seconds -- enough to say "this is Morse", not enough to
catch a callsign whole between two word gaps, so a beacon repeating every
eight seconds through an eight-second window was never identified.
And classify._psk_order took the logarithm of zero on a silent block.
1318 tests, up from 1161.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PsWPTweCT6pwxKngvVxcg
Most of what identifies itself on the air identifies itself in Morse. A
repeater, a beacon, an unattended transmitter: four to six characters, over
in a second or two, and no speech anywhere in the capture. Every one of
those was being thrown away, in three separate places.
The callsign book and the map were built inside the transcription branch,
on the reasoning that callsigns come out of transcripts. They also come out
of Morse and out of APRS headers, neither of which involves a speech
recogniser -- so a receiver with none installed found none of them, and a CW
ident reached the sidecar and stopped there. Both are now built whenever
classification is on, and all three sources go through one place.
The CW decoder only ran where the classifier had already said cw, ook or
carrier. A two-second ident is a fraction of a capture named after whatever
filled the rest of it. Every capture is offered to it now, once it has
finished; a decode does not relabel a capture that plainly holds speech.
And the decoder's own gates were written for a paragraph. Three characters,
eight elements, and any repeated character refused -- which read VVV, DE, AR
and K correctly and then discarded them. Short is the normal case now, on a
second bar: perfect timing, nothing undecoded, and the keyed tone at least
20 dB over its band. That last is not decoration. With four elements the
dot length is fitted to those very elements, so noise lands on the grid as
neatly as keying does; a third of a second of white noise decodes as a
perfectly timed V. Over 200 noise blocks the loudest bin never rose 13 dB
above the median while keying at 3 dB SNR sits above 40. One keyed element
is still refused: a single pulse is an E or a T whether a person sent it or
the squelch opened on a click. 288 non-Morse cases, no false positives.
Feeding that text to a callsign lookup made truncation matter. A capture
opens when the squelch does, halfway through an element as often as not, and
half a character is not a smaller reading -- a K missing its first dash is
an A. So the sliced character is dropped, and so is the rest of its word,
because what is left can read as a whole one: K1AA caught halfway through is
K1A, which is somebody else. Across 1805 truncated captures that is 107
invented callsigns down to none, with 550 correct ones still found.
Phonetics, which is the other half of the ask. A recogniser has never heard
of the alphabet -- it writes what the words sounded like:
Whiskey-One-Alpha-Whiskey hyphenated
WhiskeyOneAlphaWhiskey run together
Whiskey1AlphaWhiskey and half in digits
wiskey one alfa whisky spelled the way it sounded
whiskey one alpha, uh, whiskey with the hesitation written down
All read back to W1AW now. A word is only taken apart when it is phonetic
all the way through, which is what keeps it off "kilometre" and "victorious".
Two bugs found on the way, both of which invented a callsign:
- Nothing is joined across a slash any more. The beacon W1AW/B came back
as W1AWB, which belongs to nobody, and W1AW-4 came back as nothing at all.
- Nor across a gap the sender chose. A transcript's spacing is the
recogniser's guess and may be closed up; a word gap in Morse is seven dot
units, so "KU0W K" is a station signing off, not a longer callsign.
Also here:
- saunterbrowse gives Morse a panel of its own, with the licence under it,
searchable with / and readable with t.
- --simulate no longer looks anything up or writes a map. The demo band is
invented but W1AW is the ARRL's own station, and it would have been
pinned to the same map a real scan writes.
- classify._psk_order took the logarithm of zero on a silent block.
- The demo band has a repeater ident in it, and its Morse no longer runs
one repeat into the next.
- conftest refuses a real licence lookup from any test.
1161 tests, up from 1014.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PsWPTweCT6pwxKngvVxcg
The Dust Council