dustcouncil/bandsaunter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Read the stations that never say a word

Browse source 
Most of what identifies itself on the air identifies itself in Morse.  A
repeater, a beacon, an unattended transmitter: four to six characters, over
in a second or two, and no speech anywhere in the capture.  Every one of
those was being thrown away, in three separate places.

The callsign book and the map were built inside the transcription branch,
on the reasoning that callsigns come out of transcripts.  They also come out
of Morse and out of APRS headers, neither of which involves a speech
recogniser -- so a receiver with none installed found none of them, and a CW
ident reached the sidecar and stopped there.  Both are now built whenever
classification is on, and all three sources go through one place.

The CW decoder only ran where the classifier had already said cw, ook or
carrier.  A two-second ident is a fraction of a capture named after whatever
filled the rest of it.  Every capture is offered to it now, once it has
finished; a decode does not relabel a capture that plainly holds speech.

And the decoder's own gates were written for a paragraph.  Three characters,
eight elements, and any repeated character refused -- which read VVV, DE, AR
and K correctly and then discarded them.  Short is the normal case now, on a
second bar: perfect timing, nothing undecoded, and the keyed tone at least
20 dB over its band.  That last is not decoration.  With four elements the
dot length is fitted to those very elements, so noise lands on the grid as
neatly as keying does; a third of a second of white noise decodes as a
perfectly timed V.  Over 200 noise blocks the loudest bin never rose 13 dB
above the median while keying at 3 dB SNR sits above 40.  One keyed element
is still refused: a single pulse is an E or a T whether a person sent it or
the squelch opened on a click.  288 non-Morse cases, no false positives.

Feeding that text to a callsign lookup made truncation matter.  A capture
opens when the squelch does, halfway through an element as often as not, and
half a character is not a smaller reading -- a K missing its first dash is
an A.  So the sliced character is dropped, and so is the rest of its word,
because what is left can read as a whole one: K1AA caught halfway through is
K1A, which is somebody else.  Across 1805 truncated captures that is 107
invented callsigns down to none, with 550 correct ones still found.

Phonetics, which is the other half of the ask.  A recogniser has never heard
of the alphabet -- it writes what the words sounded like:

  Whiskey-One-Alpha-Whiskey    hyphenated
  WhiskeyOneAlphaWhiskey       run together
  Whiskey1AlphaWhiskey         and half in digits
  wiskey one alfa whisky       spelled the way it sounded
  whiskey one alpha, uh, whiskey   with the hesitation written down

All read back to W1AW now.  A word is only taken apart when it is phonetic
all the way through, which is what keeps it off "kilometre" and "victorious".

Two bugs found on the way, both of which invented a callsign:

 - Nothing is joined across a slash any more.  The beacon W1AW/B came back
   as W1AWB, which belongs to nobody, and W1AW-4 came back as nothing at all.
 - Nor across a gap the sender chose.  A transcript's spacing is the
   recogniser's guess and may be closed up; a word gap in Morse is seven dot
   units, so "KU0W K" is a station signing off, not a longer callsign.

Also here:

 - saunterbrowse gives Morse a panel of its own, with the licence under it,
   searchable with / and readable with t.
 - --simulate no longer looks anything up or writes a map.  The demo band is
   invented but W1AW is the ARRL's own station, and it would have been
   pinned to the same map a real scan writes.
 - classify._psk_order took the logarithm of zero on a silent block.
 - The demo band has a repeater ident in it, and its Morse no longer runs
   one repeat into the next.
 - conftest refuses a real licence lookup from any test.

1161 tests, up from 1014.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PsWPTweCT6pwxKngvVxcg
This commit is contained in:
The Dust Council 2026-08-29 14:59:25 -07:00
parent f9f0d94000
commit b4718aa425
21 changed files with 1434 additions and 129 deletions
Download patch file Download diff file Expand all files Collapse all files

149
tests/test_morse.py
View file

@ -48,3 +48,152 @@ def test_silence_is_rejected():
def test_encode_round_trip():
assert encode_morse("SOS") == "... --- ..."
assert encode_morse("A B") == ".- / -..."
# ---------------------------------------------------------------------------
# Short bursts
#
# Most of the CW on the air is not a conversation. It is a repeater, a
# beacon or an unattended transmitter saying who it is and stopping, which is
# four to six characters and over in a second or two. Those used to be read
# correctly and then thrown away by gates written for a paragraph of text.
# ---------------------------------------------------------------------------
SHORT = ["W1AW", "K1AA", "KU0W", "N0CALL", "VVV", "DE", "AR", "K", "73", "QRZ"]
@pytest.mark.parametrize("msg", SHORT)
@pytest.mark.parametrize("wpm", [12, 20, 30, 45])
def test_a_short_burst_is_read_and_believed(msg, wpm):
r = decode_morse(morse_audio(msg, wpm, FS, 18), FS)
assert r.text.strip() == msg
assert r.is_morse, f"read {msg} correctly and then refused it: {r.notes}"
def test_a_callsign_on_its_own_is_under_two_seconds_at_thirty_words():
"""The length this is all about, stated so a regression is obvious."""
from morse_gen import morse_keying
keyed = np.flatnonzero(morse_keying("W1AW", 30, FS) > 0)
assert (keyed[-1] - keyed[0]) / FS < 2.0
assert decode_morse(morse_audio("W1AW", 30, FS, 18), FS).is_morse
@pytest.mark.parametrize("msg", ["E", "T"])
def test_one_keyed_element_is_not_an_identification(msg):
"""A single pulse is an E or a T whether a person sent it or not.
This is where "no matter how short" stops, and it stops here because
below it there is nothing left to be right about.
"""
assert not decode_morse(morse_audio(msg, 20, FS, 20), FS).is_morse
def test_a_repeated_character_with_structure_is_still_text():
"""VVV is the oldest thing anyone sends, and it is not a pulse train.
A uniform train of identical pulses decodes to EEEE or TTTT, which is
what the check against one repeated character is for -- but V has dots
and a dash of its own, which no train of identical pulses can produce.
"""
r = decode_morse(morse_audio("VVV", 20, FS, 20), FS)
assert r.text.strip() == "VVV" and r.is_morse
def test_a_uniform_pulse_train_is_still_refused():
dot = 1.2 / 20.0
env, fs = [], FS
for _ in range(14):
env += [1.0] * int(dot * fs) + [0.0] * int(3 * dot * fs)
env = np.array(env)
t = np.arange(env.size) / fs
audio = env * np.sin(2 * np.pi * 700 * t)
assert not decode_morse(audio, fs).is_morse
# -- what the capture window cut off ----------------------------------------
def _clipped(msg, wpm, head, tail):
audio = morse_audio(msg, wpm, FS, 20)
return audio[int(head * FS):audio.size - int(tail * FS)]
def test_a_character_sliced_by_the_window_is_dropped_not_guessed():
"""A K with its first dash missing is an A, not a worse K."""
r = decode_morse(_clipped("K1AA", 20, 0.55, 0.55), FS)
assert "A1AA" not in r.text, "half a character was reported as a whole one"
assert any("cut off" in note for note in r.notes)
def test_what_is_left_of_a_cut_word_is_not_reported_as_a_whole_one():
"""K1AA caught halfway through reads as K1A, which is somebody else."""
r = decode_morse(_clipped("K1AA", 20, 0.05, 0.9), FS)
assert r.tail_cut
assert "K1A" not in r.complete_text.split()
def test_the_words_that_survived_are_still_reported():
r = decode_morse(_clipped("VVV DE W1AW", 20, 0.6, 0.1), FS)
assert r.head_cut and not r.tail_cut
assert "W1AW" in r.complete_text.split()
def test_a_complete_transmission_keeps_every_word():
r = decode_morse(morse_audio("VVV DE W1AW", 20, FS, 20), FS)
assert not r.head_cut and not r.tail_cut
assert r.complete_text == r.text == "VVV DE W1AW"
@pytest.mark.parametrize("head,tail", [(0.05, 0.4), (0.4, 0.05), (0.7, 0.7),
(1.1, 0.3), (0.3, 1.1)])
def test_a_truncated_capture_never_invents_a_callsign(head, tail):
"""The point of all of the above: this text is looked at for callsigns.
A station heard through half its ident is better reported as half an
ident than as a different station, because the different station gets
looked up and pinned to a map.
"""
from bandsaunter.callsign import find_callsigns
r = decode_morse(_clipped("VVV DE W1AW/B FN31", 20, head, tail), FS)
if not r.is_morse:
return
found = find_callsigns(r.complete_text, join_words=False)
assert set(found) <= {"W1AW"}, f"invented {found} from {r.text!r}"
# -- and nothing that was not sent ------------------------------------------
@pytest.mark.parametrize("seed", range(10))
def test_a_blip_of_noise_is_not_a_short_transmission(seed):
"""The gates that let a two-character ident in must not let this in.
With three or four elements the dot length is fitted to those very
elements, so they land on the grid whatever produced them: a third of a
second of noise decodes as a perfectly timed V. What separates them is
that keying is a tone and noise is not.
"""
rng = np.random.default_rng(seed)
for size in (1200, int(FS * 0.3), FS * 2):
assert not decode_morse(rng.standard_normal(size), FS).is_morse
@pytest.mark.parametrize("seed", range(6))
def test_a_squelch_click_is_not_a_transmission(seed):
rng = np.random.default_rng(seed)
t = np.arange(int(FS * 0.5)) / FS
for edges in ([(0.20, 0.25)], [(0.10, 0.14), (0.30, 0.34)]):
env = np.zeros_like(t)
for a, b in edges:
env[int(a * FS):int(b * FS)] = 1.0
audio = env * np.sin(2 * np.pi * 700 * t) + 0.02 * rng.standard_normal(t.size)
assert not decode_morse(audio, FS).is_morse
def test_a_tone_that_never_keys_is_not_morse():
t = np.arange(FS * 2) / FS
assert not decode_morse(np.sin(2 * np.pi * 700 * t), FS).is_morse
@pytest.mark.parametrize("seed", range(4))
def test_speech_is_not_morse(seed):
from speech import synth_speech
assert not decode_morse(synth_speech(2.5, FS, seed=seed), FS).is_morse