Frame the map on the receiver, and throw out what never happened

A first real capture came back as a map spanning 240 degrees north to 20
south, with the aircraft an indistinguishable smudge in one corner.  Two
separate faults, one of them mine from the start.

A position is sent as half a position -- an even frame and an odd one --
and the pair only means anything while the aircraft has not moved between
them.  The registry kept the last of each forever and paired them
regardless of age, so an even frame from ten minutes ago decoded against
a fresh odd one to a place on the wrong side of the world.  Measured: a
pair 300 seconds apart puts the aircraft 2,566 nm from where it is, and
one night's log had it happening to two aircraft in three, with eleven
positions off the planet altogether.  A pair is now good for ten seconds,
the answer has to be on Earth, and the aircraft has to have been able to
reach it.

--radius, defaulting to a hundred, frames the picture on the receiver
rather than on whatever was heard, so the scale is the same from one
evening to the next.  In the same unit as the speeds.  The centre is the
median of everything heard, which a handful of wrong positions cannot
move, or --at LAT,LON says where the aerial is.

--recheck repairs a log recorded before all this: for each aircraft it
keeps the longest run of positions that could describe one aeroplane.
Not a forward walk dropping whatever disagrees with the last position
kept -- that lets one bad fix become the reference, and on the same log
it discarded a fifth of everything, most of it the truth.

Two calibrations came from the recording rather than from taste.  The
failures separate cleanly -- a hundred artefacts under a mile, twelve
hundred real errors over fifty, and nothing in between -- because
positions are stamped to the millisecond, so two a thousandth of a second
apart imply thousands of knots across a few yards.  Nothing under two
miles is called an error.  Afterwards the worst surviving jump is 513 kt.

One rendering bug the tighter frame exposed: an aircraft just off the top
left painted 743,774 pixels of an 844,200-pixel picture, because the dot
at a marker's centre clamped its near edge and left the far one alone, and
numpy reads a negative slice end as counting back from the far side.  And
a still of a whole evening was dead-reckoning every aircraft forward to
the final moment, which for a ten-hour log flew 291 of 335 clean off the
picture; a still now draws each where it was last actually heard.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PsWPTweCT6pwxKngvVxcg

tests/test_aircraft_menu.py

@@ -344,3 +344,88 @@ def test_the_map_underneath_is_an_option_that_can_be_turned_off(monkeypatch,
     cfg = ScanConfig(output_dir=str(tmp_path))
     run(monkeypatch, console, [number("basemap"), "no", "s", "b"], cfg)
     assert air.load_options().basemap is False
+
+
+# ---------------------------------------------------------------------------
+# How far the map reaches
+# ---------------------------------------------------------------------------
+
+def test_the_radius_defaults_to_a_hundred():
+    """An aerial hears about that far; a map drawn to fit everything heard
+    is drawn to fit the mistakes."""
+    assert air.AircraftOptions().radius == 100.0
+
+
+def test_the_radius_is_changed_from_the_menu(monkeypatch, console,
+                                             settings_dir, tmp_path):
+    cfg = ScanConfig(output_dir=str(tmp_path))
+    run(monkeypatch, console, [number("radius"), "40", "s", "b"], cfg)
+    assert air.load_options().radius == 40.0
+
+
+def test_the_radius_follows_the_unit_the_speeds_are_in():
+    """A picture measuring its speeds in one unit and its own extent in
+    another would be a puzzle rather than a map."""
+    assert air.radius_in_nm(air.AircraftOptions(radius=100)) == 100.0
+    assert air.radius_in_nm(
+        air.AircraftOptions(radius=100, speed_unit="mph")) == pytest.approx(
+            86.9, abs=0.1)
+    assert air.radius_in_nm(
+        air.AircraftOptions(radius=100, speed_unit="kph")) == pytest.approx(
+            54.0, abs=0.1)
+
+
+def test_no_radius_at_all_goes_back_to_fitting_what_was_heard():
+    assert air.radius_in_nm(air.AircraftOptions(radius=0)) == 0.0
+
+
+def test_the_receiver_position_can_be_given_or_worked_out():
+    from bandsaunter.flightlog import read_position
+
+    assert read_position("32.54,-111.17") == pytest.approx((32.54, -111.17))
+    assert read_position("") is None
+    assert read_position("somewhere near Tucson") is None
+    assert read_position("240.0,-111.0") is None      # not a place
+    assert air.AircraftOptions().location == ""       # worked out by default
+
+
+def test_rechecking_is_an_option_and_is_off_unless_asked(monkeypatch, console,
+                                                         settings_dir,
+                                                         tmp_path):
+    """Logs written by this version have the check applied as they are
+    written, so it is a repair for older ones rather than a default."""
+    assert air.AircraftOptions().recheck is False
+    cfg = ScanConfig(output_dir=str(tmp_path))
+    run(monkeypatch, console, [number("recheck"), "yes", "s", "b"], cfg)
+    assert air.load_options().recheck is True
+
+
+def test_rechecking_says_what_it_threw_out(capsys):
+    from bandsaunter.flightlog import Fix, Track
+
+    loud = Console(width=100)
+    track = Track(icao="4CA1FA", fixes=[
+        Fix(at=0.0, latitude=51.5, longitude=-0.12),
+        Fix(at=0.5, latitude=-9.5, longitude=-112.5),
+        Fix(at=1.0, latitude=51.5, longitude=-0.12)])
+    air.checked(loud, air.AircraftOptions(recheck=True), [track])
+    printed = capsys.readouterr().out
+    assert "dropped" in printed and "could have been in" in printed
+
+
+def test_a_clean_log_is_told_so_rather_than_left_silent(capsys):
+    loud = Console(width=100)
+    air.checked(loud, air.AircraftOptions(recheck=True), [])
+    assert "checks out" in capsys.readouterr().out
+
+
+def test_leaving_it_off_changes_nothing(capsys):
+    from bandsaunter.flightlog import Fix, Track
+
+    loud = Console(width=100)
+    track = Track(icao="4CA1FA", fixes=[
+        Fix(at=0.0, latitude=51.5, longitude=-0.12),
+        Fix(at=0.5, latitude=-9.5, longitude=-112.5)])
+    same = air.checked(loud, air.AircraftOptions(recheck=False), [track])
+    assert same[0] is track
+    assert capsys.readouterr().out.strip() == ""

tests/test_flightmap.py

@@ -460,3 +460,276 @@ def test_the_animation_takes_the_unit_through_to_the_file(tmp_path):
     out = fm.animate(two_aircraft(), tmp_path / "mph.png", width=500,
                      unit="mph")
     assert out is not None and out.path.is_file()
+
+
+# ---------------------------------------------------------------------------
+# Framing the picture on the receiver
+# ---------------------------------------------------------------------------
+
+def far_away(icao="BAD001", lat=-9.5, lon=-112.5) -> Track:
+    """One aircraft in the wrong hemisphere: a decode that went wrong."""
+    return Track(icao=icao, callsign="GHOST",
+                 fixes=[Fix(at=1_000_000.0, latitude=lat, longitude=lon,
+                            altitude_ft=35_000, ground_speed_kt=400.0)],
+                 first_seen=1_000_000.0, last_seen=1_000_000.0)
+
+
+def test_the_middle_of_what_was_heard_is_where_the_receiver_is():
+    """A median, so a handful of wrong positions cannot drag it anywhere."""
+    from bandsaunter.flightlog import centre_of
+
+    tracks = two_aircraft() + [far_away(), far_away("BAD002", 60.0, 120.0)]
+    lat, lon = centre_of(tracks)
+    assert lat == pytest.approx(51.0, abs=1.0)
+    assert lon == pytest.approx(-1.0, abs=1.5)
+
+
+def test_a_radius_leaves_the_far_ones_off_the_map():
+    tracks = two_aircraft() + [far_away()]
+    view_all = fm.fit([t for t in tracks if t.located])
+    assert view_all.north - view_all.south > 50        # dragged across a globe
+    from bandsaunter.flightlog import centre_of, within
+
+    lat, lon = centre_of(two_aircraft())
+    near = [t for t in within(tracks, lat, lon, 100.0) if t.located]
+    assert {t.icao for t in near} == {t.icao for t in two_aircraft()}
+
+
+def test_the_frame_is_the_radius_rather_than_whatever_turned_up():
+    """The scale should not change with the traffic."""
+    from bandsaunter.flightlog import box_around, centre_of, distance_nm
+
+    lat, lon = centre_of(two_aircraft())
+    box = box_around(lat, lon, 100.0)
+    view = fm.fit(two_aircraft(), width=600, box=box)
+    assert view.south == pytest.approx(box[0]) and view.north == pytest.approx(box[2])
+    across = distance_nm(lat, box[1], lat, box[3])
+    assert across == pytest.approx(200.0, rel=0.02)    # a hundred each way
+
+
+def test_the_radius_is_kept_by_the_animation(tmp_path):
+    """Without one the frame stretches to reach a bad position on the other
+    side of the equator, and the real aircraft come out a pixel wide."""
+    from bandsaunter.flightlog import box_around, centre_of
+
+    tracks = two_aircraft() + [far_away()]
+    wide = fm.animate(tracks, tmp_path / "wide.png", width=400, radius_nm=0)
+    tight = fm.animate(tracks, tmp_path / "tight.png", width=400, radius_nm=100)
+    assert wide.aircraft == 3 and tight.aircraft == 2
+
+    everything = fm.fit([t for t in tracks if t.located], width=400)
+    lat, lon = centre_of(two_aircraft())
+    framed = fm.fit(two_aircraft(), width=400,
+                    box=box_around(lat, lon, 100.0))
+    assert everything.north - everything.south > 50      # most of a hemisphere
+    assert framed.north - framed.south < 4               # a hundred miles
+
+
+def test_an_explicit_receiver_position_is_used_as_given(tmp_path):
+    """Somewhere with no aircraft near it: everything falls outside."""
+    out = fm.animate(two_aircraft(), tmp_path / "elsewhere.png", width=400,
+                     radius_nm=50, centre=(0.0, 0.0))
+    assert out is None
+
+
+def test_a_track_with_one_bad_fix_keeps_the_rest_of_its_flight():
+    """Per fix rather than per aircraft: one wrong position in the middle of
+    a real flight must not take the flight off the map with it."""
+    from bandsaunter.flightlog import within
+
+    track = straight()
+    good = len(track.fixes)
+    track.fixes.insert(len(track.fixes) // 2,
+                       Fix(at=track.fixes[0].at + 1, latitude=-9.5,
+                           longitude=-112.5, altitude_ft=35_000))
+    kept = within([track], 51.0, -1.0, 100.0)[0]
+    assert len(kept.fixes) == good
+
+
+# ---------------------------------------------------------------------------
+# Drawing off the edge
+# ---------------------------------------------------------------------------
+
+@pytest.mark.parametrize("x,y", [(-28, -89), (-4000, 400), (450, -9000),
+                                 (2000, 400), (450, 4000)])
+def test_an_aircraft_off_the_picture_paints_nothing(x, y):
+    """Clamping the near edge of the centre dot and leaving the far one alone
+    reads as a negative slice, which numpy counts back from the far side --
+    and fills most of the frame with one colour."""
+    img = np.zeros((300, 500), dtype=np.uint8)
+    fm._marker(img, x, y, 90.0, 20)
+    assert int((img != 0).sum()) == 0
+
+
+@pytest.mark.parametrize("x,y", [(0, 0), (499, 299), (250, 150)])
+def test_an_aircraft_on_the_picture_is_still_drawn(x, y):
+    img = np.zeros((300, 500), dtype=np.uint8)
+    fm._marker(img, x, y, 90.0, 20)
+    assert 0 < int((img != 0).sum()) < 60
+
+
+def test_a_still_draws_where_they_were_not_where_they_might_have_got_to():
+    """A still of a whole evening is drawn hours after most of the aircraft
+    stopped transmitting; flying them on for those hours would scatter them
+    across three states."""
+    tracks = [straight(seconds=600.0)]
+    view = fm.fit(tracks, width=500)
+    base = fm.background(view)
+    hours_later = tracks[0].last_seen + 6 * 3600
+    projected = fm.render_frame(base, view, tracks, hours_later,
+                                stale=7 * 3600, project=True, labels=False)
+    still = fm.render_frame(base, view, tracks, hours_later,
+                            stale=7 * 3600, project=False, labels=False)
+
+    def markers(frame):
+        """Just the aircraft: what changed, in the aircraft colours, inside
+        the map itself.  The altitude key along the bottom is drawn in those
+        same colours and belongs to the background."""
+        body = slice(view.top, view.top + view.height), \
+            slice(view.left, view.left + view.width)
+        drawn = (frame[body] != base[body]) & (frame[body] >= fm.RAMP) & \
+            (frame[body] < fm.TRAIL)
+        return int(drawn.sum())
+
+    assert markers(projected) == 0             # flown clean off the picture
+    assert markers(still) > 0                  # drawn where it last was
+    last = tracks[0].fixes[-1]
+    x, y = view.xy(last.latitude, last.longitude)
+    assert (still[y - 6:y + 7, x - 6:x + 7] >= fm.RAMP).any()
+
+
+def test_a_whole_evening_as_one_picture_keeps_every_aircraft_on_it(tmp_path):
+    tracks = two_aircraft()
+    for t in tracks:                            # heard hours ago
+        for f in t.fixes:
+            f.at -= 6 * 3600
+        t.first_seen -= 6 * 3600
+        t.last_seen -= 6 * 3600
+    tracks.append(straight(icao="C0FFEE", callsign="LATE", lat=51.2, lon=-0.9))
+    out = fm.animate(tracks, tmp_path / "evening.png", width=500)
+    assert out is not None and out.aircraft == 3
+
+
+# ---------------------------------------------------------------------------
+# Throwing out what could not have happened
+# ---------------------------------------------------------------------------
+
+def _with_bad_fix(track: Track, at_index: int, lat=-9.5, lon=-112.5) -> Track:
+    """Drop one impossible position into the middle of a real flight."""
+    when = track.fixes[at_index].at + 0.001
+    track.fixes.insert(at_index + 1,
+                       Fix(at=when, latitude=lat, longitude=lon,
+                           altitude_ft=35_000, ground_speed_kt=400.0))
+    return track
+
+
+def test_a_position_no_aircraft_could_reach_is_thrown_out():
+    from bandsaunter.flightlog import recheck
+
+    track = _with_bad_fix(straight(), 10)
+    good = len(track.fixes) - 1
+    clean, dropped = recheck([track])
+    assert dropped == 1
+    assert len(clean[0].fixes) == good
+    assert all(-90 <= f.latitude <= 90 for f in clean[0].fixes)
+
+
+def test_a_flight_that_is_entirely_real_loses_nothing():
+    from bandsaunter.flightlog import recheck
+
+    clean, dropped = recheck(two_aircraft())
+    assert dropped == 0
+    assert [len(t.fixes) for t in clean] == [len(t.fixes) for t in two_aircraft()]
+
+
+def test_a_bad_position_early_on_does_not_take_the_flight_with_it():
+    """Keeping whatever is reachable from the last position kept lets one
+    bad fix become the reference, and then the truth is what gets thrown
+    away.  On a real recording that discarded a fifth of everything."""
+    from bandsaunter.flightlog import recheck
+
+    track = _with_bad_fix(straight(), 0)
+    clean, dropped = recheck([track])
+    assert dropped == 1
+    assert len(clean[0].fixes) == len(straight().fixes)
+
+
+def test_a_run_of_bad_positions_that_agree_with_each_other_still_goes():
+    """Three bad decodes can land in the same wrong place and agree
+    perfectly; they are still wrong."""
+    from bandsaunter.flightlog import recheck
+
+    track = straight()
+    where = 10
+    for i in range(3):
+        track.fixes.insert(where + 1 + i,
+                           Fix(at=track.fixes[where].at + 0.001 * (i + 1),
+                               latitude=-9.5 + i * 0.001, longitude=-112.5,
+                               altitude_ft=35_000))
+    clean, dropped = recheck([track])
+    assert dropped == 3
+    assert all(f.latitude > 0 for f in clean[0].fixes)
+
+
+def test_a_position_that_is_not_on_earth_never_survives():
+    from bandsaunter.flightlog import recheck
+
+    track = _with_bad_fix(straight(), 5, lat=239.6, lon=-111.0)
+    clean, _ = recheck([track])
+    assert all(-90 <= f.latitude <= 90 for f in clean[0].fixes)
+
+
+def test_an_aircraft_that_went_quiet_and_came_back_is_not_an_error():
+    """Out of range for ten minutes, then heard again a long way off: that
+    is an aeroplane, not a bad decode, and there is no way to say otherwise."""
+    from bandsaunter.flightlog import recheck
+
+    track = straight(seconds=200.0)
+    last = track.fixes[-1]
+    track.fixes.append(Fix(at=last.at + 900, latitude=last.latitude + 1.5,
+                           longitude=last.longitude + 2.0,
+                           altitude_ft=35_000, ground_speed_kt=480.0))
+    clean, dropped = recheck([track])
+    assert dropped == 0
+
+
+def test_two_positions_that_contradict_each_other_do_not_both_stand():
+    """One of them is wrong and nothing says which; what comes out has at
+    least to be a story."""
+    from bandsaunter.flightlog import recheck
+
+    track = Track(icao="4CA1FA", fixes=[
+        Fix(at=0.0, latitude=51.5, longitude=-0.12),
+        Fix(at=0.5, latitude=-9.5, longitude=-112.5)])
+    clean, dropped = recheck([track])
+    assert dropped == 1
+    assert len(clean[0].fixes) == 1
+
+
+def test_a_jitter_of_a_few_yards_is_never_called_an_error():
+    """Positions arrive twice a second and are stamped to the millisecond,
+    so two of them a thousandth of a second apart imply thousands of knots
+    across a few yards."""
+    from bandsaunter.flightlog import recheck
+
+    track = Track(icao="4CA1FA", fixes=[
+        Fix(at=0.0, latitude=51.5000, longitude=-0.1200),
+        Fix(at=0.001, latitude=51.5001, longitude=-0.1201),
+        Fix(at=0.002, latitude=51.5002, longitude=-0.1202)])
+    clean, dropped = recheck([track])
+    assert dropped == 0
+
+
+def test_nothing_survives_that_needed_an_impossible_speed():
+    """The promise of the whole thing, said as one assertion."""
+    from bandsaunter.flightlog import (MAX_GROUND_SPEED_KT, distance_nm,
+                                       implied_speed_kt, recheck)
+
+    tracks = [_with_bad_fix(straight(), i) for i in (3, 12, 20)]
+    clean, dropped = recheck(tracks)
+    assert dropped == 3
+    for track in clean:
+        for a, b in zip(track.fixes, track.fixes[1:]):
+            if 0 < b.at - a.at <= 300 and distance_nm(
+                    a.latitude, a.longitude, b.latitude, b.longitude) > 2:
+                assert implied_speed_kt(a, b) <= MAX_GROUND_SPEED_KT

tests/test_signals_named.py

@@ -392,3 +392,107 @@ def test_a_payload_that_says_what_it_is_is_named():
 def test_an_ordinary_payload_claims_no_format():
     bits = "".join(format(b, "08b") for b in b"\x01\x02\x03\x04\x05\x06\x07\x08")
     assert not [r for r in interpret(bits) if r.kind == "fields"]
+
+
+# ---------------------------------------------------------------------------
+# A position is only as good as the pair it was decoded from
+# ---------------------------------------------------------------------------
+
+def _position_frame(icao, lat, lon, odd, when, altitude=35000):
+    """One position frame, read back the way the decoder would read it."""
+    from bandsaunter.adsb import _read, encode_position
+
+    data = encode_position(icao, lat, lon, altitude, odd=odd)
+    frame = _read("".join(format(b, "08b") for b in data), data)
+    frame.received_at = when
+    return frame
+
+
+def test_a_stale_pair_is_not_a_position():
+    """Compact position reporting sends a fraction of a zone, so an even
+    frame from ten minutes ago read against a fresh odd one puts the
+    aircraft on the wrong side of the world.  Measured against one night's
+    recording it was doing exactly that to two aircraft in three."""
+    registry = AircraftRegistry()
+    registry.add(_position_frame(0xABCDEF, 32.55, -111.16, False, 1000.0),
+                 when=1000.0)
+    registry.add(_position_frame(0xABCDEF, 33.22, -111.16, True, 1300.0),
+                 when=1300.0)
+    assert not registry.aircraft["ABCDEF"].located
+
+
+def test_a_fresh_pair_still_places_it_exactly():
+    registry = AircraftRegistry()
+    registry.add(_position_frame(0xABCDEF, 33.22, -111.16, False, 1000.0),
+                 when=1000.0)
+    registry.add(_position_frame(0xABCDEF, 33.22, -111.16, True, 1000.5),
+                 when=1000.5)
+    craft = registry.aircraft["ABCDEF"]
+    assert craft.latitude == pytest.approx(33.22, abs=0.01)
+    assert craft.longitude == pytest.approx(-111.16, abs=0.01)
+
+
+@pytest.mark.parametrize("gap", [0.0, 0.5, 4.0, 9.5])
+def test_a_pair_inside_the_allowed_gap_is_used(gap):
+    registry = AircraftRegistry()
+    registry.add(_position_frame(0xA0B1C2, 51.5, -0.12, False, 100.0),
+                 when=100.0)
+    registry.add(_position_frame(0xA0B1C2, 51.5, -0.12, True, 100.0 + gap),
+                 when=100.0 + gap)
+    assert registry.aircraft["A0B1C2"].located
+
+
+def test_a_position_that_is_not_on_earth_is_refused():
+    """A latitude of 240 degrees is not a place.  One night's log held
+    eleven of them."""
+    from bandsaunter.adsb import _on_earth
+
+    assert not _on_earth(239.6, -111.0)
+    assert not _on_earth(45.0, 200.0)
+    assert _on_earth(-33.9, 151.2)
+
+
+def test_an_aircraft_cannot_cross_a_continent_between_two_frames():
+    """A position needing nine hundred thousand knots to reach is not a
+    position, whatever the checksum said about the frames it came from."""
+    registry = AircraftRegistry()
+    for odd in (False, True):
+        registry.add(_position_frame(0xA0B1C2, 51.5, -0.12, odd, 100.0),
+                     when=100.0)
+    craft = registry.aircraft["A0B1C2"]
+    assert craft.located
+    was = (craft.latitude, craft.longitude)
+    # Two seconds later, a pair that decodes to the far side of the Atlantic.
+    for odd in (False, True):
+        registry.add(_position_frame(0xA0B1C2, 40.7, -74.0, odd, 102.0),
+                     when=102.0)
+    assert (craft.latitude, craft.longitude) == was
+
+
+def test_an_aircraft_that_really_moved_is_still_followed():
+    """The bar has to be above anything that flies, or a fast aircraft is
+    called an error."""
+    registry = AircraftRegistry()
+    for odd in (False, True):
+        registry.add(_position_frame(0xA0B1C2, 51.5, -0.12, odd, 100.0),
+                     when=100.0)
+    # Sixty seconds on and eight miles further east: 480 knots.
+    for odd in (False, True):
+        registry.add(_position_frame(0xA0B1C2, 51.5, 0.08, odd, 160.0),
+                     when=160.0)
+    assert registry.aircraft["A0B1C2"].longitude == pytest.approx(0.08,
+                                                                  abs=0.02)
+
+
+def test_a_gap_in_reception_is_not_treated_as_an_error():
+    """Nothing heard for an hour, then a position a long way off: that is an
+    aircraft that flew away and came back, not a bad decode."""
+    registry = AircraftRegistry()
+    for odd in (False, True):
+        registry.add(_position_frame(0xA0B1C2, 51.5, -0.12, odd, 100.0),
+                     when=100.0)
+    for odd in (False, True):
+        registry.add(_position_frame(0xA0B1C2, 48.8, 2.3, odd, 4000.0),
+                     when=4000.0)
+    assert registry.aircraft["A0B1C2"].latitude == pytest.approx(48.8,
+                                                                 abs=0.05)