Cache what has been looked up, make it browsable, and say where this lives
Four things asked for in turn, landing together because they run through the same files. THE MAPS. The tiles were already cached and always had been -- a second evening on the same view was measured at nought network requests -- but the work done on them was not. Every window open decoded forty PNGs and resampled a megapixel and a half into this program's own projection, for an answer that cannot have changed, because a coastline does not move. The finished map is kept beside the tiles now: 0.61 seconds become 0.01, byte for byte identical, two hundred kilobytes a view. Both windows and both kinds of still picture get it, all four reaching the ground through one function. A map with squares missing is deliberately not kept, since caching a hole would keep it for a month and the point of calling a partial map provisional is that it is asked for again. And because this adds a disk consumer, the whole cache is now pruned to four hundred megabytes, least recently *used* first: a tile fetched a year ago and looked at last night is the receiver's own neighbourhood, and discarding that to keep last week's holiday is the wrong way round. THE LOOKUPS. APRS and FT8 now ask who each station is licensed to. Every other mode that hears a callsign already did -- speech transcripts, Morse idents, the recording browser -- and all five resolve through one file, so a callsign heard on two bands is asked about once. The rules for finding the licensed callsign inside a heard one are now in one place rather than per band, because getting them wrong is silent: a register asked about W1AW-9 returns nothing, which looks exactly like a station that is not licensed. An SSID, a rover suffix, a guest prefix, a digipeater alias and an unspelled hash all come off or are refused. The bug worth recording is that the first cut of this did the lookups and threw them away. The book has to be told to save and was not, so every evening would have asked the register about the same net again -- which is the one thing caching them was for, and is invisible from inside a single run because the answers are all in memory while it lasts. Caught by looking at the file on disk rather than at the display. There is a test for each side of it now, and a register of which modules resolve callsigns at all, which fails when a new one starts so that somebody has to decide whether it should. THE REGISTER. c in saunterbrowse opens everything ever looked up: sixty-odd callsigns and sixteen hundred aircraft here, every field of each in two columns because a licence has a dozen and a screen is wider than it is tall. tab switches, / searches every field rather than the name -- the question is usually "who was in Arizona" rather than "which callsign" -- and g opens the place in a browser. Only the coordinates go into that link: a map does not need to be told whose licence it is looking at, and the link is the one part of this that leaves the machine. A headless box, which is the normal case for a receiver, gets the coordinates printed instead. Callsigns no register could place are kept rather than dropped, because "asked about, and in no register reachable from here" is a fact about a station. THE FRONT OF IT. A title screen for each program: five rows of blocks cut by hand, a figlet dependency to draw eleven letters being the largest thing that would then be in the requirements, coloured blue to red across the width, which is the ramp every waterfall here already uses because it is what a spectrum looks like. The interesting part is where it does not appear -- everything here can be piped into something else and a banner in the middle of that is corruption rather than decoration, so anything that is not a terminal gets nothing, --help is untouched because it is drawn after parsing, --no-splash turns it off for a run and BANDSAUNTER_NO_SPLASH=1 for good. And the address. INSTALL.md said "git clone <the repository>" for a long time: a placeholder in the first command anybody types, unnoticed because nothing reads install instructions except somebody installing, who then cannot. It is filled in, along with the readme, the metadata, both manuals and the Homepage field of all three packages. The tile server's User-Agent pointed at a topic listing on somebody else's site for want of an address of its own; the usage policy of that service asks for one naming the application and giving somewhere to look it up, so an operator with a question about the traffic has somebody to ask, and now it gives the real one. Seven tests so the placeholder cannot come back, verified by putting it back and watching two of them fail. One thing forced by all this: the keys page in saunterbrowse was exactly as tall as an eighty-by-twenty-four terminal, so the register entry pushed "q quit" off the bottom. A test caught it. Home and End have merged into the Page Up line, which were always the same thought. THE WINDOWS. They open maximised now, this being a map and the thing anybody wants more of being map; f goes to true full screen and back, and is written along the top of the screen because a window with no frame is one somebody has to know a key to get out of. Maximised rather than frameless by default, because the title bar is where the band and the frequency are written. And a map made bigger now gets a sharper map, which it did not. The window only ever re-examined the ground when the view left the box that had been fetched, so a window opened at its default size and taken to the whole screen kept the map it started with until an aircraft wandered far enough to move it -- on a quiet band, a long time to look at a blurred coastline. Measured before it was believed: 1100 to 1920 asked for nothing and stretched a 1364-pixel map across 1920, then across 3840. It now compares map pixels per degree in hand against what the view wants, and asks when it is being blown up by more than fifteen per cent. That comparison has to be per degree rather than pixel against pixel, which a surviving mutation was what established: the fetched box is a quarter wider than the view, so a map with exactly as many pixels as the window is wide has only four fifths of them on the screen. The two ways of measuring agree everywhere except a narrow band, and the realistic case sits inside it. There is a test pinning that case now. Asking is safe at any size, because the request is keyed on the window's dimensions: once answered, nothing more is asked, which is what stops a window larger than the tile budget can cover from asking all evening. The braille, while this was open. The banners were blocks in capitals; they are braille in mixed case, two dots wide and four tall to a character, which is eight times the detail and is what makes room for two heights of letter at once -- a five-row block font has no second height to spend, so the name came out shouted. Two attempts failed first: thin one-dot strokes came out as confetti, because braille dots render as dots and a one-dot stroke reads as a dotted line rather than a line. What worked was cutting the font small, at cap height seven where there is only one way to draw each letter, and doubling it. Coloured deep blue through cyan to a cool white, which is deliberately not the waterfall ramp the rest of the program draws in: that one has to run to red because it stands in for a spectrum, and a title screen stands in for nothing. One hundred and four new tests. Full suite 2892 passed. Built as 2026-09-24_01. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016PsWPTweCT6pwxKngvVxcg
This commit is contained in:
The Dust Council
parent
93120b80a6
commit
2665a17a22
27 changed files with 3733 additions and 49 deletions
|
|
@ -5,6 +5,7 @@ in through the same door the real ones come through, and the PNGs are built
|
|||
here from the specification rather than by the decoder they are testing.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import struct
|
||||
import time
|
||||
import zlib
|
||||
|
|
@ -333,10 +334,14 @@ def test_a_missing_tile_is_a_gap_rather_than_the_brightest_thing_on_the_map():
|
|||
so a square that never arrived used to come out as the brightest thing
|
||||
on the picture: a glowing rectangle where the map should be."""
|
||||
zoom, hole = _hole_at()
|
||||
# remember=False throughout: these two are the same view with different
|
||||
# tiles under it, which cannot happen on the air and would otherwise be
|
||||
# answered from the first one's cache entry.
|
||||
whole, _ = bm.ground_under(*BOX, 400, 400, fetch=_patchy(()), pause=0,
|
||||
zoom=zoom)
|
||||
zoom=zoom, remember=False)
|
||||
holed, _ = bm.ground_under(*BOX, 400, 400, fetch=_patchy({hole}),
|
||||
pause=0, retry_pause=0, zoom=zoom)
|
||||
pause=0, retry_pause=0, zoom=zoom,
|
||||
remember=False)
|
||||
assert whole is not None and holed is not None
|
||||
gap = _hole_mask(hole, zoom)
|
||||
assert gap.sum() > 1000, "the hole is not where this test thinks it is"
|
||||
|
|
@ -358,9 +363,10 @@ def test_a_missing_tile_does_not_dim_the_rest_of_the_map():
|
|||
not there."""
|
||||
zoom, hole = _hole_at()
|
||||
whole, _ = bm.ground_under(*BOX, 400, 400, fetch=_patchy(()), pause=0,
|
||||
zoom=zoom)
|
||||
zoom=zoom, remember=False)
|
||||
holed, _ = bm.ground_under(*BOX, 400, 400, fetch=_patchy({hole}),
|
||||
pause=0, retry_pause=0, zoom=zoom)
|
||||
pause=0, retry_pause=0, zoom=zoom,
|
||||
remember=False)
|
||||
# Outside the gap and the averaged band along its edge, the map is the
|
||||
# map: the hole took nothing else with it.
|
||||
elsewhere = ~_hole_mask(hole, zoom, grow=2)
|
||||
|
|
@ -374,10 +380,10 @@ def test_a_map_with_squares_missing_says_it_is_not_the_whole_answer():
|
|||
again."""
|
||||
_zoom, hole = _hole_at()
|
||||
whole, settled = bm.ground_under(*BOX, 400, 400, fetch=_patchy(()),
|
||||
pause=0)
|
||||
pause=0, remember=False)
|
||||
assert whole is not None and settled
|
||||
holed, settled = bm.ground_under(*BOX, 400, 400, fetch=_patchy({hole}),
|
||||
pause=0, retry_pause=0)
|
||||
pause=0, retry_pause=0, remember=False)
|
||||
assert holed is not None and not settled
|
||||
|
||||
|
||||
|
|
@ -394,9 +400,10 @@ def test_a_tile_that_fails_once_is_asked_for_again():
|
|||
return None
|
||||
return solid(40 if (x + y) % 3 == 0 else 230)
|
||||
|
||||
whole, _ = bm.ground_under(*BOX, 400, 400, fetch=_patchy(()), pause=0)
|
||||
whole, _ = bm.ground_under(*BOX, 400, 400, fetch=_patchy(()), pause=0,
|
||||
remember=False)
|
||||
healed, settled = bm.ground_under(*BOX, 400, 400, fetch=flaky, pause=0,
|
||||
retry_pause=0)
|
||||
retry_pause=0, remember=False)
|
||||
assert refused["left"] == 0, "the tile was never asked for a second time"
|
||||
assert settled, "a recovered map is the whole answer"
|
||||
assert bool((healed == whole).all())
|
||||
|
|
@ -878,3 +885,157 @@ def test_a_map_asked_for_at_more_detail_than_the_tiles_hold_still_works():
|
|||
fetch=gradient_tile, zoom=9, pause=0)
|
||||
assert levels.shape == (2000, 2000)
|
||||
assert levels[0].mean() != levels[-1].mean()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Keeping the finished map
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_a_map_already_built_is_not_built_again(monkeypatch):
|
||||
"""The tiles were always cached; the work done on them was not. Opening
|
||||
the same window twice used to decode forty PNGs and resample a
|
||||
megapixel and a half into this program's own projection, both times,
|
||||
for an answer that cannot have changed."""
|
||||
box = (47.0, -123.0, 48.0, -122.0)
|
||||
asked = []
|
||||
|
||||
def counting(z, x, y, **kw):
|
||||
asked.append((z, x, y))
|
||||
return gradient_tile(z, x, y, **kw)
|
||||
|
||||
first, settled = bm.ground_under(*box, 300, 300, shades=32,
|
||||
fetch=counting, pause=0)
|
||||
assert settled and first is not None
|
||||
built = len(asked)
|
||||
assert built > 1
|
||||
|
||||
asked.clear()
|
||||
again, settled = bm.ground_under(*box, 300, 300, shades=32,
|
||||
fetch=counting, pause=0)
|
||||
assert settled
|
||||
assert asked == [], "it went back to the tiles for a map it already had"
|
||||
assert bool((again == first).all()), "the kept map is a different map"
|
||||
|
||||
|
||||
def test_a_map_with_squares_missing_is_not_kept():
|
||||
"""Caching a hole would keep it for a month, and the whole point of
|
||||
calling a partial map provisional is that it gets asked for again."""
|
||||
zoom, hole = _hole_at()
|
||||
holed, settled = bm.ground_under(*BOX, 400, 400, fetch=_patchy({hole}),
|
||||
pause=0, retry_pause=0, zoom=zoom)
|
||||
assert holed is not None and not settled
|
||||
assert list(bm.ground_cache_dir().glob("*.npz")) == []
|
||||
|
||||
|
||||
def test_a_different_picture_is_a_different_map():
|
||||
"""Same piece of world, bigger window: reusing the smaller one would be
|
||||
exactly the upscale this program went to some trouble to avoid."""
|
||||
box = (47.0, -123.0, 48.0, -122.0)
|
||||
small, _ = bm.ground_under(*box, 200, 200, shades=32,
|
||||
fetch=gradient_tile, pause=0)
|
||||
large, _ = bm.ground_under(*box, 400, 400, shades=32,
|
||||
fetch=gradient_tile, pause=0)
|
||||
assert small.shape == (200, 200) and large.shape == (400, 400)
|
||||
elsewhere, _ = bm.ground_under(48.0, -123.0, 49.0, -122.0, 200, 200,
|
||||
shades=32, fetch=gradient_tile, pause=0)
|
||||
assert len(list(bm.ground_cache_dir().glob("*.npz"))) == 3
|
||||
|
||||
|
||||
def test_a_view_that_drifted_a_few_metres_is_the_same_view():
|
||||
"""Rounded to about a hundred metres, which is finer than a tile and
|
||||
far finer than anything visible: a map rebuilt because the middle
|
||||
moved by a pixel would defeat the point of keeping it."""
|
||||
a, _ = bm.ground_under(47.0, -123.0, 48.0, -122.0, 200, 200, shades=32,
|
||||
fetch=gradient_tile, pause=0)
|
||||
asked = []
|
||||
|
||||
def counting(z, x, y, **kw):
|
||||
asked.append((z, x, y))
|
||||
return gradient_tile(z, x, y, **kw)
|
||||
|
||||
b, _ = bm.ground_under(47.00002, -123.00002, 48.00002, -121.99998,
|
||||
200, 200, shades=32, fetch=counting, pause=0)
|
||||
assert asked == []
|
||||
assert bool((a == b).all())
|
||||
|
||||
|
||||
def test_keeping_the_map_can_be_turned_off():
|
||||
box = (47.0, -123.0, 48.0, -122.0)
|
||||
bm.ground_under(*box, 250, 250, shades=32, fetch=gradient_tile,
|
||||
pause=0, remember=False)
|
||||
assert list(bm.ground_cache_dir().glob("*.npz")) == []
|
||||
|
||||
|
||||
def test_a_truncated_cache_file_is_a_miss_rather_than_a_crash():
|
||||
"""Which is what a cache looks like after a power cut."""
|
||||
box = (47.0, -123.0, 48.0, -122.0)
|
||||
bm.ground_under(*box, 220, 220, shades=32, fetch=gradient_tile, pause=0)
|
||||
kept = list(bm.ground_cache_dir().glob("*.npz"))
|
||||
assert len(kept) == 1
|
||||
kept[0].write_bytes(b"PK\x03\x04 truncated")
|
||||
levels, settled = bm.ground_under(*box, 220, 220, shades=32,
|
||||
fetch=gradient_tile, pause=0)
|
||||
assert levels is not None and settled and levels.shape == (220, 220)
|
||||
|
||||
|
||||
def test_a_map_kept_too_long_ago_is_built_again(monkeypatch):
|
||||
box = (47.0, -123.0, 48.0, -122.0)
|
||||
bm.ground_under(*box, 210, 210, shades=32, fetch=gradient_tile, pause=0)
|
||||
later = time.time() + bm.GROUND_CACHE_DAYS * 86_400 + 1
|
||||
monkeypatch.setattr(bm.time, "time", lambda: later)
|
||||
asked = []
|
||||
|
||||
def counting(z, x, y, **kw):
|
||||
asked.append(1)
|
||||
return gradient_tile(z, x, y, **kw)
|
||||
|
||||
bm.ground_under(*box, 210, 210, shades=32, fetch=counting, pause=0)
|
||||
assert asked, "a month-old map was believed"
|
||||
|
||||
|
||||
def test_forgetting_the_maps_keeps_the_tiles():
|
||||
"""The maps are quick to rebuild from tiles and the tiles are not quick
|
||||
to fetch again, so the cheap thing goes first."""
|
||||
bm.ground_under(47.0, -123.0, 48.0, -122.0, 230, 230, shades=32,
|
||||
fetch=gradient_tile, pause=0)
|
||||
before = bm.cache_size()
|
||||
assert before["ground"]["files"] == 1
|
||||
assert bm.forget_ground() == 1
|
||||
after = bm.cache_size()
|
||||
assert after["ground"]["files"] == 0
|
||||
assert after["tiles"]["files"] == before["tiles"]["files"]
|
||||
|
||||
|
||||
def test_the_cache_says_how_big_it_is(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("XDG_CACHE_HOME", str(tmp_path))
|
||||
empty = bm.cache_size()
|
||||
assert empty["total"]["bytes"] == 0 and empty["total"]["files"] == 0
|
||||
root = tmp_path / "bandsaunter" / "tiles" / "9" / "81"
|
||||
root.mkdir(parents=True)
|
||||
(root / "178.png").write_bytes(b"x" * 1000)
|
||||
got = bm.cache_size()
|
||||
assert got["tiles"] == {"bytes": 1000, "files": 1}
|
||||
assert got["total"]["bytes"] == 1000
|
||||
|
||||
|
||||
def test_pruning_throws_away_the_least_recently_used(tmp_path, monkeypatch):
|
||||
"""Least recently *used*, not oldest: a tile fetched a year ago and
|
||||
looked at last night is the receiver's own neighbourhood, and throwing
|
||||
that away to keep last week's holiday is the wrong way round."""
|
||||
monkeypatch.setenv("XDG_CACHE_HOME", str(tmp_path))
|
||||
where = tmp_path / "bandsaunter" / "tiles" / "9" / "81"
|
||||
where.mkdir(parents=True)
|
||||
now = time.time()
|
||||
for i, age in enumerate((100.0, 0.0, 50.0)): # middle one is freshest
|
||||
path = where / f"{i}.png"
|
||||
path.write_bytes(b"x" * 400_000)
|
||||
os.utime(path, (now - age * 86_400, now - age * 86_400))
|
||||
assert bm.cache_size()["tiles"]["files"] == 3
|
||||
gone = bm.prune_cache(limit_mb=0.5)
|
||||
assert gone == 2
|
||||
left = list(where.glob("*.png"))
|
||||
assert [p.name for p in left] == ["1.png"], "it kept the wrong one"
|
||||
|
||||
|
||||
def test_pruning_a_cache_that_already_fits_does_nothing():
|
||||
assert bm.prune_cache(limit_mb=10_000) == 0
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue