diff --git a/INSTALL.md b/INSTALL.md index baa9e40..a1a6f32 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -9,8 +9,11 @@ If you are on Debian, Ubuntu or Mint, [build the package](#a-debian-ubuntu-mint- [a virtual environment](#b-anywhere-else-a-virtual-environment). **You can try the whole program before buying or plugging in a receiver.** -`--simulate` runs the scanner against a synthetic band, and `bandsaunter adsb ---simulate` flies imaginary aircraft past an imaginary receiver. Neither needs +`--simulate` runs the scanner against a synthetic band, `bandsaunter adsb +--simulate` flies imaginary aircraft past an imaginary receiver, `bandsaunter +weather --simulate` puts six weather sensors on a fence that does not exist, +and `bandsaunter aprs --simulate` fills a channel with amateur stations that +are not there. None of the four needs hardware or a network. --- @@ -35,7 +38,7 @@ a Raspberry Pi is the easier answer. ## A. Debian, Ubuntu, Mint: the package ```bash -git clone bandsaunter +git clone https://frostwarning.com/git/dustcouncil/bandsaunter cd bandsaunter ./packaging/build-deb.sh # writes dist/bandsaunter__all.deb sudo apt install ./dist/bandsaunter_*.deb @@ -80,7 +83,7 @@ sudo apt install librtlsdr0 # Debian/Ubuntu/Mint # sudo pacman -S rtl-sdr # Arch # brew install librtlsdr # macOS -git clone bandsaunter +git clone https://frostwarning.com/git/dustcouncil/bandsaunter cd bandsaunter python3 -m venv .venv . .venv/bin/activate @@ -116,6 +119,10 @@ bandsaunter bands # the built-in US band plan; no hardware need bandsaunter scan -b 2m --simulate # a whole scan against a synthetic band bandsaunter adsb --simulate --seconds 30 bandsaunter flights # draws what the last command heard +bandsaunter weather --simulate --seconds 60 +bandsaunter readings --csv # turns what it heard into a spreadsheet +bandsaunter aprs --simulate --seconds 120 +bandsaunter packets --kml # turns what it heard into a map ``` With a receiver plugged in: @@ -205,6 +212,33 @@ package. `python3-pyqt6` on Debian and Fedora, `python-pyqt6` on Arch, or `pip install PyQt6` in the environment bandsaunter runs from. Without it the window is the only thing missing, and the program says so rather than failing. +**The weather sensors need nothing at all** beyond what is already in the +table above — no network, no extra package, no key. They are a few milliwatts +on 433.92 MHz and the only thing that decides whether they are heard is the +aerial: a quarter-wave whip is 17 cm, which the stock telescopic aerial does +if it is collapsed to about that. `bandsaunter weather --simulate` runs the +whole thing without one. + +It listens at 250 kS/s, tuned straight at 433.92 MHz, with the RTL2832's +digital gain control left off — the same configuration the established tools +for this band use, because differing from it turned out to buy nothing. + +If sensors you know are in range are not appearing, `bandsaunter weather +--diagnose` prints each second of band taken apart stage by stage and says +which of five possible faults it is — nothing arriving, nothing above the +noise, something never keyed, a burst that framed as nothing, or a message +that framed and arrived only once. The README section on it explains how to +read the output. `--save-iq FILE` keeps the raw samples (2 MB a second, so +bound it with `--seconds 60`) and `--from-iq FILE` reads one back, so a +recording made where the aerial is can be worked on anywhere. + +**APRS needs nothing extra either** — no network, no key, no package beyond +the table above. The only thing that decides whether you hear it is the aerial +and whether a digipeater is in range: a quarter-wave whip for 144 MHz is 49 cm, +which is longer than the one most dongles ship with. Check `--region` before +anything else, because on the wrong channel there is silence rather than a bad +signal. `bandsaunter aprs --simulate` runs the whole thing without an aerial. + **Aircraft and callsign lookups need no installation**, only a network. They ask public registers about a callsign or a 24-bit address and cache the answers for a month; `--no-lookup` turns them off, and what the address and diff --git a/README.md b/README.md index deb7624..f838bbe 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,12 @@ built-in US band plan — and it sweeps them, stops on anything above the noise floor, records it, and works out what kind of signal it was. CW/Morse is decoded to text. +Three things get sections of their own, because none of them fits through a +scan: [the aircraft overhead](#aircraft) on 1090 MHz, drawn on a moving map; +[the weather sensors](#weather-sensors-on-433-mhz) on 433 MHz, which you can +give names to as they arrive; and [APRS](#aprs-on-144-mhz) on 144 MHz, where +amateur stations report where they are and talk to each other. + Two programs: `bandsaunter` scans, and [`saunterbrowse`](#browsing-what-you-recorded) reads back what it collected — transcripts, identifications and playback, in one screen. @@ -35,6 +41,11 @@ transcripts, identifications and playback, in one screen. **[INSTALL.md](INSTALL.md) has the step-by-step version**, including the optional dependencies and what goes wrong first. The short forms: +```bash +git clone https://frostwarning.com/git/dustcouncil/bandsaunter +cd bandsaunter +``` + ### From a package (Debian, Ubuntu, Mint) ```bash @@ -240,6 +251,9 @@ bandsaunter # the menus: set up and scan bandsaunter scan -b 2m -b marine-vhf # band-plan presets bandsaunter scan -r 144M-148M -r 420M-450M # your own ranges bandsaunter scan -b 2m --simulate # try it without hardware +bandsaunter adsb --window # aircraft overhead, on a map +bandsaunter weather # the weather sensors on 433 MHz +bandsaunter aprs # amateur packet on 144.39 MHz ``` ## Two ways to drive it @@ -253,11 +267,22 @@ table of settings, so neither can offer something the other cannot. 2 Band plan 107 US presets 3 Settings record no limit, hang 6s, squelch +12 dB, keep voice, cw 4 Saved settings and profiles + 5 Aircraft (ADS-B) listen on 1090 MHz, draw where they went + 6 Weather sensors listen on 433 MHz, name what is out there + 7 APRS (144 MHz packet) positions, weather and messages from amateurs h Help s Start scanning q Quit ``` +Items 5, 6 and 7 are sections of their own, with their own options and their +own menus, because none of them fits through a scan: ADS-B is a megabit a +second and a scan channel is 12.5 kHz wide; a weather sensor message is a burst +of a carrier switched on and off that a scan would record as clicks; and APRS +is a two-second transmission every few minutes from a hundred stations sharing +one frequency, of which a sweep would catch whichever happened to key up as it +passed. + Settings are grouped, show their current value against the built-in default, and carry their own help: @@ -569,6 +594,54 @@ deliberately not measured from the spectrum -- a spread estimated from the data reads five times higher across the packed broadcast FM band than on empty spectrum, which would suppress exactly the stations you are looking for. +## The title screen + +Both programs draw one when they start, in a gradient that runs deep blue +through electric blue and cyan to a cool white across the width. Deliberately +*not* the waterfall ramp the rest of the program draws in: that one carries on +through green and yellow to red because it stands in for a spectrum and has to +mean something. A title screen means nothing, so it is allowed to be a colour +scheme — and blue is never the weakest channel anywhere along it, which is +what keeps it cold. + +``` +⣿⠛⠛⠛⣤⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⠀⣤⠛⠛⠛⠛⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣤⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ +⣿⣤⣤⣤⠛⠀⠀⠛⠛⠛⣤⠀⣿⣤⠛⠛⣤⠀⣤⠛⠛⠛⣤⠀⠛⣤⣤⣤⠀⠀⠀⠛⠛⠛⣤⠀⣿⠀⠀⠀⣿⠀⣿⣤⠛⠛⣤⠀⣤⣿⣤⣤⠀⠀⣤⠛⠛⠛⣤⠀⣿⣤⠛⠛⣤⠀ +⣿⠀⠀⠀⣿⠀⣤⠛⠛⠛⣿⠀⣿⠀⠀⠀⣿⠀⣿⠀⠀⠀⣿⠀⠀⠀⠀⠀⣿⠀⣤⠛⠛⠛⣿⠀⣿⠀⠀⣤⣿⠀⣿⠀⠀⠀⣿⠀⠀⣿⠀⠀⠀⠀⣿⠛⠛⠛⠛⠀⣿⠀⠀⠀⠀⠀ +⠛⠛⠛⠛⠀⠀⠀⠛⠛⠛⠛⠀⠛⠀⠀⠀⠛⠀⠀⠛⠛⠛⠀⠀⠛⠛⠛⠛⠀⠀⠀⠛⠛⠛⠛⠀⠀⠛⠛⠀⠛⠀⠛⠀⠀⠀⠛⠀⠀⠛⠛⠛⠀⠀⠀⠛⠛⠛⠀⠀⠛⠀⠀⠀⠀⠀ + + Conceived by: The Dust Council + 100% AI Coded by Claude Code. +``` + +**Drawn in braille**, two dots wide and four tall to a character. That is eight +times the detail a block character gives, and it is what makes room for +capitals and lowercase at the same height — a five-row block font has no +second height to spend, so the name comes out as BANDSAUNTER, which is not its +name. It also reads as something drawn rather than as masonry. + +The font is cut by hand rather than pulled in: twelve letters at cap height +seven, and a figlet library to draw that would be the largest thing in the +package's requirements. Cut small on purpose — at seven rows there is only one +way to draw each letter, and so no room to draw it badly — and then doubled, +because **a stroke two dots thick reads as a line where one dot thick reads as +a dotted line**. That is the one real difficulty in drawing with braille, and +the reason a first attempt at thin elegant strokes came out as confetti. + +**It never appears where it would be in the way.** Everything here can be +piped into something else — the band table, the sensor list, a line per packet +— and a banner in the middle of that is corruption rather than decoration, so +anything that is not a terminal gets nothing at all. `--help` and a bad +argument say their piece without one over the top, since it is drawn after the +arguments are parsed. `--no-splash` turns it off on a terminal too, and +`BANDSAUNTER_NO_SPLASH=1` turns it off for good. + +`saunterbrowse` pauses for a moment on its own banner, because what follows +takes the whole screen: the browser runs on the alternate screen, so without a +beat there the banner would be replaced in the same tenth of a second it was +drawn in. It is still on the normal screen afterwards, and comes back when you +quit. + ## Signal identification Every recording is classified from its own IQ. The classifier measures @@ -1097,6 +1170,49 @@ which leaves the coastline faintly visible through it, where a flat wash would not. The animated pictures had **no card at all** before this, so `0` is what they used to look like. +### Pulsing and echoes + +Two things that move on their own, both on the window and in the animated +pictures, and both turned off with a flag. + +**`--pulse`** swells each aircraft from bright to dim and back. At the top of +the swell it burns: drawn in a set of peak colours and wearing a halo grown +out of its own shape, which is what a phosphor does when the beam sits in one +place a little too long. A halo drawn as a *circle* round it would be a ring +of dots; this one is the aeroplane itself, spread outward a pixel at a time, +so the glow has the shape of the thing casting it. It goes only where the +picture was still empty — over the ground, the grid and the background, and +never over another aircraft. + +**`--echo`** sends a ring travelling outward from each aircraft, growing and +dimming as it goes. What a radar repeater does, and what the eye reads as +*this thing is transmitting* — which is exactly what an aeroplane on this +picture is doing, twice a second, and is how it got on the picture at all. +One ring at a time per aircraft: a new one leaves as the last reaches the end +of its reach. + +`--pulse-rate SECONDS` (2.2), `--echo-every SECONDS` (3) and `--echo-size +PIXELS` (46) set the rest. The two times are **seconds of watching, not of +flying**, so a pulse looks the same whatever speed an evening is being run +through. + +**Each aircraft is offset by its own address**, so a sky full of them swells +and rings separately rather than beating as one — which would read as a +display flashing rather than as a lot of separate things transmitting. The +offset comes from the address, so an aeroplane keeps its own rhythm from one +frame to the next and from one drawing of the same log to the next. + +**Only an aircraft still being heard pulses.** One that has gone quiet is +fading, and a thing that is fading and beating at once says two contradictory +things about itself. + +The window can blend, so its swell is continuous. The animation cannot — a GIF +is indexed colour — so there the swell is the handful of steps a palette +allows: which family of colours the aeroplane is drawn from, and how far its +halo reaches. Between them that is six steps, which at a couple of seconds a +cycle reads as a swell rather than as a blink. The peak has sixteen colours of +its own, sixteen being what was left of the palette. + **Range rings** put faint discs at a quarter, a half and three quarters of the radius, concentric on the receiver and each labelled with its distance. They are translucent and they stack, so the ground inside the innermost is lifted @@ -1122,9 +1238,21 @@ only in the same pass as a piece of map, which meant they queued behind a hundred and twenty tiles coming off a network, and once the map was in hand there were no more passes and they were never fetched at all. +**The window opens maximised.** This is a map, and the thing anybody wants +more of is map; it used to open at 1100×800 whatever the screen was. Un-maximise +it and you get a sensible window back, because the restored size is set before +it maximises rather than left to the toolkit to guess. + `d` cycles the detail — full box, just height and speed, or symbols alone — -for when the sky is busy. `t` toggles trails, `g` the map underneath, `[`/`]` -its brightness, `+`/`-` the range, `q` closes it. +for when the sky is busy. `t` toggles trails, `g` the map underneath, `f` true +full screen and back, `[`/`]` its brightness, `+`/`-` the range, `q` closes it. + +`f` is maximised rather than full screen by default on purpose: the title bar +is where the band and the frequency are written, and a window with no frame is +one you have to know a key to get out of. The key is along the top of the +screen for that reason, not only in the manual. Leaving full screen returns to +maximised rather than to the small size it was built at — coming out of full +screen should not shrink the map to a quarter of the display. Aircraft fade out here too, over `--fade` seconds, keeping their symbol and losing their box as they go — a box at a tenth of its colour is something in @@ -1551,6 +1679,33 @@ are met rather than assumed: - **the attribution is drawn onto the picture**, because a GIF travels without the readme that would otherwise carry it. +**The finished map is cached too**, in `~/.cache/bandsaunter/ground`. The +tiles always were, so a second evening on the same view has never touched the +network — but it still cost decoding forty PNGs and resampling a megapixel and +a half into this program's own projection, every time a window opened, for an +answer that cannot have changed. A coastline does not move. Measured on a +150-mile view at 1364×992: + +``` +first ever 5.12 s 18 tiles off the network +tiles cached 0.61 s no network, all the work redone +map cached 0.01 s no network, no work +``` + +Both windows and both kinds of still picture share it, because all four reach +the ground through one function. The key is the piece of world, the size of +the picture and the number of shades, with the box rounded to about a hundred +metres so a view that drifted by a pixel is answered rather than rebuilt. A +map with squares missing is **not** kept: caching a hole would keep it for a +month, and the whole point of calling a partial map provisional is that it +gets asked for again. + +Caches that only grow are a liability, so this one is pruned to 400 MB +whenever a map is written, throwing away the least recently *used* files +first — a tile fetched a year ago and looked at last night is the receiver's +own neighbourhood, and discarding that to keep last week's holiday would be +the wrong way round. + `--no-basemap` draws the tracks on their own, `--tiles URL` points at another server (your own, if you run one), and when there is no network and nothing cached the picture falls back to the plain grid it drew before. @@ -1561,6 +1716,27 @@ at 960, and about 1.4× the width is fetched deliberately and then averaged down — a downscaled tile is sharp and an upscaled one is not, so it is better to fetch too much and shrink it than to fetch too little and stretch it. +**A window made bigger fetches a sharper map.** The map underneath is fetched +for the size of the window at the time, and a map of the right piece of world +goes on being a map of the right piece of world however far it is then +stretched — so nothing else notices. A window opened at its default size and +taken to the whole screen used to keep the map it started with until an +aircraft wandered far enough to move the view out of the fetched box, which on +a quiet band is a long time to look at a blurred coastline. It now compares +**map pixels per degree in hand against what the view wants**, and asks for a +better one when it is being blown up by more than 15%. Less than that and a +window nudged a few pixels would send an evening to a tile server. + +The measurement has to be per degree rather than in raw pixels, because the +fetched box is a quarter wider than the view: a map with exactly as many +pixels as the window is wide has only four fifths of them on the screen, and +counting pixel-for-pixel would call that sharp. + +Asking is safe at any size. The request is keyed on the window's dimensions, +so once it has been answered nothing more is asked — which is what stops a +window bigger than the tile budget can cover from asking all evening. At 4K +the map is enlarged by design, as below. + The window fetches a little more world than it shows, so that panning does not leave the ground blank, and it fetches that bigger piece **at the bigger piece's own size**: what the window then shows comes out pixel for pixel with @@ -1583,6 +1759,37 @@ map is enlarged after all. Somebody else's tile server is not a thing to fetch a thousand tiles from for one picture. A smaller `--radius` buys the detail back, since the same budget then covers less ground. +**Tiles that do not arrive.** Resizing the window is the demanding case: a +wider picture picks a sharper zoom, and a hundred tiles that have never been +on this disk are asked for at once. A busy server refuses some of them, and +what the window used to do with a refusal was draw it. + +A tile that never arrived leaves its square of the canvas black, and black is +not a neutral colour here — the brightness is inverted on the way in, so the +darkest possible square comes out as **the brightest thing on the picture**: a +glowing rectangle where the map should be. It also dragged the floor of the +map's own contrast down to black, so every other pixel was drawn dimmer to +make room for a square that was not there. And it was kept: the map was +cached under the view it was fetched for, and the hole stayed until the view +changed. + +So three things happen instead. The missing squares are **asked for again**, +straight away, because the usual reason for a refusal is the ninety-nine tiles +asked for just before it — and only the missing ones, since the rest are on +the disk by then. What is still missing is **drawn as bare ground** rather +than as a light, and left out of the reckoning when the darkest and brightest +of the map are worked out, so a hole costs nothing but itself. And the map is +**remembered as provisional**: the window keeps drawing it and asks for the +rest of it half a minute later — four attempts in all, each of which retries +its own misses once, so a square gets eight chances before one that will not +come is accepted as one that is not there. + +The politeness pause between requests is now paid only on a tile that had to +be fetched. It had been paid on every tile including the ones read back off +the disk, which put twenty-six seconds of sleeping into redrawing a view that +was entirely cached — and would have made asking again for three missing +squares cost the wait for the two hundred that were not. + `--map-brightness PERCENT` (70 by default) is how far up its range the map is drawn, and **the vector themes bend the middle of that range down hard** — because a tinted photograph of a county behind the vectors is the one thing @@ -1657,7 +1864,7 @@ entirely. ## Every ADS-B option -Thirty-four of them, in the six groups the menu shows. Each is a flag on the +Thirty-nine of them, in the seven groups the menu shows. Each is a flag on the command line and a line in `bandsaunter` → 5, and both are generated from one table in the source, so they cannot disagree. `man bandsaunter` has the long form of every one. @@ -1726,35 +1933,997 @@ form of every one. | Label the aircraft | `--labels` `--no-labels` | `yes` | write the callsign, height and speed beside each aircraft | | Speed in | `--speed-unit` | `knots` | what to show speeds and distances in | +**effects** + +| option | flag | default | what it does | +| --- | --- | --- | --- | +| Pulsing aircraft | `--pulse` `--no-pulse` | `yes` | swell each aircraft from bright to dim and back, over and over | +| Pulse takes | `--pulse-rate` | `2.2 s` | how long one swell from dim to bright and back again takes | +| Echo circles | `--echo` `--no-echo` | `yes` | rings travelling outward from each aircraft, growing and dimming | +| Echo every | `--echo-every` | `3 s` | how long between one ring leaving an aircraft and the next | +| Echo reaches | `--echo-size` | `46 px` | how far a ring gets from its aircraft before it has faded away | + **Where they live.** Saved with `s` in the menu to `~/.config/bandsaunter/aircraft.yaml`, which is plain YAML you can edit or copy between machines. API keys are deliberately **not** kept there — see [Knowing the leg for certain](#knowing-the-leg-for-certain). -## Meters and weather sensors +## Weather sensors on 433 MHz -Two things on the ISM bands are worth naming rather than reporting as hex. +A consumer weather station is two things. The display on the kitchen wall is +one of them; the other is a plastic box on a fence post that says what it can +see every sixteen seconds, in the clear, on 433.92 MHz, to anyone who happens +to be listening. `bandsaunter weather` reads the box. + +``` +bandsaunter weather listen, and name things as they arrive +bandsaunter weather --simulate a garden of sensors that are not there +bandsaunter readings --csv turn a log into something plottable +bandsaunter sensors what is out there, and what it is called +``` + +It is a section of its own, like the aircraft one, and for the same reason: +it does not fit through the scanner. A sensor message is a burst of a carrier +switched on and off, a fifth of a second long, and the scan path is a squelch +and a recorder — it would record the bursts as clicks in a WAV file and decode +nothing. + +``` +╭──────────────────────────────────────────────────────────────────────────────────────╮ +│ 433.92 MHz 6 sensors 148 messages 41:02 weather_2026-09-07_11_31_04.jsonl │ +│ n to name (2 unnamed) control-C to stop │ +╰──────────────────────────────────────────────────────────────────────────────────────╯ + # name id model readings batt msgs ago + 1 back fence 1A2B Tower 592TXR temperature 8.4 C humidity 88% ok 41 0:07 + 2 mast 0777 5-in-1 06014RM wind 14.2 km/h wind from 248° WSW ok 36 0:11 + rain 41.4 mm temperature 8.1 C + humidity 90% + 3 greenhouse 0C41 Tower 592TXR temperature 15.9 C humidity 61% ok 39 0:04 + 4 unnamed 0311 Lightning 6045M temperature 8.6 C humidity 87% ok 24 0:19 + strikes 4 storm 19 km + 5 shed 5C 609TXC temperature 3.1 C humidity 94% low 21 0:12 + 6 unnamed 93 606TX temperature -18.2 C ok 19 0:22 +``` + +### Naming things while they arrive + +A sensor broadcasts an identity, and that identity is a number that came out +of a hat in a factory — or a different number out of the same hat the next +time the batteries were changed. It is enough to tell one sensor from another +and it is no use at all for telling which is which. `1A2B` is not a place. + +**The identity is shown in both bases**, because the same number gets written +two ways. These identities are bit fields with the channel packed above them, +so this program prints them in hexadecimal where that shape shows; rtl_433 and +everything built on it prints them in decimal. `3935` and `14645` are the same +sensor. Anyone arriving with a list of their own sensors already has it in +decimal and has no reason to convert it, so `bandsaunter sensors` puts the two +side by side and either can be typed at `--name`: + +``` +name id decimal key model ch msgs last heard +back fence 3935 14645 tower/3935 Tower 592TXR A 412 2026-09-20 14:17 +— 271B 10011 tower/271B Tower 592TXR B 408 2026-09-20 14:17 +— 21E5 8677 tower/21E5 Tower 592TXR C 405 2026-09-20 14:16 +``` + +The one case that needs care is an identity that is a valid number in both +bases — `3935` is hexadecimal 3935 and also decimal 3935, which are different +sensors. If both are out there it says so and asks for the whole key rather +than picking one. + +So **press `n` while listening**. The display comes down, the sensors are +listed with numbers, you pick one and type a name, and it goes back up. The +receiver keeps running throughout: a slow typist loses a few seconds of +weather and nothing else. + +That is the moment it is possible to do. The sensor is on the screen saying +3.1 degrees, and the person watching is the one who knows that the cold one is +the shed. An hour later it is a list of hexadecimal again. + +Names can also be given from the command line, before or after anything has +been heard: + +``` +bandsaunter weather --name 1A2B="back fence" --name 5C=shed +bandsaunter sensors --name 14645="back fence" # decimal works too +bandsaunter sensors --name 0311="lightning detector" --note 0311="south gable" +bandsaunter sensors --forget 93 +``` + +A name given before the sensor has ever been received waits under its +identity alone — nothing yet knows which model it is, because that is in the +message and there has not been one — and moves across the moment the first +message arrives. So naming the garden from a list and then listening works, +and the display says `shed` from the first reception rather than listing the +shed and the sensor on it as two separate things. + +They live in `sensors.yaml` beside the settings, one entry per sensor, meant +to be opened and edited by hand — it is a list of things in a garden, and +typing them is often quicker than tagging them one at a time off the air. +Names are written the moment they are given rather than when the program +exits, because it exits on control-C; and they are written to a neighbouring +file which is then renamed over the old one, so a machine losing power halfway +through leaves either the old names or the new ones and never half of each. + +Nothing is ever dropped for being stale. A sensor whose battery ran out two +winters ago keeps its name, because the alternative is that putting a battery +back in loses it. + +### What it reads + +Five families, each with its own framing and its own check: + +| model | bytes | what it says | +|---|---|---| +| Tower 592TXR / 06002RM | 7 | temperature, humidity | +| 5-in-1 06014RM / VN1TXC | 8 | wind speed, wind direction, rainfall / wind speed, temperature, humidity | +| Lightning 6045M | 9 | temperature, humidity, strike count, how far off the storm is | +| 609TXC | 5 | temperature, humidity | +| 606TX | 4 | temperature | + +Battery state comes from all of them. The 5-in-1 has more to say than fits in +one message and alternates two of them, so its last message is always either +the wind and the rain or the temperature and the humidity, never both — the +display keeps the newest value of each quantity rather than the newest +message, so the line is the whole sensor rather than half of it flickering. + +**What is not read.** The Atlas, the 986 and 515 fridge thermometers, the +00275rm room monitor and the 899 standalone rain gauge. They are on the same +band and are not decoded. A message from one of them whose framing happens to +match is reported as an unknown message type with its identity and nothing +else, rather than guessed at — because knowing that something is out there +transmitting, with an identity that stays the same, is worth a line on its +own. `--no-unknown` leaves them off. + +These formats are implemented from their published descriptions and are +checked against frames built from the same descriptions — which proves the +framing, the parity, the checksums and the arithmetic, and proves nothing +about anything the description and this both get wrong. The tower sensor is +also checked against five messages recovered from real hardware, byte for +byte, and those are worth more than all the rest put together: they are what +caught the parity, which no amount of testing an encoder against its own +decoder ever could. + +### Why nothing false gets through + +433 MHz is a crowded band. Doorbells, car keys, tyre-pressure sensors, garage +doors and the neighbours' weather station are all on it, and a decoder that +looks at every bit offset of every burst will find a message in noise if it is +allowed to. Four things stop it. + +**The checks the message carries.** The three newer models have an eight-bit +sum plus **even** parity in the top bit of every payload byte — twelve to +fourteen bits of check on a message of seven to nine bytes. Even, not odd: +that one bit of convention was wrong here for four days and the cost was that +nothing decoded at all, every other check passing and saying so. + +**Arriving twice.** The two older models carry one byte of check between them, +which is one false message in two hundred and fifty-six, and that is not a +rate a display can be trusted at. So those two are only believed when the same +message arrives twice. It costs nothing: these sensors send everything three +times in a row, for exactly this reason. + +**A plausibility range.** A checksum can be satisfied by a message the +hardware could not have sent. Nothing outside −40 to 70 °C, 0 to 100% or a +wind the anemometer cannot physically report is accepted. + +**How many readings are tried.** The readings of a burst are tried in order of +likelihood and the search stops at the first that yields anything, rather than +pooling all of them. Half a dozen readings at two byte orders is sixteen times +as many chances for a coincidence to satisfy a checksum, and twelve bits of +check is a rate that is comfortable once and uncomfortable sixteen times over +— sensors that were not there began appearing the moment the alternatives went +in. Stopping early costs nothing: a burst that reads correctly the ordinary +way never reaches the alternatives, and one that does not reaches them exactly +as before. + +**Where the message sits.** This is the one that is easy to get wrong. A +seven-byte message read out of the front of a real eight-byte one is made of +that message's own payload bytes, whose parity is *already correct* — so the +parity bits contribute nothing at all and what is left is one byte of sum. +Corroboration does not help either, because the copies of a message are +identical, so a coincidence in one copy is a coincidence in all three. What +gives that window away every time is that it ends a whole byte before the +burst does. A real message begins after the sync and runs to the end. + +### Off the air + +The receiver is tuned straight at 433.92 MHz, at 250 kS/s, with the tuner's own +gain control doing its job and the RTL2832's digital AGC left off — which is +what the established tools for this band do, and there is nothing to be gained +by differing from them. + +Each of those was once something else, and each was wrong. Tuning to one side +and shifting the signal back in software avoids the spike every RTL-SDR puts at +whatever it is tuned to, which sounds worth doing until you notice that a +filter narrow enough to reject that spike is narrow enough to lose a +transmitter that has drifted — and that which way round a dongle presents its +samples is not a thing to depend on. The spike is a steady addition to the +envelope and the burst rises clear of it. `--offset` is still there and does +nothing at all at the default rate: shifting a signal only matters if something +afterwards is narrower than the band, and at 250 kS/s nothing is. + +The digital AGC is off because it pumps. It winds the gain up through the +silence between one burst and the next, which lifts the noise towards the +signal and squeezes the very difference this depends on. It matters here in a +way it does not for aircraft, where a frame is found by correlating a preamble +over a few microseconds rather than by comparing a burst with the quiet around +it. + +**Finding the bursts is done in two passes, and the reason is having more than +one sensor.** The first pass only asks where anything is happening at all, and +asks it against the noise — the bottom fifth of the second, which is noise +however busy the rest was. Whatever clears that is grouped into regions, and +the second pass re-thresholds each region against its own high and low, so +every sensor is sliced at its own amplitude. + +The obvious way to write this is one threshold per second, set halfway between +the noise floor and the loudest thing in it. That is wrong, and wrong in a way +worth describing because the symptom is so odd: a sensor on the windowsill and +a sensor at the end of the garden differ by forty decibels, so a threshold +halfway to the near one sits above everything the far one ever does — and the +far ones disappear, completely, and only while the near one is transmitting. A +block with one loud sensor in it yields exactly one sensor however many are out +there. + +**Slicing the envelope into bits never measures anything against a clock**, +and assumes as little as it can about how a bit is drawn — nor about which end +of a byte goes first, both orders being tried and the checksums saying which. +(There is a fingerprint for that one: reversing the bits of a byte does not +change how many are set, so parity survives it and a sum does not. A message +read from the wrong end shows every parity holding and every checksum failing.) +Nor about how long the silences in it are: the gaps inside one message range from 200 µs on the +newer sensors to 4000 µs on the oldest, so what belongs to one transmission is +settled from each burst's own widest gap rather than from a figure that would +have to suit every model at once. Anything that comes out longer than the +longest message there is gets cut at its largest gaps, because that is what a +boundary between two copies physically is. + +Three further things are not assumed. Which of the pulse and the gap carries the bit — the newer sensors +vary the pulse, the older two vary the gap. Whether the gap is the complement +of the pulse, so that every bit takes the same time, or just a fixed spacer: +judged against a fixed 200 µs spacer a short pulse of 220 µs is longer than its +own gap and reads as a one, which is the wrong bit, and every message then +fails its checksum saying nothing about why. And which of long and short means +one. + +So the same burst is read half a dozen ways — the pulse against its own gap, +the pulse against each threshold the pulse lengths themselves suggest, the +same for the gaps, and each of those inverted — and the checksums say which +reading it was. At most one of them can satisfy a checksum. It costs a few +microseconds per burst. + +A transmitter running ten per cent fast is therefore read correctly and never +noticed, which matters: these are unlocked and drift tens of kilohertz and a +few per cent of rate with the temperature. An outdoor sensor in January is not +the one that was on the fence in July. + +### Afterwards + +``` +sensors heard +name id model ch msgs every battery last heard +back fence 1A2B Tower 592TXR A 148 16 s ok 23:14:07 +mast 0777 5-in-1 06014RM A 131 18 s ok 23:14:02 +greenhouse 0C41 Tower 592TXR B 140 17 s ok 23:14:05 +shed 5C 609TXC 76 30 s low 23:13:58 + +what they said +sensor reading first last lowest highest +back fence temperature 8.4 C 6.1 C 5.9 C 9.2 C + humidity 88% 94% 86% 94% +mast wind 14.2 km/h 3.1 km/h 0.0 km/h 38.6 km/h + wind from 248° WSW 202° SSW + rain 41.4 mm 43.9 mm 41.4 mm 43.9 mm +``` + +The two tables are split on purpose. The first is about reception — who, how +often, how well — and is the one to look at when something is missing. The +second is about the weather, and is the one to look at when nothing is. + +### How well each sensor is heard + +Three columns of the first table answer that, and they answer different halves +of it. + +**`signal`** is how far the sensor's burst stood above the noise, in decibels, +coloured red below 14 dB, amber below 22, green above. It is a ratio of two +amplitudes off the same receiver in the same second and nothing more — *not* a +power at the aerial, which an RTL-SDR cannot give: it has no reference level, +and on automatic gain it does not have a fixed gain either. What a ratio is +good for is comparing one sensor with another at the same moment, watching one +sensor over an evening, and pointing an aerial. Set `--gain 40` (or any fixed +figure) if you want the numbers comparable between one run and the next; +on automatic they drift with whatever the tuner decided. + +It is on the live display too, and kept there even on a narrow terminal, +because moving a whip around while watching a number go up is the single most +useful thing it does. + +**`every`** is the average wait between messages, and **`heard`** turns that +into a share. These transmit on a fixed cycle, so the shortest wait ever seen +between two of a sensor's messages *is* that cycle, and the average wait is +the cycle divided by the fraction getting through — one over the other is +therefore the share, without ever needing to know what model it is or how +often it is supposed to speak. + +The two are worth reading together, and they can disagree usefully. A strong +signal with a low share is interference or a collision with another +transmitter rather than a range problem. A weak signal at a hundred per cent +is a sensor at the edge that is nonetheless getting through, and is worth +leaving alone. + +There is no average. These arrive every sixteen seconds when the sensor is in +range and not at all when it is not, and rain and cold both shorten the range +of a 433 MHz transmitter — so the mean of what was received is the mean of a +sample whose gaps are themselves the weather, and it would read like a number +when it is not one. A bearing gets no lowest or highest either: north is 0 and +also 360. + +`--csv`, or `bandsaunter readings --csv` afterwards, writes a column per +quantity and a row per reading, with the sensor's name in the second column +and the unit in the heading rather than beside every number — a column of +"21.5 C" is text, and a column of 21.5 is a temperature. + +The log keeps the raw bytes of every message underneath whatever was made of +them, because the message is the evidence and the rest of the line is an +opinion about it. A later version of this program that reads a model this one +cannot will be able to go back through old logs and read them properly. + +Readings are converted once, on the way in, to degrees Celsius, kilometres an +hour, millimetres and kilometres — different models report in different units, +and the tower sends Celsius while the 5-in-1 and the lightning detector send +Fahrenheit. The log holds the converted value, so `--units imperial` changes +only what is shown and can be changed afterwards on an old log. + +### Every weather option + +| option | flag | default | what it does | +|---|---|---|---| +| Receiver | `--device` | 0 | which receiver, when more than one is plugged in | +| Gain | `--gain` | auto | tuner gain in dB, or automatic | +| Sample rate | `--rate` | 250 kS/s | how fast to sample; this is also the minimum | +| Sensors on | `--frequency`, `--freq` | 433.92 MHz | where the sensors transmit | +| Tuning offset | `--offset` | 0 | how far to one side of them to tune; does nothing at the default rate | +| Invent a garden | `--simulate` / `--no-simulate` | no | six sensors that are not there | +| Listen for | `--seconds` | until stopped | how long before stopping | +| Write a log | `--log` / `--no-log` | yes | one line of JSON per message | +| Print every message | `--messages` / `--no-messages` | no | a stream of lines instead of a table | +| Say what is arriving | `--diagnose` / `--no-diagnose` | no | each second taken apart stage by stage | +| Keep on screen for | `--hold` | 1800 s | how long a sensor stays after its last message | +| Show readings in | `--units` | metric | metric or imperial, for the display and the export | +| — | `--window` | — | open the live map instead of a table | +| Map reaches | `--radius` | 50 km | how far around the aerial the window reaches | +| Theme | `--theme` | night | how the window looks; the same five as the aircraft map | +| Map brightness | `--map-brightness` | 70 | how bright the ground under the stations is | +| Draw a real map | `--basemap` / `--no-basemap` | yes | fetch map tiles | +| Range rings | `--window-rings` / `--no-window-rings` | yes | faint discs around the aerial | +| Box translucency | `--box-opacity` | 85 | how solid the card behind each box is | +| Leave a trail | `--trails` / `--no-trails` | yes | the path behind anything that moved | +| Map tiles from | `--tiles` | OpenStreetMap | where tiles come from | +| Only named sensors | `--only-named` / `--all-sensors` | no | ignore anything without a name | +| Show unreadable models | `--unknown` / `--no-unknown` | yes | list sensors whose model cannot be read | +| Report at the end | `--report` / `--no-report` | yes | print what each sensor said | +| Also write a spreadsheet | `--csv` / `--no-csv` | no | CSV beside the log | + +`--name ID=NAME` is not a setting: it names a sensor and is repeatable. +Neither is `--save-iq FILE`, a one-off capture of the raw samples, nor +`--from-iq FILE`, which reads one back instead of the receiver. + +Saved in `weather.yaml` beside the other settings, from the menu's **s** or by +hand. `bandsaunter readings` also takes `--sensor NAME` to narrow a log to one +sensor, and `--no-report` to write the spreadsheet and say nothing. + +### Without a sensor + +`--simulate` puts six sensors on a fence that does not exist — one of every +model — transmitting real messages with real checksums, keyed on and off as a +real one does, through the real filter, the real slicer and the real decoders. +Nothing touches the receiver. The naming, the log, the report and the export +can all be tried before an aerial exists. + +The weather moves at roughly a real afternoon's pace, which is slower than it +sounds like it should be: about a degree an hour of drift, wind gusting around +a mean, rain falling a tenth of a millimetre at a time. A garden that swung +six degrees a minute would exercise everything downstream perfectly well and +would look, to anyone running `--simulate` to see what the program does, like +a program that cannot read a thermometer. + +### If nothing is heard + +**`bandsaunter weather --diagnose` is the answer to this**, because "nothing +was heard" is four different faults wearing the same coat and they want four +different answers. It prints each second of band taken apart stage by stage: + +``` + 12.4s noise 0.0219 gate 0.0450 peak 1.0179 (46.4x the noise) 3 bursts + 60 pulses pulses 219×29 399×27 597×4 gaps 221×26 402×29 600×4 + Tower 592TXR 1A2B ch A temperature 8.4 C humidity 88% +``` + +Read it from the left. + +**`peak` is barely above `noise`, no bursts.** Nothing is arriving. That is an +aerial. These are a few milliwatts: a quarter-wave whip for 433.92 MHz is 17 cm +of wire, which is the stock telescopic aerial collapsed to about that, and +indoors behind a wall with the dongle in the back of a machine is usually the +problem. Try `--gain 40` if the automatic gain control is not finding them. +Once anything at all is being heard, the `signal` column on the live display +is the thing to watch while moving the aerial about. + +**`peak` is well above `noise` and there are no bursts.** Something is there +and did not group — usually a continuous transmitter rather than a keyed one, +which is not one of these. + +**Bursts, but the pulse lengths are not two or three clean groups.** The +receiver is hearing it and the slicing is wrong. A real message shows two or +three lengths with nothing in between, like the `219×29 399×27 597×4` above. +A smear of lengths means noise is being sliced as signal, or two sensors are +transmitting over each other. + +**Bursts with clean pulse lengths and `nothing framed`.** The radio is fine +and the message is from a model this does not read, or reads differently. +Under it comes the closest thing to a message that was found and which of its +checks held: + +``` + 61 pulses pulses 216×36 403×21 612×4 gaps 209×21 395×35 601×4 + nothing framed 8 ways of reading it tried + closest: 7 bytes at bit 4: ✓sum ✗parity ✓type ✗plausible DA 2B C4 B0 89 BF C1 +``` + +Which check fails says what kind of fault it is. A sum that holds and a parity +that does not is a different thing from neither holding, and the bytes are +printed so the arithmetic can be done by hand. That line and the pulse lengths +above it are between them everything needed to add a format. + +**`framed, but this model needs the same message twice`.** It was read +correctly and arrived once. The two older models are only believed on a second +copy; a stronger signal fixes it. + +When the listening stops it says which of those it was, once: + +``` +╭───────────────────────── what that came to ──────────────────────────╮ +│ 40 bursts were received and sliced, and not one of them framed as a │ +│ message. The radio end is working: what is arriving is a model this │ +│ cannot read, or reads differently. The pulse lengths printed above │ +│ are exactly what is needed to add it. │ +╰──────────────────────────────────────────────────────────────────────╯ +``` + +`--save-iq FILE` writes the raw samples alongside, for working out anything the +diagnosis cannot. It is 2 MB a second at the default rate, so bound it with +`--seconds 60` — that is plenty, since every sensor reports at least twice in +a minute. It writes the settings it was taken at beside it, because a file of +raw samples with no idea what rate it was recorded at cannot be read back by +anything: at the wrong rate every pulse in it is the wrong length. + +`--from-iq FILE` reads one back instead of the receiver, so a recording made +where the aerial is can be worked on anywhere, as many times as it takes, with +different settings each time. Everything downstream is the real thing — the +same filter, the same slicer, the same decoders — because the only part being +stood in for is the dongle. That is what tells a receiver problem and a +decoder problem apart: a capture that yields nothing on replay yields nothing +for anybody, and a capture that yields readings on replay but not on the air +is a setting. + +## APRS on 144 MHz + +One channel, one frequency, everybody. 144.390 MHz across North America and a +different number in every other region, carrying position reports, weather, +messages, objects and telemetry from every amateur station within earshot — +and from every hilltop digipeater repeating them onward, which is most of what +you will actually hear. + +```bash +bandsaunter aprs listen, and show who is out there +bandsaunter aprs --region europe ...on 144.800 instead +bandsaunter aprs --window a live map of the stations, which accumulate +bandsaunter aprs --simulate a channel full of stations that are not there +bandsaunter packets --kml turn a log into something Google Earth opens +``` + +It is a section of its own for the same reason the other two are: a scan stops +on a signal, records it and moves on, and APRS is a two-second transmission +every few minutes from a hundred stations sharing one frequency. A sweep +catches whichever one happened to key up while the sweep was pointed there. + +Unlike the others this is a conversation rather than a broadcast. Stations +address each other, acknowledge each other and relay for each other, so what +is worth showing is not only who is out there but what was said. And nothing +here has to be named: a weather sensor broadcasts a number out of a hat, but +an APRS station broadcasts a callsign issued by a government, which is already +the name. + +``` +╭──────────────────────────────────────────────────────────────────────────────────────────────╮ +│ 144.39 MHz 6 stations 12 seen 148 packets 9 messages 41:02 aprs_2026-09-20.jsonl │ +╰──────────────────────────────────────────────────────────────────────────────────────────────╯ +station what said away signal pkts ago +KU0W-9 car 47.5570N 122.2783W ENE 54 km/h mobile 2 km ENE 38 dB 41 0:08 +KB1XYZ weather station 47.5900N 122.2700W temperature 15.0 C 5 km NNE 34 dB 12 0:22 + humidity 62% pressure 1013.2 hPa +W1AW-1 digipeater 47.6062N 122.3322W Seattle wide digi 7 km NNW 37 dB 4 1:14 +EVENT1 fire 47.5900N 122.3300W 5 km NNW 36 dB 1 3:40 +``` + +### Which channel + +The frequency is agreed between amateurs rather than allocated, so it differs +by region and **there is no way to discover it from the air**: on the wrong one +you hear silence, not a bad signal. That is the one fault that looks exactly +like a dead aerial and is not, so it is the first thing to settle. + +```bash +bandsaunter aprs --find-channel # listen on each in turn, 20 s each +bandsaunter aprs --region europe # or set it, if you know +``` + +`--find-channel` answers the one question about APRS that cannot be answered +on any single frequency, because the answer *is* a frequency: + +``` +what was on each channel +region frequency packets stations strongest used in +north-america 144.390 MHz 21 6 38 dB United States, Canada, Mexico +europe 144.800 MHz — — — IARU Region 1, including the UK +australia 145.175 MHz — — — Australia and New Zealand +japan 144.640 MHz — — — Japan +brazil 145.570 MHz — — — Brazil +thailand 145.525 MHz — — — Thailand +north-america had the most on it — 21 packets from 6 stations +``` + +A quiet channel is not proof of an empty one — a fixed station beacons every +half hour — so what it finds is traffic, and what it misses is only the absence +of traffic while it listened. It says so when everything comes back silent. + +In the menus the channel is on the front page rather than a level down with the +gain and the sample rate, because it is the setting that decides whether +anything is heard at all: + +``` + l Listen north-america 144.39 MHz, until stopped, everything, metric + c Channel / region north-america 144.39 MHz + f Find the channel listen on each region's in turn and see which has traffic + r Read a log back 3 logs in ~/bandsaunter +``` + +**c** lists the regions with their frequencies and also takes a number in MHz +for a channel that is not any region's — a local packet network, or one of the +9600 baud links some areas run alongside the main one. **f** runs the search +and offers to adopt whichever channel had the most on it. + +### What it reads + +The information field of an APRS packet is not one format. It is about twenty, +chosen by the first character, accreted over thirty years, and several of them +exist only because a particular radio shipped with them. + +| what | how it arrives | +|---|---| +| Position | uncompressed, or compressed into thirteen characters of base-91 | +| Mic-E | every Kenwood and Yaesu mobile, with half the position hidden in the destination callsign | +| Weather | attached to a position, or positionless | +| Messages | to a station, acknowledged, rejected, or broadcast as a bulletin | +| Objects and items | something placed on the map that is not the station placing it | +| Status | what the operator typed | +| Telemetry | five analogue channels and eight bits | +| Third-party traffic | a packet relayed in from another network, credited to whoever originally sent it | + +Riding in the comment: course and speed, altitude, transmitter power and +antenna height, pre-computed range, direction-finding reports, and the +precision extension. + +**Mic-E deserves a note**, because it is a quarter of everything on the channel +and it is the least readable thing in amateur radio. In 1995 the destination +address of an APRS frame carried nothing but the word "APRS", and somebody +noticed that six bytes is exactly enough for a latitude. So a Mic-E packet puts +the latitude, the north/south bit, the east/west bit, a hundred degrees of +longitude and a three-bit status message into *the callsign it is addressed +to*, and the rest into the information field as characters chosen so the whole +thing survives being typed into a logbook. It is also the reason APRS fits in a +two-second transmission. + +### Refusing to guess + +A packet whose format does not match what its first character promised comes +back as unparsed, with its text kept, rather than as a position. + +That sounds like a nicety and is not. Thirteen characters of a *malformed* +uncompressed position are perfectly good base-91, so a decoder that tries one +format and falls back to the other does not fail on a bad packet — it succeeds, +as a confident and completely different place, usually a thousand miles away. +The specification makes the two unambiguous (a leading digit always means +uncompressed, which is why the compressed format writes a numeric overlay as a +letter), so the rule is read rather than guessed at. + +`--unparsed` is on by default: a packet that defeats every parser here still +arrived, still came from a real station, and showing it is how the next format +gets added. + +### Off the air + +Four things happen between the aerial and a frame. + +**The tones become a soft symbol.** APRS is Bell 202 — 1200 Hz for a mark, +2200 Hz for a space, twelve hundred a second, inside an ordinary FM +transmission. Two correlators, one at each tone, and the difference between +them. A correlator rather than a frequency discriminator because the tones are +less than an octave apart and radio audio is distorted enough that +instantaneous frequency wanders badly. + +De-emphasis is turned off, which matters and is easy to miss: voice FM lifts +the high frequencies on transmit and drops them again on receive, and packet +radio takes its audio from the discriminator *before* that happens. Dropping +the highs would take four decibels off the 2200 Hz tone and leave the 1200 Hz +one alone, which is exactly the difference being measured. + +**The soft symbol becomes bits**, sampled once a bit at an instant held in the +middle of the bit by a loop nudged at every zero crossing. The loop carries its +phase from one block of audio to the next, which is what makes this a receiver +rather than a decoder of recordings — a packet is most of a second and a block +is about one, so frames straddling the boundary are not an edge case, they are +most of them. + +**The bits become a frame.** NRZI first, where a zero is a change of tone and a +one is no change — which makes the whole thing immune to being wired up +backwards, since inverting the stream decodes to identical data. Then HDLC: +frames delimited by 01111110, with a zero stuffed after every five ones so the +flag cannot occur inside one. + +**The frame is believed or it is not.** Sixteen bits of CRC, and nothing +without a correct one is reported. That is what makes it safe to leave running +for hours with the squelch open: a frame either checks out or it never existed. +A frame whose callsigns are unprintable is refused as well — on a band this +busy, sixteen bits is strong but not infinite. + +It decodes down to about 8 dB of signal-to-noise and finds nothing at all in +pure noise. + +### A window, while it happens + +`bandsaunter aprs --window` opens the same map the aircraft use, with the +stations on it instead of aeroplanes — a real map underneath, an information +box beside each station, a leader line to the mark it belongs to, range rings +around the aerial and a red flag where you are. + +``` +144.39 MHz 6 on the map 6 seen 812 packets 1:01:40 d detail t trails g map f full q quit + ┌──────────────────┐ + ┌────────────────────┐ ◈────────┤ W1AW-1 │ + │ K7XYZ-3 │ │ │ symbol digipeater│ + │ symbol yacht │ │ │ away 7 km NNW │ + │ away 19 km NNW │ ◈╌╌╌╌╌╌╯ │ says Seattle… │ + │ says yagi │ ╎ └──────────────────┘ + └────────────────────┘ ╎ ⌂ +``` + +**The one difference from the aircraft map is that nothing fades.** An +aeroplane that stops transmitting has flown out of range, and drawing it an +hour later where it was would be drawing something that is certainly not +there. A fixed amateur station that stops transmitting is still exactly where +it was — it beacons every half hour, and the gaps are silence rather than +absence. So the picture accumulates, and an evening of listening fills a map. + +Marks are drawn by what they are, and coloured off the same altitude ramp the +aircraft use — which is the one set of colours every theme defines, so a +digipeater stays distinguishable from a car on all five: + +| what | drawn as | +|---|---| +| something moving | a body with a stalk, pointing where it is going | +| a digipeater or gateway | a diamond, brightest of the fixed things | +| a weather station | a diamond, mid-ramp | +| an object somebody placed | a diamond, low | +| a fixed station | a diamond, dimmest | + +A diamond because it is the one shape on the picture with no front: a house +that beacons twice an hour is a place, and drawing it as a triangle would +have it pointing north for no reason. + +The box says what the station is, **where it is in figures**, how far off and +in which bearing, what it is doing if it is moving, its altitude, its weather, +its status or comment, the digipeaters it came through, how many packets and +how many of those arrived directly, and how strongly. The mark shows where a +station is; the figures are what gets written down, read out over the air or +typed into something else — and where a station blanked its minutes, the +figures are the only place the vagueness shows (`49.0000N 72.0000W ±340 km`), +because a mark on a map is as definite at ten miles of doubt as at ten yards. +Boxes are placed where they cover nothing else and glide when their station +moves, and where there is no room for one the mark is still drawn — with an +evening's accumulation there are usually more marks than there is room for +boxes, so the most recently heard get them. + +The same keys as the aircraft map: `d` cycles how much each box says, `t` +trails, `g` the map underneath, `f` true full screen, `[` and `]` its +brightness, `+`/`-` the range, `q` quits. It opens maximised, as that one +does. `--radius` sets how far it reaches to begin with and `--theme` picks +from the same five. + +**The window measures in whatever unit you are being shown.** `--units +imperial` puts statute miles round the rings, along the scale at the bottom +and on `--radius` itself, and `--units metric` puts kilometres on all three. +The rings are then *drawn* at the distance they are labelled: the outermost +one at `--radius 100 --units imperial` stands seventy-five statute miles from +the red flag, measured on the ground, not seventy-five kilometres with miles +written beside it. A ring is the thing a distance gets judged against by eye, +so a ring labelled in one unit and drawn in another is a wrong answer given +confidently — and the tables printed afterwards have always followed this +setting, so the window disagreeing with them was the window being wrong. + +**Where the aerial is** puts the red flag on the map, centres the range rings +and gives every station a distance and a bearing. Set it with `--at LAT,LON`, +or in the menu under **Receiver at** — and if you have already told the +aircraft side where you are, that is used and nothing needs saying twice. One +aerial on one roof does not move because the receiver was pointed at a +different band. It says which it used, because an inherited position is a +convenience right up until you have moved and changed only one of them. + +### Who each station is + +An APRS callsign is issued by a government, so unlike a weather sensor it can +be looked up. Each station heard is resolved once — the name on the licence, +the town, and the licensed position, which is a street address where a beacon +only gives a grid square — and the answer appears in the live table, in the +information box on the map and in the report. + +``` +station licensed to what away +W1AW-9 ARRL HQ Operators Club — Newington, CT car 3140 km E +KU0W Rod R Gowdy — Tucson, AZ digipeater 12 km NNE +VK4BLE-7 Australia car 13800 km W +``` + +Three things make it behave: + +- **The SSID comes off first.** `W1AW-9` is the ninth radio W1AW runs — a car, + a digipeater, a weather box — and a digipeated path marks the hop it came + through with a star. No licence register has heard of either. +- **Nothing waits.** The lookup runs on its own thread and the name appears in + a later frame; a table that stopped for a network request would stop for + every new station on a busy channel. +- **A station the register cannot know still says where it is from.** The + databases are United States registers, so `VK4BLE` comes back unlisted — and + the country beside it comes out of the callsign's own structure, needing no + network at all. A blank cell would read as a lookup that failed rather than + as a question that was never going to be answered. + +Objects are not looked up. An object is a marker one station placed on behalf +of something with no licence of its own — a net, a hilltop, a storm — so +asking who it is licensed to is asking the wrong question. + +**Answers are cached** in `~/.cache/bandsaunter/callsigns.json` for a month, +and that file is shared with every other part of this program that resolves a +callsign: the scanner's identification, the recording browser and this. The +same net logged night after night is asked about once. `--no-lookup` turns the +network off entirely and leaves the country and district, which cost nothing. + +### Afterwards + +Three tables, because they answer different questions. **Stations heard** is +about the band and the aerial: where each was, how far off, how many packets, +how many of those arrived *directly* rather than through a digipeater, and how +strongly. **What they said** is the weather, the speeds and the status lines. +**What passed between them** is the messages, in order — the only part of APRS +that is a conversation. + +`--at LAT,LON` turns on the distance and bearing columns. `--direct-only` +leaves out anything that reached you through a digipeater, which is a much +shorter list and is the honest measure of what your aerial can actually reach. + +`--csv` writes a row per packet with position, speed and weather in their own +columns. `--kml` writes a pin where each station was last heard and a line for +anything that moved. Both can be made later from a log with `bandsaunter +packets --csv --kml`, and `--station CALL` narrows either to one callsign. + +The log keeps the whole AX.25 frame in hexadecimal under whatever was made of +it. That is not a hypothetical precaution: the list of APRS formats is still +growing, so a packet this version cannot read is on the disk in full for a +version that can. + +### Every APRS option + +| option | flag | default | what it does | +|---|---|---|---| +| Receiver | `--device` | 0 | which receiver, when more than one is plugged in | +| Gain | `--gain` | auto | tuner gain in dB, or automatic | +| Sample rate | `--rate` | 240 kS/s | 96 kS/s is the least that holds the channel | +| Region | `--region` | north-america | which APRS channel to listen on | +| — | `--find-channel [SECONDS]` | — | listen on each region's in turn and say which has traffic | +| Listen on | `--frequency`, `--freq` | 144.390 MHz | the exact frequency, if the region's is not what you want | +| Invent a channel | `--simulate` / `--no-simulate` | no | stations that are not there | +| Listen for | `--seconds` | until stopped | how long before stopping | +| Write a log | `--log` / `--no-log` | yes | one line of JSON per packet | +| Print every packet | `--packets` / `--no-packets` | no | a stream of lines instead of a table | +| Keep on screen for | `--hold` | 3600 s | how long a station stays after its last packet | +| Receiver at | `--at` | the aircraft setting | where the aerial is: the flag, the rings and the distances | +| Show readings in | `--units` | metric | metric or imperial, for the display and the export | +| — | `--window` | — | open the live map instead of a table | +| Map reaches | `--radius` | 50 km | how far around the aerial the window reaches | +| Theme | `--theme` | night | how the window looks; the same five as the aircraft map | +| Map brightness | `--map-brightness` | 70 | how bright the ground under the stations is | +| Draw a real map | `--basemap` / `--no-basemap` | yes | fetch map tiles | +| Range rings | `--window-rings` / `--no-window-rings` | yes | faint discs around the aerial | +| Box translucency | `--box-opacity` | 85 | how solid the card behind each box is | +| Leave a trail | `--trails` / `--no-trails` | yes | the path behind anything that moved | +| Map tiles from | `--tiles` | OpenStreetMap | where tiles come from | +| Show what cannot be read | `--unparsed` / `--no-unparsed` | yes | list packets in unknown formats | +| Include relayed packets | `--digipeated` / `--direct-only` | yes | count what reached you through a digipeater | +| Report at the end | `--report` / `--no-report` | yes | print what was heard | +| Also write a spreadsheet | `--csv` / `--no-csv` | no | CSV beside the log | +| Also write a map | `--kml` / `--no-kml` | no | KML beside the log | + +Saved in `aprs.yaml` beside the other settings, from the menu's **s** or by +hand. + +### Without an aerial + +`--simulate` puts a dozen stations on a channel that does not exist — cars +moving, a weather station, a digipeater, objects, messages passing between +them — keyed as real Bell 202 audio on a real FM carrier, through the real +demodulator and the real parsers. Nothing touches the receiver. + +### If nothing is heard + +**Check the region first**, and `bandsaunter aprs --find-channel` does it for +you. It is the one fault that looks like a dead aerial and is not: on the wrong +channel there is silence rather than a bad signal. + +Then give it time. A fixed station beacons every twenty or thirty minutes and a +mobile every minute or two, so five minutes of an ordinary suburb might be +three packets and an hour is a fair picture. If you are somewhere without a +digipeater in range you may genuinely hear nothing — APRS coverage is built +from volunteers' hilltops, and it has holes. + +A quarter-wave whip for 144 MHz is 49 cm, which is longer than the aerial most +dongles ship with; the stock telescopic one extended properly does well. +`--packets` shows each frame as it arrives, which is the thing to watch while +moving an aerial about. + +## FT8 + +Fifteen seconds of everybody at once. + +``` +bandsaunter ft8 --band 20m --grid IO91 +``` + +Every station on the band transmits in the same quarter-minute slots, on the +same dial frequency, fifty hertz wide each, stacked across three kilohertz of +audio. So one receiver parked on one frequency hears the whole band's worth +of stations simultaneously — and hears most of them **well below the noise**. + +``` +FT8 20m — 14.074 MHz 12 slots 187 decodes 63 stations 15.6 a slot 3:00 from IO91 + +last slot — 16 decoded +snr dt hz message +-19 -0.5 309 G4CUS SP4FCA RRR + -6 0.1 528 VK3EVE SQ3MZM RR73 + 18 0.1 1110 9A9TT IK4LZH -10 + 6 0.0 2535 CQ IZ3XJM JN55 + +heard so far — 63 stations +station grid away n best hz last +IZ3XJM JN55 1180 km 4 6 2535 2s +IK4LZH JN54 1140 km 7 18 1110 2s +``` + +Half of what is transmitted is error-correcting code, and that is the whole +trick: it is what buys a mode that decodes twenty-odd decibels under what an +operator can hear. A receiver that took the loudest tone of each symbol and +hoped would decode almost nothing. + +### What it needs + +**The clock has to be right**, to a second or two. The slots are +quarter-minutes of UTC and every station on earth agrees about which one it +is. A receiver a second out still decodes; one a slot out hears every +transmission split across two captures and decodes none of them. This is the +one failure that looks exactly like a dead band, so the display and the +report both say so when nothing arrives. + +**Almost all the activity is on shortwave**, which a plain dongle cannot +reach. The default here is therefore the two-metre channel at 144.174 MHz, +which it can. All thirteen channels are in the list and the shortwave ones +work perfectly well through an upconverter or a receiver in direct-sampling +mode — the menu says which is which rather than leaving you to find out by +listening to silence. + +### What comes out + +`--grid IO91` is what turns decodes into geography: every station calling CQ +says where it is, so with your own square filled in the report gives each one +a distance and a bearing, and names the furthest heard. On shortwave that is +the entire interest of the thing. + +Three tables afterwards: **stations heard** with grid, distance, count and +best and worst report; **calling CQ**, which is who is available; and **who +was working whom**, which is the band as a social event rather than a list. + +`--adif` writes the log again in the form every amateur logging program +imports — marked as *heard*, not worked. Nothing here transmits, so nothing +here is a contact, and an ADIF that let a logging program treat these as +worked would put claims into somebody's log they cannot make. + +### Who each station is + +Every FT8 exchange is two callsigns, and a callsign is issued by a government, +so it can be looked up. Both stations in an exchange are resolved — the one +being answered may never transmit within earshot and is still a station this +receiver knows about — and the name appears in the live table and the report. + +``` +station licensed to grid away decodes best +KU0W Rod R Gowdy — Tucson, AZ DM42 0 km 0° 1 -7 +VK4BLE Australia QG62 12125 km 249° 1 -7 +W1AW ARRL HQ Operators Club — Newington, CT FN31 3489 km 62° 1 -7 +``` + +The licensed callsign is found inside the heard one: `ET3RFG/R` is `ET3RFG` +operating away from home, `DL/G4ABC` is `G4ABC` as a guest in Germany, and a +hashed `<...>` — a compound callsign this receiver has not yet heard spelled +out — is not asked about at all, because there is nothing to ask. Those rules +are shared with APRS rather than written twice, since getting them wrong is +silent: a register asked about `W1AW-9` returns nothing, which looks exactly +like a station that is not licensed. + +Nothing waits on the network. A slot has to be decoded in well under fifteen +seconds or the next one is missed, so the lookup runs on its own thread and +the name appears in a later slot. + +**The databases are United States registers**, so the DX that makes this mode +worth listening to comes back unlisted — and the country beside it comes out +of the callsign's own structure, needing no network. On a band whose whole +interest is distance, that is most of what a name would have added. + +### How well it works + +Checked against eleven off-air recordings with published decodes, which is +the only honest way to test a decoder — an encoder tested against its own +decoder agrees with it about anything they are both wrong about, a lesson +this project learned expensively on 433 MHz. + +| | | +|---|---| +| messages decoded | 97 of 150 (65%), **no false decodes** | +| timing | +0.00 s, spread 0.04 | +| frequency | +0.1 Hz, spread 1.1 | +| signal report | −0.4 dB, spread 4.8 | +| weakest decoded | −24 dB on this program's own scale | + +The 35% not decoded are the weakest signals in each slot. A mature decoder +subtracts each signal it decodes and looks again in what is left, and does +ordered-statistics decoding when belief propagation fails; neither is built +here. What *is* here decodes nothing that other receivers did not also hear, +which is the property that matters in a log. + +## Meters on 900 MHz + +A scan of 902–928 MHz that turns up a burst gets it named rather than reported +as hex. ``` 915.0 MHz decoded: electricity meter 12345678 reading 987654 -433.92 MHz decoded: AcuRite sensor 1234 temperature 21.5 C humidity 48% channel A +433.92 MHz decoded: Tower 592TXR 1A2B temperature 21.5 C humidity 48% channel A ``` **Utility meters.** The Itron ERT modules fitted to electricity, gas and water meters across North America broadcast their reading every thirty seconds or so on 902–928 MHz, in the clear, so a van can drive past and read a street. The message says which meter, what kind, what the register reads, and whether the -tamper switches have been tripped. +tamper switches have been tripped. It is not guessed at: the message carries a +16-bit BCH check and nothing is reported that has not satisfied it. -**AcuRite sensors.** The 433.92 MHz outdoor sensors sold with every consumer -weather station send temperature, humidity, battery state and a channel letter -every sixteen seconds. - -Neither is guessed at: a meter message carries a 16-bit BCH check and a sensor -message a checksum and four parity bits, and nothing is reported that has not -satisfied them. Both are implemented from their published descriptions and -checked against frames built from the same descriptions — which proves the -framing and the arithmetic, and is not the same as having held a meter. +**Weather sensors.** A scan of 433 MHz that catches one of these names it the +same way, using the same decoders as +[the weather section](#weather-sensors-on-433-mhz) — which is where they +belong, because a scan catches one burst and the weather is a night of them. +The two thinly-checked models are not reported from a scan at all: they need +the same message twice before they are believed, and a scan hears one. ## Hex into words @@ -2216,6 +3385,67 @@ takes effect on the next scan; one already running read its settings when it started. Locking a frequency out does not delete what has already been recorded on it, so pressing `m` and then `d` is the usual thing to do. +### The register — everything ever looked up + +`c` opens it. The scanner's identification, the Morse idents, APRS, FT8 and +this browser all resolve callsigns into one file, and the aircraft side keeps +its own. Between them they are a record of who and what has been heard from +this aerial — and until now the only way to read either was to open the JSON. + +``` +╭─ the register ───────────────────────────────────────────────────────────────╮ +│ callsigns — 3 of 62 shown, 3 with somewhere to point a map at 'tucson' │ +╰──────────────────────────────────────────────────────── tab → aircraft ──────╯ + callsign who where looked up map +› AB7IC Thomas G Britton Tucson, AZ 26-09-23 05:32 ◉ + KC7UGN Roger F Wheaton Tucson, AZ 26-09-23 05:31 ◉ + KU0W Rod R Gowdy Tucson, AZ 26-09-23 22:59 ◉ + +╭─ AB7IC ──────────────────────────────────────────────────────────────────────╮ +│ licensed to Thomas G Britton district district 7 (Northwest) │ +│ class EXTRA grid DM42og │ +│ licence PERSON expires 12/21/2033 │ +│ street 8410 E Brookwood Dr position 32.27504, -110.81312 │ +│ town Tucson, AZ status found in a register │ +│ postcode 85750-2468 answered by callook.info │ +│ country United States looked up 2026-09-23 05:32 │ +╰────────────────────────────────── g opens this place in a browser ───────────╯ +↑↓ move tab switch / search g map c or esc back q quit +``` + +| key | | +|---|---| +| `↑` `↓` | move; Page Up/Down a screen, Home/End to the ends | +| `tab` | switch between callsigns and aircraft | +| `/` | search **every field**, not just the name | +| `g` | open where this one is, in a browser | +| `r` | read the caches again, after a scan in another window | +| `c` `esc` | back to the recordings | + +**`g` opens a map.** Only the coordinates go into the link — a map does not +need to be told whose licence it is looking at in order to show a place, and +the link is the one part of this that leaves the machine. Where nothing is +known about the position it says so rather than guessing, and on a machine +with no browser, which is the normal case for a receiver, it prints the +coordinates instead. + +**Searching looks at everything**, because the interesting question is usually +not *which callsign* but "who was in Arizona" or "what Bombardiers have gone +over", and both of those live in the detail. Every word has to match, so a +search can be narrowed. + +Callsigns no register could place are **kept rather than dropped**. "Asked +about, and in no register reachable from here" is a fact about a station, and a +list that quietly dropped them would make an evening of unlisted foreign +stations look like an evening when nothing was looked up. An aircraft is put on +the map at the airport its flight started from, labelled as such: this is a +register of airframes rather than a position report, and the origin is the +nearest true thing. + +The addresses are licence records, public by law and already printed in the +scanner's own reports; this shows them the same way rather than more +prominently. + ### Detected callsigns Under the transcript, every callsign heard in it is listed with the name and @@ -2533,6 +3763,8 @@ restricted. Check your local rules. ## Licence +The source is at . + Copyright © 2026 The Dust Council. bandsaunter is free software: you can redistribute it and modify it under the @@ -2541,3 +3773,21 @@ by the Free Software Foundation. It is distributed in the hope that it will be useful, but with no warranty whatsoever — not even the implied warranty of merchantability or fitness for a particular purpose. The full text is in [LICENSE](LICENSE), and at . + +### Borrowed material + +One file is not original work. `bandsaunter/ft8tables.py` holds the two fixed +tables that *define* the FT8 error-correcting code — the generator and the +sparse parity-check matrix — taken from +[ft8_lib](https://github.com/kgoba/ft8_lib), MIT licensed, copyright © 2018 +Kārlis Goba, which took them in turn from WSJT-X. They are reproduced under +the MIT terms, which permit it, and that file carries the attribution they +ask for. + +They are there because they cannot be derived. Everything else about FT8 in +this program is worked out from first principles — the tones, the sync, the +parity arithmetic, the belief propagation, the way a callsign is squeezed +into twenty-eight bits — but those two tables are not derived from anything. +They are the code itself, chosen once by its designers and published, and a +receiver that guessed at them would be speaking a different protocol. No +decoding logic was taken. diff --git a/bandsaunter/__init__.py b/bandsaunter/__init__.py index 2f8d679..6aad171 100755 --- a/bandsaunter/__init__.py +++ b/bandsaunter/__init__.py @@ -8,8 +8,8 @@ and transcribing speech. # Versions are the release date and a revision within that day, so # 2026-08-21_02 is the second build made on the 21st. The revision is padded # to two digits so versions sort as text. -VERSION_DATE = "2026-09-06" -VERSION_REVISION = 3 +VERSION_DATE = "2026-09-24" +VERSION_REVISION = 1 __version__ = f"{VERSION_DATE}_{VERSION_REVISION:02d}" diff --git a/bandsaunter/acurite.py b/bandsaunter/acurite.py new file mode 100644 index 0000000..79402ce --- /dev/null +++ b/bandsaunter/acurite.py @@ -0,0 +1,1937 @@ +"""AcuRite weather sensors: the messages they send, and how to get them off +the air. + +A consumer weather station is two things. The display on the kitchen wall is +one of them; the other is a plastic box on a fence post that says what it can +see every sixteen seconds or so, in the clear, on 433.92 MHz, to anyone who +happens to be listening. This reads the box. + +**What is here.** Five families, each with its own framing and its own check: + +=========================== ===== ================================== +model bytes what it says +=========================== ===== ================================== +Tower 592TXR / 06002RM 7 temperature, humidity +5-in-1 06014RM / VN1TXC 8 wind, direction, rain / wind, temperature, humidity +Lightning 6045M 9 temperature, humidity, strikes, how far off +609TXC 5 temperature, humidity +606TX 4 temperature +=========================== ===== ================================== + +The first three share a framing -- two bytes of identity, a byte saying what +kind of message this is, the payload, and a checksum which is the sum of +everything before it -- and the payload bytes each carry even parity in their +top bit. That is between twelve and fourteen bits of check on a message of +seven to nine bytes, which is enough to accept a message on a band where +doorbells, car keys, tyre sensors and someone's garage all transmit. + +The last two are older and cheaper and carry one byte of check between them: +a sum on the 609 and a CRC-8 on the 606. Eight bits is one false message in +two hundred and fifty-six tries, and this looks at every bit offset of every +burst, so eight bits on its own is not enough. Those two are therefore only +believed when the *same* message arrives twice in one burst -- which costs +nothing, because these sensors send every message three times in a row for +exactly this reason. + +**What is not here.** The Atlas, the 986 and 515 fridge thermometers, the +00275rm room monitor and the 899 standalone rain gauge. They are on the same +band and are not read. A message from one of them whose framing happens to +match is reported as an unknown message type with its identity and nothing +else, rather than guessed at. + +That last part is not a nicety. The Atlas is nine bytes like the lightning +detector and lays its payload out differently, and nothing but the message +type tells them apart -- so every decoder here insists on a type it knows, +and a nine-byte message that is not a 6045M is reported as a sensor of an +unknown model rather than as a lightning detector reading a temperature off +the wrong bits. Wrong weather under somebody's sensor name is a worse +answer than no weather at all. + +**Where the numbers come from.** These formats are implemented from their +published descriptions and are checked here against frames built from the +same descriptions. That proves the framing, the parity, the checksums and +the arithmetic; it is not the same as having held every one of these +sensors. The reason it is nonetheless safe to run is that nothing is +reported which has not satisfied its own check. + +**Units.** Different models report in different units -- the tower sends +tenths of a degree Celsius, the 5-in-1 and the lightning detector send tenths +of a degree Fahrenheit, and the rain gauge sends hundredths of an inch. All +of it is converted here, once, to Celsius, kilometres an hour, millimetres +and kilometres, so that a log holds one kind of number and the person reading +it can be shown whichever they prefer. The raw counts survive alongside the +converted values where they mean something on their own, which for the rain +gauge they do: it is a tipping bucket and the count only ever goes up. +""" + +from __future__ import annotations + +import math +from dataclasses import dataclass, field + +import numpy as np + +__all__ = ["Reading", "Measure", "format_measure", + "decode", "decode_burst", "candidates", "confirmed", + "readings_from", "MODELS", "MODEL_NAMES", "CHANNELS", "ACURITE_HZ", + "tower_frame", "five_in_one_wind_rain", "five_in_one_weather", + "lightning_frame", "frame_609", "frame_606", + "bursts", "Burst", "bits_pwm", "bits_ppm", "slicings", "pulse_train", + "modulate", "baseband", "WIND_POINTS", "compass", "parity8", + "crc8", "VirtualSensor", "SimulatedSensors", "default_sensors", + "survey", "Survey", "timings", "near_misses", "NearMiss", + "BIT_ORDERS"] + + +# Where every one of these sensors transmits. Nominally 433.92 MHz; the +# cheap transmitters in them drift by tens of kilohertz with temperature, +# which is why what follows works on the envelope of a band rather than on a +# carrier at an exact frequency. +ACURITE_HZ = 433_920_000.0 + + +# --------------------------------------------------------------------------- +# The small arithmetic every one of these depends on +# --------------------------------------------------------------------------- + +def parity8(value: int) -> int: + """Odd-parity bit of one byte: 1 when an odd number of bits are set.""" + value ^= value >> 4 + value ^= value >> 2 + value ^= value >> 1 + return value & 1 + + +def crc8(data, poly: int = 0x07, init: int = 0x00) -> int: + """The plain byte-at-a-time CRC-8 the 606TX carries.""" + crc = init + for byte in data: + crc ^= byte + for _ in range(8): + crc = ((crc << 1) ^ poly) & 0xFF if crc & 0x80 else (crc << 1) & 0xFF + return crc + + +# The channel switch on the back of a tower sensor has three positions, and +# they are not encoded in the order anyone would guess: A is 3, B is 2 and C +# is 0. Code 1 is not a switch position at all; the published descriptions +# call it E and so does this, rather than pretending it is a D. +CHANNELS = ("C", "E", "B", "A") + +# The sixteen wind directions, in the order the sensor numbers them, which is +# also not the order anyone would guess. Index it with the low nibble. +WIND_POINTS = (315.0, 247.5, 292.5, 270.0, 337.5, 225.0, 0.0, 202.5, + 67.5, 135.0, 90.0, 112.5, 45.0, 157.5, 22.5, 180.0) + +_COMPASS = ("N", "NNE", "NE", "ENE", "E", "ESE", "SE", "SSE", + "S", "SSW", "SW", "WSW", "W", "WNW", "NW", "NNW") + + +def compass(degrees: float) -> str: + """The sixteen-point name for a bearing, for people rather than plots.""" + return _COMPASS[int((float(degrees) % 360.0) / 22.5 + 0.5) % 16] + + +# One tip of the bucket, in millimetres. The gauge counts hundredths of an +# inch, which is 0.254 mm, and the count is cumulative for the life of the +# battery. +RAIN_PER_COUNT_MM = 0.254 + +# What a sensor can physically report. A checksum can be satisfied by a +# message the hardware could not have sent, and on a band this crowded that +# happens often enough to be worth catching. +TEMPERATURE_RANGE = (-40.0, 70.0) # Celsius +HUMIDITY_RANGE = (0, 100) # per cent +WIND_RANGE = (0.0, 260.0) # km/h; the sensor tops out below this + +# The message type a 6045M sends. Insisted on rather than assumed from the +# length; see :func:`decode_lightning` for why. +LIGHTNING_MESSAGE = 0x2F + +# Every message type any decoder here recognises, for saying whether a burst +# that framed correctly is one of them. +_KNOWN_TYPES = frozenset({0x04, 0x31, 0x38, LIGHTNING_MESSAGE}) + +# Where in a burst a message is allowed to sit. A real one begins after the +# sync and runs to the end, so a handful of bits of sync in front of it and +# almost nothing behind. See :func:`candidates`, where these do the work. +LEAD_BITS = 8 # room for twice the four sync pulses these sensors send +TAIL_BITS = 4 # room for a couple of stray edges at the end +# A message this cannot read is held to the tighter figures: it has no type +# field that means anything here and no range that can be checked, so where +# it sits is the only evidence there is that it is a message at all. +UNREAD_LEAD_BITS = 4 +UNREAD_TAIL_BITS = 1 + + +def _fahrenheit(f: float) -> float: + return (f - 32.0) * 5.0 / 9.0 + + +# --------------------------------------------------------------------------- +# What one message came to +# --------------------------------------------------------------------------- + +@dataclass(frozen=True) +class Measure: + """One quantity a sensor reported, with the unit it is held in.""" + + name: str + value: float + unit: str = "" + # What the sensor actually sent, where that is a different number from + # the one above: the rain counter, the strike distance in miles. Kept + # because it is the evidence and the converted value is a restatement. + raw: float | None = None + + @property + def text(self) -> str: + return format_measure(self) + + +def format_measure(measure: Measure, imperial: bool = False) -> str: + """One measurement as a person would write it, in either system.""" + name, value, unit = measure.name, measure.value, measure.unit + if unit == "C": + return f"{value * 9 / 5 + 32:.1f} F" if imperial else f"{value:.1f} C" + if unit == "%": + return f"{value:.0f}%" + if unit == "km/h": + return f"{value / 1.609344:.1f} mph" if imperial else f"{value:.1f} km/h" + if unit == "mm": + return f"{value / 25.4:.2f} in" if imperial else f"{value:.1f} mm" + if unit == "km": + return f"{value / 1.609344:.0f} mi" if imperial else f"{value:.0f} km" + if unit == "deg": + return f"{value:.0f}° {compass(value)}" + if unit == "lux": + return f"{value:.0f} lux" + if unit: + return f"{value:g} {unit}" + # A bare count -- strikes, bucket tips -- reads better without a ".0". + return f"{value:g}" if value != int(value) else f"{int(value)}" + + +@dataclass +class Reading: + """One message that satisfied its own checks, and what it said. + + The bits are kept because they are the evidence; everything else in here + is an opinion about them, and the two should not be confused. + """ + + model: str = "" # what to call it to a person + family: str = "" # what to file it under: tower, 5n1, ... + sensor: str = "" # the identity in the message, as hex + channel: str = "" # A, B or C, where the model has a switch + battery_low: bool | None = None + message: int = 0 # the message-type field, as sent + measures: tuple = () + bits: str = "" + checks: tuple = () + at: float = 0.0 # when it arrived, as a clock time + copies: int = 1 # how many times it arrived, identically + offset: int = 0 # where in the burst its first bit was + snr: float = 0.0 # decibels above the noise; 0 = not known + + @property + def key(self) -> str: + """What a friendly name is attached to. + + Family and identity, not channel: the channel switch is on the + outside of the box and someone will move it, and moving it should not + lose the name they gave the box. + """ + return f"{self.family}/{self.sensor}" + + def value(self, name: str): + """One measurement by name, or None.""" + for measure in self.measures: + if measure.name == name: + return measure.value + return None + + def describe(self, imperial: bool = False) -> str: + head = f"{self.model} {self.sensor}" + if self.channel: + head += f" ch {self.channel}" + rest = " ".join(f"{m.name} {format_measure(m, imperial)}" + for m in self.measures) + if self.battery_low: + rest += " battery low" + if not self.measures: + rest = rest or f"message type {self.message:#04x}, not understood" + return f"{head} {rest}".strip() + + +# Every family, in the order they are tried, with how many copies of a +# message must arrive in one burst before it is believed. See the module +# docstring: it is a function of how much check the message carries. +MODELS = ( + ("tower", "Tower 592TXR", 7, 1), + ("5n1", "5-in-1 06014RM", 8, 1), + ("6045", "Lightning 6045M", 9, 1), + ("609", "609TXC", 5, 2), + ("606", "606TX", 4, 2), +) + +MODEL_NAMES = {family: name for family, name, _bytes, _r in MODELS} +MESSAGE_BYTES = {family: n for family, _name, n, _r in MODELS} +MESSAGE_FAMILY = {n: family for family, _name, n, _r in MODELS} +REPEATS_NEEDED = {family: repeats for family, _name, _bytes, repeats in MODELS} + + +# --------------------------------------------------------------------------- +# The three that share a framing: tower, 5-in-1, lightning detector +# --------------------------------------------------------------------------- + +def _txr_framed(data: list[int]) -> bool: + """The checks the whole TXR family carries: a sum, and even parity. + + The sum covers every byte before it. The parity is in the top bit of + each payload byte -- everything but the two identity bytes and the + checksum itself -- and is *even*: the bit is set so that the number of + bits in the byte comes out even. + + Even rather than odd, which is worth saying plainly because this had it + the other way round and nothing whatever was decoded for it. Five + messages off five real sensors settled it: twenty payload bytes, every + one of them even, which is not something twenty bytes do by chance. + + One consequence has to be handled here rather than assumed away. Odd + parity rejects a byte of all zeroes and even parity accepts it, so a run + of silence read as zeroes satisfies both this and a sum of zero -- which + is why the emptiness test below is a check and not a nicety. + """ + if (sum(data[:-1]) & 0xFF) != data[-1]: + return False + if not any(data): + return False + return all(parity8(byte) == 0 for byte in data[2:-1]) + + +def _txr_identity(data: list[int], id_bits: int) -> tuple[str, str]: + """The channel letter and the sensor identity, as the family encodes it.""" + channel = CHANNELS[(data[0] >> 6) & 0x03] + mask = (1 << (id_bits - 8)) - 1 + return channel, f"{((data[0] & mask) << 8) | data[1]:04X}" + + +def decode_tower(data: list[int]) -> Reading | None: + """The 592TXR / 06002RM outdoor sensor: temperature and humidity. + + Fourteen bits of identity with the channel above them, a status byte + whose bit 6 is set while the battery is good, humidity in seven bits, and + temperature in eleven -- tenths of a degree Celsius offset by a hundred, + so that the coldest thing it can report is still a positive number. + + This one has been held against messages off real sensors rather than only + against messages built here; see the tests. + """ + if len(data) != 7 or not _txr_framed(data): + return None + if (data[2] & 0x3F) != 0x04: + return _unknown("tower", data, 14) + + channel, sensor = _txr_identity(data, 14) + humidity = data[3] & 0x7F + raw = ((data[4] & 0x0F) << 7) | (data[5] & 0x7F) + celsius = raw / 10.0 - 100.0 + if not _sane_temperature(celsius) or not _sane_humidity(humidity): + return None + return Reading( + model="Tower 592TXR", family="tower", sensor=sensor, channel=channel, + battery_low=not data[2] & 0x40, message=data[2] & 0x3F, + measures=(Measure("temperature", round(celsius, 1), "C"), + Measure("humidity", float(humidity), "%")), + checks=("checksum-8", "parity")) + + +def decode_5n1(data: list[int]) -> Reading | None: + """The 5-in-1, which says half of what it knows at a time. + + It alternates two messages, because there is more to say than fits in one + of them: a wind message with the direction and the rain counter, and a + weather message with the temperature and the humidity. Both carry the + wind speed, which is the one number worth having on every transmission. + """ + if len(data) != 8 or not _txr_framed(data): + return None + kind = data[2] & 0x3F + channel, sensor = _txr_identity(data, 12) + common = dict(model="5-in-1 06014RM", family="5n1", sensor=sensor, + channel=channel, battery_low=not data[2] & 0x40, + message=kind, checks=("checksum-8", "parity")) + + # The same eight bits in both messages: five from one byte and three + # from the next. Zero means stopped; anything else is a count the + # published description turns into kilometres an hour with a slope and + # an offset, and the offset is why a turning cup never reads zero. + raw_wind = ((data[3] & 0x1F) << 3) | ((data[4] & 0x70) >> 4) + wind = 0.0 if raw_wind == 0 else raw_wind * 0.8278 + 1.0 + if not WIND_RANGE[0] <= wind <= WIND_RANGE[1]: + return None + + if kind == 0x31: # wind, where from, and rain + degrees = WIND_POINTS[data[4] & 0x0F] + counter = ((data[5] & 0x7F) << 7) | (data[6] & 0x7F) + return Reading( + measures=(Measure("wind", round(wind, 1), "km/h"), + Measure("wind from", degrees, "deg"), + Measure("rain", round(counter * RAIN_PER_COUNT_MM, 2), + "mm", raw=float(counter))), + **common) + if kind == 0x38: # wind, temperature and humidity + raw = ((data[4] & 0x0F) << 7) | (data[5] & 0x7F) + celsius = _fahrenheit((raw - 400) / 10.0) + humidity = data[6] & 0x7F + if not _sane_temperature(celsius) or not _sane_humidity(humidity): + return None + return Reading( + measures=(Measure("temperature", round(celsius, 1), "C"), + Measure("humidity", float(humidity), "%"), + Measure("wind", round(wind, 1), "km/h")), + **common) + return _unknown("5n1", data, 12) + + +def decode_lightning(data: list[int]) -> Reading | None: + """The 6045M, which is a tower sensor that also counts lightning. + + The strike counter is cumulative and wraps at 127, and the distance is + the storm's, in miles, as the detector estimates it -- zero to thirty-one, + where thirty-one means "further away than this can tell". There is also + a bit which is set when the detector believes it is being interfered with, + and it is worth showing, because a strike count that climbs while that + bit is set is not lightning. + + The message type is insisted on, and that is the whole reason this can be + run at all. The Atlas is a nine-byte sensor too and lays its payload out + differently, and nothing but the type field tells them apart: decoded as + this, an Atlas would give a temperature read off the wrong bits, which + would pass the checksum, pass the parity, and land inside a plausible + range often enough to be believed. Wrong weather under somebody's sensor + name is a worse answer than none, so anything else nine bytes long comes + back as a sensor of a model this cannot read. + """ + if len(data) != 9 or not _txr_framed(data): + return None + kind = data[2] & 0x3F + if kind != LIGHTNING_MESSAGE: + return _unknown("6045", data, 12) + channel, sensor = _txr_identity(data, 12) + humidity = data[3] & 0x7F + raw = ((data[4] & 0x1F) << 7) | (data[5] & 0x7F) + celsius = _fahrenheit(raw / 10.0 - 100.0) + if not _sane_temperature(celsius) or not _sane_humidity(humidity): + return _unknown("6045", data, 12) + + strikes = data[6] & 0x7F + miles = data[7] & 0x1F + measures = [Measure("temperature", round(celsius, 1), "C"), + Measure("humidity", float(humidity), "%"), + Measure("strikes", float(strikes), "")] + if miles < 0x1F: + measures.append(Measure("storm", round(miles * 1.609344, 1), "km", + raw=float(miles))) + flags = ["checksum-8", "parity"] + if data[7] & 0x20: + flags.append("interference") + return Reading( + model="Lightning 6045M", family="6045", sensor=sensor, channel=channel, + battery_low=not data[2] & 0x40, message=kind, + measures=tuple(measures), checks=tuple(flags)) + + +def _unknown(family: str, data: list[int], id_bits: int) -> Reading: + """A message that framed correctly and is not one this can read. + + Reported rather than dropped, because knowing that there is a sensor of + some kind on the fence, with an identity that stays the same, is worth + something on its own -- it can be given a name and watched -- and because + silently discarding a well-formed message is how a decoder comes to look + like a receiver problem. + """ + channel, sensor = _txr_identity(data, id_bits) + return Reading(model=f"AcuRite {len(data)}-byte sensor", + family=family, sensor=sensor, channel=channel, + battery_low=not data[2] & 0x40, message=data[2] & 0x3F, + measures=(), checks=("checksum-8", "parity")) + + +# --------------------------------------------------------------------------- +# The two older ones +# --------------------------------------------------------------------------- + +def _signed12(high: int, low: int) -> int: + """Twelve bits of two's-complement temperature, in tenths of a degree.""" + raw = ((high & 0x0F) << 8) | low + return raw - 0x1000 if raw & 0x800 else raw + + +def decode_609(data: list[int]) -> Reading | None: + """The 609TXC: one byte of identity, temperature, humidity, a sum. + + The identity is only eight bits and is redrawn at random every time the + battery is changed, so two of these in one garden will collide about once + in every two hundred and fifty battery changes. Nothing can be done + about that from here; it is worth knowing when a sensor appears to have + changed its mind about the weather. + """ + if len(data) != 5 or (sum(data[:4]) & 0xFF) != data[4] or not any(data): + return None + celsius = _signed12(data[1], data[2]) / 10.0 + humidity = data[3] + if not _sane_temperature(celsius) or not _sane_humidity(humidity): + return None + return Reading( + model="609TXC", family="609", sensor=f"{data[0]:02X}", + battery_low=not data[1] & 0x80, + measures=(Measure("temperature", round(celsius, 1), "C"), + Measure("humidity", float(humidity), "%")), + checks=("checksum-8", "twice")) + + +def decode_606(data: list[int]) -> Reading | None: + """The 606TX: identity, temperature, a CRC-8, and nothing else at all.""" + if len(data) != 4 or crc8(data[:3]) != data[3] or not any(data): + return None + celsius = _signed12(data[1], data[2]) / 10.0 + if not _sane_temperature(celsius): + return None + return Reading( + model="606TX", family="606", sensor=f"{data[0]:02X}", + battery_low=not data[1] & 0x80, + measures=(Measure("temperature", round(celsius, 1), "C"),), + checks=("crc-8", "twice")) + + +def _sane_temperature(celsius: float) -> bool: + return TEMPERATURE_RANGE[0] <= celsius <= TEMPERATURE_RANGE[1] + + +def _sane_humidity(humidity: float) -> bool: + return HUMIDITY_RANGE[0] <= humidity <= HUMIDITY_RANGE[1] + + +_DECODERS = ((7, decode_tower), (8, decode_5n1), (9, decode_lightning), + (5, decode_609), (4, decode_606)) + + +# --------------------------------------------------------------------------- +# From bits to readings +# --------------------------------------------------------------------------- + +# Which end of a byte goes down the air first. Most significant bit first is +# what the descriptions of these formats assume, and it is what nearly +# everything does, but "nearly" is the word that has cost this section three +# rounds already -- so both are tried and the checksums say which. +# +# There is a signature worth knowing for this one. Reversing the bits of a +# byte does not change how many of them are set, so parity survives it and a +# sum does not: a message read from the wrong end shows its parity holding +# and its checksum failing, every time, on every copy. +BIT_ORDERS = (False, True) + +_REVERSED = [int(f"{byte:08b}"[::-1], 2) for byte in range(256)] + + +def _bytes_at(bits: str, at: int, count: int, + reflect: bool = False) -> list[int]: + out = [int(bits[at + i * 8:at + (i + 1) * 8], 2) for i in range(count)] + return [_REVERSED[byte] for byte in out] if reflect else out + + +def candidates(bits: str) -> list[Reading]: + """Every message that can be read out of a run of bits, at any offset. + + Every offset, because what arrives here has a sync pattern of some length + in front of it and the message does not begin on a byte boundary of the + recovered bits. Every length, because a burst does not say which model + sent it. The checks are what make that affordable: a message which + satisfies a sum, a parity set and a plausibility range at a bit offset it + does not belong at is rare enough to be worth the search. + """ + found: list = [] + for count, decoder in _DECODERS: + # A message has to sit where a message sits: a few bits of sync in + # front of it, and the end of the burst immediately behind it. This + # is the check that stops a short model being read out of a long one. + # + # It is needed because neither of the other two defences reaches the + # case. Corroboration does not: the copies of a message are + # identical, so a coincidence in one copy is a coincidence in all + # three, and asking for it three times over asks for nothing. Parity + # does not either, and this is the part that is easy to get wrong -- + # a seven-byte window taken from the front of a real eight-byte + # message is made of that message's own payload bytes, whose parity + # is already correct, so the four parity bits contribute nothing at + # all and what is left is one byte of sum. One in two hundred and + # fifty-six is not a rate at which a display can be trusted. + # + # Where it sits, though, gives that window away every time: it ends + # a whole byte before the burst does. + need = count * 8 + for at in range(0, len(bits) - need + 1): + if at > LEAD_BITS or len(bits) - at - need > TAIL_BITS: + continue + if not _fits(at, need, len(bits)): + continue + for reflect in BIT_ORDERS: + try: + data = _bytes_at(bits, at, count, reflect) + except ValueError: + break + reading = decoder(data) + if reading is None: + continue + if not reading.measures and not _fits(at, need, len(bits), + unread=True): + continue + reading.bits = bits[at:at + need] + reading.offset = at + found.append((at, need, reading)) + break # one order or the other, not both + return _one_per_window(found, len(bits)) + + +def _fits(at: int, need: int, length: int, unread: bool = False) -> bool: + """Whether a message of this length can sit at this offset in a burst. + + A message begins after the sync and runs to the end of the burst, so a + handful of bits in front of it and almost none behind. One that cannot + be read is held to the tighter figures: it has no type field that means + anything here and no range that can be checked, so where it sits is the + only evidence there is that it is a message at all. + """ + lead = UNREAD_LEAD_BITS if unread else LEAD_BITS + tail = UNREAD_TAIL_BITS if unread else TAIL_BITS + return at <= lead and length - at - need <= tail + + +def _one_per_window(found: list, length: int) -> list[Reading]: + """Of two messages sharing bits, keep one: they cannot both be real. + + A message forty bits long, found in a burst forty-one bits long, can be + read at two offsets, and one byte of checksum is satisfied by the wrong + one of them about once in every two hundred and fifty-six bursts. Both + survive corroboration, because both are in all three copies. What tells + them apart is that a real message ends where the burst does, and the + other one leaves a bit hanging off the end. + + Longer messages and better-understood ones are preferred first, so that + a burst which really does hold an eight-byte message is never given up + for a five-byte window inside it. + """ + found.sort(key=lambda item: (len(item[2].measures), item[1], + -(length - item[0] - item[1]), -item[0]), + reverse=True) + kept: list[Reading] = [] + taken: list[tuple[int, int]] = [] + for at, need, reading in found: + if any(at < end and start < at + need for start, end in taken): + continue + taken.append((at, at + need)) + kept.append(reading) + return kept + + +def decode(bits: str, confirm: bool = True) -> Reading | None: + """The one message a run of bits holds, or None. + + Where a model's checks are thin -- the two older ones carry a single + byte between them -- the same message has to turn up more than once + before it is believed, and one run of bits is one copy. So by default + this reads the three well-checked models and not the two thin ones; the + thin ones are reached through :func:`readings_from`, which sees a whole + block and so can see the copies. + + ``confirm=False`` reads whatever framed correctly, for a caller with its + own reason to believe a single message: a stored log, or a test holding + the encoder that made it. + """ + return _best(candidates(bits), confirm) + + +def _best(found: list[Reading], confirm: bool = True) -> Reading | None: + """The most-corroborated message in a pile of candidates. + + Ordered by how many copies arrived, then by how much check the model + carries, then by how long the message is -- a longer message that framed + correctly is better evidence than a shorter one that did, and the + longest ones are also the ones with the most to say. + """ + if not found: + return None + seen: dict[tuple, list[Reading]] = {} + for reading in found: + seen.setdefault((reading.family, reading.bits), []).append(reading) + good = [(copies, group[0]) for (family, _bits), group in seen.items() + if (copies := len(group)) >= (REPEATS_NEEDED[family] if confirm + else 1)] + if not good: + return None + good.sort(key=lambda pair: (pair[0], len(pair[1].bits), + len(pair[1].measures)), reverse=True) + return good[0][1] + + +def decode_burst(burst: "Burst") -> Reading | None: + """One burst of pulses, read as both codings, decoded as whatever it is. + + These sensors do not agree on how a bit is drawn. The newer ones vary + the length of the pulse and keep the gap between pulses roughly even; + the two older ones keep the pulse even and vary the gap. Rather than + decide which from the timings -- which is guessing, and wrong on a weak + burst where the edges have moved -- both readings of the same pulses are + tried, and the checksums say which one it was. + """ + return _best(_from_burst(burst)) + + +def _from_burst(burst: "Burst") -> list[Reading]: + """The messages in one burst, from the first reading of it that yields any. + + The readings are tried in order of likelihood and the search stops at the + first one that produces anything, rather than pooling all of them. That + matters more than it looks. + + Half a dozen readings of a burst, each tried at both byte orders, is + sixteen times as many chances for a coincidence to satisfy a checksum as + one reading was -- and the checks here are twelve to fourteen bits, which + is a rate that is comfortable once and uncomfortable sixteen times over. + Sensors that are not there began appearing in the invented garden the + moment the alternatives went in. + + Stopping early costs nothing. A burst that reads correctly under the + ordinary reading never reaches the alternatives; a burst that does not + reach them exactly as it did before. The fallbacks are there for the + sensor whose bits are drawn some other way, and that sensor is not in + competition with anything. + """ + for bits in slicings(burst): + found = candidates(bits) + if found: + return found + return [] + + + + + +# --------------------------------------------------------------------------- +# From the air to bits: an on-off-keyed envelope, sliced +# --------------------------------------------------------------------------- + +@dataclass +class Burst: + """One run of pulses with no long silence in it. + + Marks and spaces alternate and are in microseconds, starting with a mark; + there is always one more mark than there are spaces, because the silence + that ended the burst is not part of it. + """ + + at: float = 0.0 # seconds from the start of the block + marks: tuple = () + spaces: tuple = () + level: float = 0.0 # how far above the noise floor, as a ratio + + @property + def pulses(self) -> int: + return len(self.marks) + + @property + def decibels(self) -> float: + """How far the burst stood above the noise, in dB. + + A ratio of two amplitudes off the same receiver in the same second, + which is the only honest thing to say about strength here. It is not + a power at the aerial and cannot be: a dongle has no reference level, + and with the tuner left on automatic it does not have a fixed gain + either. What it is good for is comparing one sensor with another at + the same moment, watching one sensor over an evening, and pointing an + aerial -- which is most of what anybody wants a signal reading for. + """ + if self.level <= 0.0 or self.level == float("inf"): + return 0.0 + return 20.0 * math.log10(self.level) + + @property + def length_us(self) -> float: + return float(sum(self.marks) + sum(self.spaces)) + + +def baseband(iq: np.ndarray, sample_rate: float, offset: float = 0.0, + want_rate: float = 250_000.0) -> tuple[np.ndarray, float]: + """The envelope of the sensor's channel, at a rate worth slicing. + + Three things happen here, in this order and for this reason. + + The receiver is tuned a little to one side of 433.92 MHz, because every + RTL-SDR puts a spike of its own at whatever it is tuned to and a spike + sitting on top of an on-off-keyed signal is the one thing that stops it + being on and off. So the first step is to shift the sensor back to the + middle, which puts the spike out at the edge instead. + + The second is a plain running average of the complex samples, long enough + that its first null lands on the spike. It is a crude filter and a + deliberately crude one: it is four instructions wide, it runs on a whole + second of samples without noticing, and what it has to reject is one + tone at a frequency this end chose. + + Only then is the magnitude taken. Filtering before detection rather than + after is what keeps the neighbours -- a doorbell, a tyre sensor, a car + key -- from adding themselves to the envelope of the sensor. + """ + if iq.size == 0: + return np.zeros(0, dtype=np.float32), float(sample_rate) + signal = np.asarray(iq) + if offset: + turn = np.exp(-2j * math.pi * offset + * np.arange(signal.size, dtype=np.float64) / sample_rate) + signal = signal * turn.astype(np.complex64) + decimate = max(1, int(sample_rate // max(1.0, want_rate))) + if decimate > 1: + usable = (signal.size // decimate) * decimate + if usable == 0: + return np.zeros(0, dtype=np.float32), float(sample_rate) + signal = signal[:usable].reshape(-1, decimate).mean(axis=1) + return np.abs(signal).astype(np.float32), float(sample_rate) / decimate + + +def bursts(envelope: np.ndarray, rate: float, gap_us: float = 3_000.0, + min_pulses: int = 8, min_run_us: float = 70.0, + min_level: float = 1.8) -> list[Burst]: + """Split an envelope into the runs of pulses worth trying to read. + + This is done in two passes, and the reason is a garden with more than one + sensor in it. + + The first pass only asks where anything is happening at all, and asks it + against the noise rather than against the loudest thing in the block. + That distinction is the whole point. A single threshold set halfway + between the noise floor and the peak of a whole second sounds reasonable + and is not: a sensor on the windowsill and a sensor at the end of the + garden differ by forty decibels, so a threshold set halfway to the near + one sits far above everything the far one ever does, and the far one + disappears -- not weakly, but completely, and only while the near one is + transmitting. A block with one loud sensor in it would yield exactly one + sensor however many were out there. + + So the gate here is the noise floor plus a few times its own scatter, + which is a level the quietest readable burst still clears and noise does + not. Whatever clears it is grouped into regions, and the second pass + then re-thresholds each region against its own high and low. Every + sensor is sliced at its own amplitude, and a burst forty decibels down on + its neighbour is read exactly as well as the neighbour is. + """ + if envelope.size < 4: + return [] + smooth = _smoothed(envelope, rate) + # The noise, read off the bottom of the block rather than the middle of + # it, and for the same reason the gate is: the middle of a block that is + # largely signal is signal. Taken from the median, a burst occupying + # most of what it was handed looks no louder than the thing it is + # measured against, and the whole block is thrown away as empty -- which + # is what happened to any capture short enough to be mostly burst. + quiet = float(np.percentile(smooth, 20)) + peak = float(np.percentile(smooth, 99.99)) + if peak <= quiet * min_level or peak <= 0.0: + return [] # nothing above the noise worth slicing + + hot = smooth > _noise_gate(smooth, quiet, peak) + if not hot.any(): + return [] + per_us = rate / 1e6 + out: list[Burst] = [] + # Grouped tightly, then joined back up. Tightly, because the region is + # what fixes the threshold, and a region holding two sensors of unequal + # strength fixes it on the louder -- which is the fault this whole + # arrangement exists to avoid, and it comes straight back if regions are + # allowed to be generous. + # + # But a tight region cannot hold a whole message from every model: the + # gaps inside one range from two hundred microseconds on the newer + # sensors to four thousand on the oldest, and a grouping tight enough to + # keep two sensors apart tears the oldest into a bit at a time. + # + # So the joining is done afterwards, on the sliced pulses rather than on + # the samples, where each piece has already been measured at its own + # amplitude and the gaps are known. Two pieces belong to one message + # when the silence between them looks like the silences inside them. + for lo, hi in _regions(hot, int(round(gap_us * per_us))): + out += _slice(smooth, lo, hi, rate, gap_us, min_run_us, quiet) + out.sort(key=lambda b: b.at) + return [b for b in _divide(_join(out)) if b.pulses >= min_pulses] + + +def _join(found: list[Burst]) -> list[Burst]: + """Put back together the pieces of one message, and no more than that. + + The test is whether the silence between two pieces looks like the + silences within them. Inside a message every gap is much of a size; + between one copy of a message and the next it is several times that. So + a separation of the same order as the gaps either side of it is part of + the message, and one much larger is the wait for the next copy. + """ + # What a gap inside a message looks like across the whole block, for + # pieces too small to say. The first pulses of a message often come off + # as a piece of one pulse and no gaps at all, and judged on their own + # contents there is nothing to judge -- so they were left stranded, and a + # message short of its first two bits is a message short of its checksum. + everything = [gap for burst in found for gap in burst.spaces] + fallback = float(np.median(everything)) if everything else 0.0 + + out: list[Burst] = [] + for burst in found: + if out: + last = out[-1] + apart = (burst.at - (last.at + last.length_us / 1e6)) * 1e6 + gaps = list(last.spaces) + list(burst.spaces) + typical = float(np.median(gaps)) if gaps else fallback + if 0.0 <= apart <= max(2.5 * typical, 700.0): + out[-1] = Burst( + at=last.at, + marks=last.marks + burst.marks, + spaces=last.spaces + (apart,) + burst.spaces, + level=max(last.level, burst.level)) + continue + out.append(burst) + return out + + +def _smoothed(envelope: np.ndarray, rate: float) -> np.ndarray: + """A running mean over a fraction of the shortest pulse these send. + + Enough that a sample of noise on the wrong side of a threshold cannot + become a pulse, and little enough that the shortest real pulse -- about + two hundred microseconds -- survives it with its edges where they were. + """ + span = max(1, int(round(rate * 30e-6))) + if span <= 1: + return envelope + pad = np.concatenate(([0.0], np.cumsum(envelope, dtype=np.float64))) + return ((pad[span:] - pad[:-span]) / span).astype(np.float32) + + +def _noise_gate(smooth: np.ndarray, quiet: float, peak: float) -> float: + """The level below which nothing is worth looking at. + + This only has to find where something happened; how loud it was is + settled afterwards, region by region. So it wants an estimate of the + noise and nothing else, and in particular it must not move when + something loud happens -- that is exactly the case where a sensor at the + end of the garden is about to be lost. + + Everything here is read off the bottom of the distribution rather than + the middle of it. A second of band holds a few hundred milliseconds of + sensor at the very most, so the lowest fifth of it is noise however busy + the rest was. The middle is not safe in the same way: the median and the + deviation about it both climb once a fair fraction of the block is + signal, and a gate built on them can end up above the peak and pass + nothing at all. + + The other two terms are floors under the gate, for a signal so clean that + the noise has almost no scatter to measure -- a synthetic envelope, or a + receiver with nothing connected -- where the first term alone would be + zero and everything would be above it. + + Note what none of the three terms is: a fraction of the way to the peak. + That is the obvious way to write this and it is the bug it replaced. Any + gate proportional to the loudest thing in the block rises when a sensor + close to the aerial transmits, and rises past every quieter sensor out + there -- so the far ones vanish, and vanish only while the near one is + talking, which is as confusing a symptom as radio produces. + """ + scatter = float(np.percentile(smooth, 40) - np.percentile(smooth, 10)) + gate = quiet + max(6.0 * scatter, 0.5 * quiet, 0.001 * (peak - quiet)) + # And never above the loudest thing in the block, whatever the arithmetic + # above comes to. A gate over the peak is the one setting that cannot be + # right: it reports silence on a second that plainly had something in it. + # + # It binds when the carrier is on for most of what was handed over -- a + # sensor with short gaps, or a capture trimmed close around one -- where + # the percentiles the scatter is taken from straddle the edge between off + # and on, and six times the resulting figure is a gate above everything + # in the block. Reading those percentiles lower down avoids that case + # instead of catching it, and was tried; but there turns out to be no + # signal it rescues that this does not, because a block cannot be mostly + # one sensor and also hold a much quieter one, so the simpler of the two + # is what is here. + return min(gate, quiet + 0.8 * (peak - quiet)) + + +def _regions(hot: np.ndarray, gap_samples: int) -> list[tuple[int, int]]: + """Spans of activity, with short silences inside them kept. + + The silences between the pulses of one message are part of the message, + so they must not end a region; only a silence longer than any gap within + a message does that. + """ + lengths, values, starts = _runs(hot) + spans: list[list[int]] = [] + for length, live, start in zip(lengths, values, starts): + if not live: + continue + if spans and start - spans[-1][1] <= gap_samples: + spans[-1][1] = start + length + else: + spans.append([start, start + length]) + return [(lo, hi) for lo, hi in spans] + + +def _slice(smooth: np.ndarray, lo: int, hi: int, rate: float, gap_us: float, + min_run_us: float, block_floor: float) -> list[Burst]: + """Cut one region into marks and spaces, at its own amplitude. + + The threshold is halfway between what this burst does when it is on and + what it does when it is off, both taken from the region itself. A little + of the silence either side is included so that there is an "off" to + measure even when the region is nearly all pulses. + """ + per_us = rate / 1e6 + margin = int(round(400 * per_us)) + lo = max(0, lo - margin) + hi = min(smooth.size, hi + margin) + part = smooth[lo:hi] + if part.size < 4: + return [] + off = float(np.percentile(part, 5)) + on_level = float(np.percentile(part, 98)) + if on_level <= off: + return [] + on = part > (off + on_level) / 2.0 + + lengths, values, starts = _runs(on) + lengths, values, starts = _merge_short(lengths, values, starts, + min_run_us * per_us) + out: list[Burst] = [] + marks: list[float] = [] + spaces: list[float] = [] + began = 0 + ends = _copy_gap(lengths, values, per_us, gap_us) + # A silence only becomes a bit's gap once another pulse follows it. The + # silence at the end of a burst is not part of the last bit -- it is the + # wait until the next copy, and it is as long as that wait happens to be + # -- so measuring the last bit against it reads the last bit wrongly, + # which is the last bit of the checksum, which is the whole message. + # Held back rather than appended, and dropped if nothing follows. + waiting: float | None = None + for length, high, start in zip(lengths, values, starts): + micro = length / per_us + if high: + if not marks: + began = start + elif waiting is not None: + spaces.append(waiting) + waiting = None + marks.append(micro) + elif marks: + if micro > ends: + out.append(_burst_of(marks, spaces, lo + began, rate, + block_floor, on_level)) + marks, spaces, waiting = [], [], None + else: + waiting = micro + if marks: + out.append(_burst_of(marks, spaces, lo + began, rate, block_floor, + on_level)) + return out + + +def _copy_gap(lengths, values, per_us: float, ceiling: float) -> float: + """The silence that means "that was the whole message", in microseconds. + + Measured from the message rather than fixed, because the gaps inside one + differ by model: two hundred microseconds on the newer sensors, four + thousand on the oldest. A fixed figure has to be larger than the largest + gap inside any message and smaller than the smallest gap between two + copies of one, and there is no such figure -- pick it high and the three + copies of a message merge into one burst that matches nothing, pick it + low and a single message is torn into a bit at a time. + + So it is taken from the burst. From the widest gap in it rather than + the middle one, which is the part that is easy to get wrong: where a + message draws a one as a gap three times the length of a zero, and most + of the bits are zeroes, the middle gap is the short one and twice it + still falls inside the message -- so every one-bit ends a burst and the + message comes apart into pieces of two or three pulses. The widest gap + inside a message is a fact about the message; the middle one is a fact + about the data it happens to be carrying that day. + """ + gaps = [length / per_us for length, live in zip(lengths, values) + if not live] + if len(gaps) < 4: + return ceiling + widest = float(np.percentile(gaps, 90)) + return min(ceiling, max(2.0 * widest, 700.0)) + + +# The longest message here is nine bytes, which with the four sync pulses in +# front of it is seventy-six. Anything appreciably longer than that is more +# than one message however it came to be in one piece. +MAX_PULSES = 88 + + +def _divide(found: list[Burst]) -> list[Burst]: + """Cut apart anything too long to be a single message. + + Joining pieces back together goes by whether the silence between them + looks like the silences within them, and on the oldest sensor those two + are only a factor of two apart -- four thousand microseconds inside a + message and eight thousand between copies of it -- which is too close to + call from a ratio. So the join is allowed to be greedy and this undoes + it where the result is plainly too long to be one message: the largest + gaps in such a burst are the boundaries between the copies, because that + is what they physically are. + + Kept separate from the joining rather than folded into it because the + two answer different questions. The join asks whether two pieces belong + together and can be wrong in one direction only; this asks how many + messages are in front of it, which is a question that can be answered by + counting. + """ + out: list[Burst] = [] + for burst in found: + if burst.pulses <= MAX_PULSES or len(burst.spaces) < 8: + out.append(burst) + continue + cut = 1.5 * float(np.percentile(burst.spaces, 90)) + pieces, marks, spaces = [], [], [] + at = burst.at + began = at + for i, mark in enumerate(burst.marks): + marks.append(mark) + at += mark / 1e6 + gap = burst.spaces[i] if i < len(burst.spaces) else None + if gap is None: + continue + if gap > cut: + pieces.append(Burst(at=began, marks=tuple(marks), + spaces=tuple(spaces), level=burst.level)) + marks, spaces = [], [] + began = at + gap / 1e6 + else: + spaces.append(gap) + at += gap / 1e6 + if marks: + pieces.append(Burst(at=began, marks=tuple(marks), + spaces=tuple(spaces), level=burst.level)) + out += pieces if len(pieces) > 1 else [burst] + return out + + +def _burst_of(marks, spaces, began, rate, floor, peak) -> Burst: + """One burst, from the marks and spaces it was cut into. + + There is always one more mark than there are spaces: the silence that + ended the burst belongs to whatever comes after it, not to this. + + Everything is cast to an ordinary float on the way out. These numbers + came from numpy and end up as a timestamp in a log and in a file of + names, and neither JSON nor YAML will write a numpy float -- which is the + sort of thing that only shows up when a sensor is finally named at two in + the morning. + """ + marks = [float(m) for m in marks][:len(spaces) + 1] + return Burst(at=float(began) / float(rate), marks=tuple(marks), + spaces=tuple(float(s) for s in spaces), + level=float(peak / floor) if floor > 0 else float("inf")) + + +def _runs(on: np.ndarray): + """Every run of equal values: how long, which value, where it started.""" + change = np.flatnonzero(np.diff(on.view(np.int8))) + edges = np.concatenate(([0], change + 1, [on.size])) + return np.diff(edges), on[edges[:-1]], edges[:-1] + + +def _merge_short(lengths, values, starts, floor_samples: float): + """Fold runs too short to be a pulse into whatever is around them. + + A single sample the wrong side of the threshold in the middle of a mark + would otherwise split one four-hundred-microsecond pulse into three, and + everything after that is arithmetic on nonsense. Smoothing removes most + of it; this removes the rest. + """ + if lengths.size == 0: + return lengths, values, starts + keep = lengths >= max(1.0, floor_samples) + if keep.all(): + return lengths, values, starts + if not keep.any(): + return lengths[:0], values[:0], starts[:0] + out_len, out_val, out_start = [], [], [] + for length, value, start in zip(lengths[keep], values[keep], starts[keep]): + if out_val and out_val[-1] == value: + # Two runs of the same value now touching: the short run between + # them was noise, and its samples belong to the pulse it split. + out_len[-1] = start + length - out_start[-1] + else: + out_len.append(int(length)) + out_val.append(bool(value)) + out_start.append(int(start)) + return (np.array(out_len), np.array(out_val, dtype=bool), + np.array(out_start)) + + +def bits_pwm(burst: Burst) -> str: + """Read a burst as pulse width: a long mark is a one, a short one a zero. + + Nothing here is measured against a clock. The bit is decided by + comparing each mark with the gap that follows it, which is a comparison + between two numbers that arrived a few hundred microseconds apart and so + cannot have drifted with respect to each other. A transmitter running + ten per cent fast is read correctly and never noticed. + + The last pulse of a burst is the exception, and has to be handled or the + final bit of the checksum is lost and with it the message: the gap after + it is the silence before the next copy rather than part of the bit. What + stands in for that gap is the bit period -- a mark and its gap together, + which in this coding is the same length whichever bit it is -- taken as + the median over the burst. A pulse longer than half a bit period is a + one, which is what the coding means. + """ + if not burst.marks: + return "" + spaces = list(burst.spaces) + if len(burst.marks) > len(spaces): + if not spaces: + return "" + period = float(np.median([mark + space for mark, space + in zip(burst.marks, spaces)])) + spaces.append(period / 2.0) + return "".join("1" if mark > space else "0" + for mark, space in zip(burst.marks, spaces)) + + +def bits_ppm(burst: Burst) -> str: + """Read a burst as pulse position: a long gap is a one, a short one zero. + + Here there is nothing to compare a gap against except the other gaps, so + the threshold is halfway between the shortest and the longest of them. + A burst whose gaps are all much the same length is not this coding, and + comes back as an empty string rather than as a row of zeroes that + something downstream might believe. + """ + if len(burst.spaces) < 2: + return "" + low, high = min(burst.spaces), max(burst.spaces) + if high < low * 1.5: + return "" + middle = (low + high) / 2.0 + return "".join("1" if space > middle else "0" for space in burst.spaces) + + +def _levels(values, most: int = 3) -> list[float]: + """Thresholds that might separate short from long, best division first. + + The lengths in a burst fall into clusters -- two of them for the bits, + and often a third for the sync, which is longer than either. Rather than + decide which cluster boundary is the one that means something, this + returns a few of the widest gaps in the sorted lengths and lets the + checksums say. Guessing here is what a fixed threshold does, and a fixed + threshold is wrong the moment a transmitter warms up. + """ + ordered = sorted(float(v) for v in values) + if len(ordered) < 4: + return [] + span = ordered[-1] - ordered[0] + if span <= 0: + return [] + steps = sorted(((ordered[i + 1] - ordered[i], i) + for i in range(len(ordered) - 1)), reverse=True) + out = [] + for step, i in steps[:most]: + if step < 0.08 * span: + break # not a division, just jitter + out.append((ordered[i] + ordered[i + 1]) / 2.0) + return sorted(out) + + +_FLIP = str.maketrans("01", "10") + + +def _by_width(values, level: float) -> str: + return "".join("1" if value > level else "0" for value in values) + + +def slicings(burst: Burst) -> list[str]: + """Every way this burst might be read, for the checksums to choose from. + + There are two things not to assume about a burst, and both of them are + assumptions that hold on the sensor in front of you and fail on the next + one. + + The first is which of the pulse and the gap carries the bit. The newer + models vary the pulse; the two older ones vary the gap. + + The second is subtler and is the one that had this reading nothing at + all. Where the pulse carries the bit, the gap may be the complement of + it -- so that every bit takes the same time, and a pulse can be judged + against the gap that follows it -- or the gap may simply be a fixed + spacer. Judged against a fixed spacer of two hundred microseconds, a + short pulse of two hundred and twenty is longer than its gap and reads as + a one, which is the wrong bit, and every message fails its checksum with + nothing to say why. + + So nothing is assumed. The pulse against its own gap, the pulse against + each threshold the pulse lengths themselves suggest, and the same for the + gaps: half a dozen readings of the same burst, of which at most one will + satisfy a checksum. It costs a few microseconds per burst and it is the + difference between working on the sensors this was written against and + working on the ones in somebody's garden. + """ + out = [bits_pwm(burst), bits_ppm(burst)] + for values in (burst.marks, burst.spaces): + for level in _levels(values): + # And the other way up. Which of long and short means one is a + # third thing not to assume, and complementing a bit string is + # free. + reading = _by_width(values, level) + out += [reading, reading.translate(_FLIP)] + seen: set = set() + keep = [] + for bits in out: + if bits and bits not in seen: + seen.add(bits) + keep.append(bits) + return keep + + +def readings_from(iq: np.ndarray, sample_rate: float, offset: float = 0.0, + when: float = 0.0) -> list[Reading]: + """Every distinct sensor message in one block of samples, in order. + + ``when`` is the clock time the block began, so that each reading carries + the real moment it arrived rather than its offset in a buffer. A log of + weather that started again from zero would be unusable, and a sensor + reports every sixteen seconds, so the difference matters. + + The three copies of a message are looked for across the whole block + rather than inside one burst, because they are not one burst: these + sensors leave ten milliseconds of silence between the copies, which is + three times what ends a burst. Corroboration therefore has to happen + here, where the whole second is in view, and a message that arrived three + times comes back as one reading which knows it arrived three times. + """ + envelope, rate = baseband(iq, sample_rate, offset) + found = [] + for burst in bursts(envelope, rate): + # One burst is one copy of one message, however many ways of reading + # it happened to produce it. Counted per reading rather than per + # slicing, or two readings of the same burst would corroborate each + # other -- and corroboration is the only thing standing between the + # two thinly-checked models and a display full of sensors that are + # not there. + here: dict = {} + for reading in _from_burst(burst): + here.setdefault((reading.family, reading.bits), reading) + for reading in here.values(): + reading.at = when + burst.at + reading.snr = burst.decibels + found.append(reading) + return confirmed(found) + + +def confirmed(found: list[Reading]) -> list[Reading]: + """One reading per distinct message, once enough copies have arrived. + + Identical bits are one message heard more than once; different bits are + different messages, even from the same sensor, which is how the 5-in-1's + two halves both survive a block that holds them both. + """ + seen: dict[tuple, list[Reading]] = {} + for reading in found: + seen.setdefault((reading.family, reading.bits), []).append(reading) + out = [] + for (family, _bits), group in seen.items(): + if len(group) < REPEATS_NEEDED[family]: + continue + first = min(group, key=lambda r: r.at) + first.copies = len(group) + # The best of the copies, not the first: three copies of a message go + # out a few milliseconds apart and arrive at whatever the fading does + # to each, so the strongest is the fairer answer to how well that + # sensor is being heard. + first.snr = max(r.snr for r in group) + out.append(first) + return sorted(out, key=lambda r: (r.at, r.family, r.sensor)) + + +# --------------------------------------------------------------------------- +# Looking at what actually arrived +# --------------------------------------------------------------------------- + +def timings(values, tolerance: float = 0.25) -> list[tuple[float, int]]: + """The distinct lengths in a burst, each with how many there were. + + A message drawn as pulses has two or three lengths in it and nothing in + between, so this is the shape of the protocol written out: (220, 28) + (402, 28) (598, 4) is a burst of fifty-six bits and four sync pulses, + and anyone who knows the sensor can see that at a glance. It is the + single most useful thing to print when nothing is decoding, because it + says whether the trouble is the radio or the arithmetic. + """ + ordered = sorted(float(v) for v in values) + groups: list[list[float]] = [] + for value in ordered: + if groups and value <= groups[-1][-1] * (1.0 + tolerance): + groups[-1].append(value) + else: + groups.append([value]) + return [(sum(g) / len(g), len(g)) for g in groups] + + +@dataclass(frozen=True) +class NearMiss: + """One way of framing a burst, and which of its checks held. + + "Nothing framed" is not an answer, it is the absence of one. A message + that satisfies its checksum and fails its parity is a different fault + from one that fails both, and both are different from a burst that never + lined up on a byte boundary at all -- and the three want three different + fixes. This is what turns the first into the second. + """ + + family: str = "" + length: int = 0 # in bytes + offset: int = 0 # where in the burst it starts, in bits + data: tuple = () + checks: tuple = () # (name, passed) in the order they are run + read: bool = False # whether the decoder accepted it outright + reflect: bool = False # whether the bytes were read the other way + + @property + def score(self) -> int: + return sum(1 for _name, passed in self.checks if passed) + + @property + def hex(self) -> str: + return " ".join(f"{byte:02X}" for byte in self.data) + + def describe(self) -> str: + marks = " ".join(("\u2713" if passed else "\u2717") + name + for name, passed in self.checks) + order = ", least significant bit first" if self.reflect else "" + return (f"{self.length} bytes at bit {self.offset}{order}: {marks} " + f"{self.hex}") + + +def _checked(count: int, at: int, data: list[int], + reflect: bool = False) -> NearMiss: + """Every check that framing would have to pass, run one at a time.""" + if count in (7, 8, 9): + checks = (("sum", (sum(data[:-1]) & 0xFF) == data[-1]), + ("parity", all(parity8(byte) == 0 for byte in data[2:-1])), + ("type", (data[2] & 0x3F) in _KNOWN_TYPES)) + elif count == 5: + checks = (("sum", (sum(data[:4]) & 0xFF) == data[4]),) + else: + checks = (("crc", crc8(data[:3]) == data[3]),) + family = MESSAGE_FAMILY[count] + decoder = dict(_DECODERS)[count] + return NearMiss(family=family, length=count, offset=at, data=tuple(data), + checks=checks + (("plausible", decoder(data) is not None),), + read=decoder(data) is not None, reflect=reflect) + + +def near_misses(bits: str, most: int = 2) -> list[NearMiss]: + """The framings that came closest, best first. + + Only the positions a real message could occupy, because a window halfway + through a burst failing its checksum says nothing anybody needs. + """ + out = [] + for count, _decoder in _DECODERS: + need = count * 8 + for at in range(0, len(bits) - need + 1): + if at > LEAD_BITS or len(bits) - at - need > TAIL_BITS: + continue + for reflect in BIT_ORDERS: + try: + out.append(_checked(count, at, + _bytes_at(bits, at, count, reflect), + reflect)) + except ValueError: + break + out.sort(key=lambda miss: (miss.score, miss.length), reverse=True) + return out[:most] + + +@dataclass +class Survey: + """What one block of samples looked like at every stage. + + Only ever built to be printed. It exists because "nothing was heard" is + four different faults wearing the same coat -- no signal at all, a signal + below the gate, a burst that sliced into the wrong number of pulses, or + bits that came out and failed their checksums -- and they want four + different answers. Each stage here separates one of them from the next. + """ + + quiet: float = 0.0 # the noise, as the gate measures it + gate: float = 0.0 # what a burst has to clear + peak: float = 0.0 # the loudest thing in the block + rate: float = 0.0 # what the envelope was sliced at + seen: list = field(default_factory=list) # (burst, slicings, framed) + readings: list = field(default_factory=list) + misses: list = field(default_factory=list) # per burst, the closest tries + + @property + def loudest(self) -> float: + return self.peak / self.quiet if self.quiet > 0 else float("inf") + + +def survey(iq: np.ndarray, sample_rate: float, offset: float = 0.0, + when: float = 0.0) -> Survey: + """One block, taken apart stage by stage, for when nothing decodes. + + The readings are obtained by running the ordinary path rather than by + repeating it here, so that what this reports and what the program does + cannot come apart -- a diagnostic that disagrees with the thing it is + diagnosing is worse than none. + """ + envelope, rate = baseband(iq, sample_rate, offset) + smooth = _smoothed(envelope, rate) + peak = float(np.percentile(smooth, 99.99)) if smooth.size else 0.0 + quiet = float(np.percentile(smooth, 20)) if smooth.size else 0.0 + seen, misses = [], [] + for burst in bursts(envelope, rate): + tries = slicings(burst) + # Without the corroboration rule, so that a message which framed + # correctly and merely arrived once is reported as what it is, + # rather than as silence. + framed = _best([r for bits in tries for r in candidates(bits)], + confirm=False) + seen.append((burst, tries, framed)) + # What almost worked, for a burst that came to nothing. Judged + # across every reading of it, because which reading was the right one + # is the question and not the answer. + closest = sorted((m for bits in tries for m in near_misses(bits)), + key=lambda m: (m.score, m.length), reverse=True) + misses.append(closest[:2]) + return Survey(quiet=quiet, + gate=_noise_gate(smooth, quiet, peak) if smooth.size else 0.0, + peak=peak, rate=rate, seen=seen, misses=misses, + readings=readings_from(iq, sample_rate, offset, when)) + + +# --------------------------------------------------------------------------- +# Building the same messages, so the decoder can be held to them +# --------------------------------------------------------------------------- +# +# Every one of these lives beside the decoder that reads it, so that the two +# cannot drift apart, and so a test can put a temperature in and take the +# same one out. They are also what the simulator transmits, which means +# `--simulate` runs the real slicer and the real decoder rather than a +# shortcut past them. + +def _txr_bits(data: list[int]) -> str: + """Finish a TXR-family message: parity on the payload, then the sum.""" + data = list(data) + for i in range(2, len(data)): + data[i] &= 0x7F + if parity8(data[i]) != 0: # even, as the sensors send it + data[i] |= 0x80 + data.append(sum(data) & 0xFF) + return "".join(format(byte, "08b") for byte in data) + + +def _status(message: int, battery_low: bool) -> int: + return (message & 0x3F) | (0x00 if battery_low else 0x40) + + +def _wind_raw(kph: float) -> int: + return 0 if kph <= 0 else max(1, min(255, int(round((kph - 1.0) / 0.8278)))) + + +def tower_frame(sensor: int, celsius: float, humidity: int, + channel: str = "A", battery_low: bool = False) -> str: + """One 592TXR message: temperature, humidity, checksum and parity.""" + raw = int(round((celsius + 100.0) * 10.0)) + return _txr_bits([((CHANNELS.index(channel) & 3) << 6) | ((sensor >> 8) & 0x3F), + sensor & 0xFF, + _status(0x04, battery_low), + humidity & 0x7F, + (raw >> 7) & 0x0F, + raw & 0x7F]) + + +def five_in_one_wind_rain(sensor: int, kph: float, degrees: float, + rain_counter: int, channel: str = "A", + battery_low: bool = False) -> str: + """The 5-in-1's wind message: speed, where from, and the rain counter.""" + wind = _wind_raw(kph) + point = min(range(16), key=lambda i: abs(WIND_POINTS[i] - degrees % 360)) + counter = rain_counter & 0x3FFF + return _txr_bits([((CHANNELS.index(channel) & 3) << 6) | ((sensor >> 8) & 0x0F), + sensor & 0xFF, + _status(0x31, battery_low), + (wind >> 3) & 0x1F, + ((wind & 0x07) << 4) | point, + (counter >> 7) & 0x7F, + counter & 0x7F]) + + +def five_in_one_weather(sensor: int, kph: float, celsius: float, + humidity: int, channel: str = "A", + battery_low: bool = False) -> str: + """The 5-in-1's other message: speed, temperature and humidity.""" + wind = _wind_raw(kph) + raw = int(round((celsius * 9 / 5 + 32) * 10.0)) + 400 + return _txr_bits([((CHANNELS.index(channel) & 3) << 6) | ((sensor >> 8) & 0x0F), + sensor & 0xFF, + _status(0x38, battery_low), + (wind >> 3) & 0x1F, + ((wind & 0x07) << 4) | ((raw >> 7) & 0x0F), + raw & 0x7F, + humidity & 0x7F]) + + +def lightning_frame(sensor: int, celsius: float, humidity: int, + strikes: int = 0, miles: int = 31, + channel: str = "A", battery_low: bool = False, + interference: bool = False) -> str: + """One 6045M message: the weather, the strike count and how far off.""" + raw = int(round(((celsius * 9 / 5 + 32) + 100.0) * 10.0)) + return _txr_bits([((CHANNELS.index(channel) & 3) << 6) | ((sensor >> 8) & 0x0F), + sensor & 0xFF, + _status(LIGHTNING_MESSAGE, battery_low), + humidity & 0x7F, + (raw >> 7) & 0x1F, + raw & 0x7F, + strikes & 0x7F, + (0x20 if interference else 0x00) | (miles & 0x1F)]) + + +def _twelve(celsius: float) -> int: + raw = int(round(celsius * 10.0)) + return raw + 0x1000 if raw < 0 else raw + + +def frame_609(sensor: int, celsius: float, humidity: int, + battery_low: bool = False) -> str: + """One 609TXC message: identity, temperature, humidity, a sum.""" + raw = _twelve(celsius) + data = [sensor & 0xFF, + (0x00 if battery_low else 0x80) | ((raw >> 8) & 0x0F), + raw & 0xFF, + humidity & 0xFF] + data.append(sum(data) & 0xFF) + return "".join(format(byte, "08b") for byte in data) + + +def frame_606(sensor: int, celsius: float, battery_low: bool = False) -> str: + """One 606TX message: identity, temperature, a CRC-8, and that is all.""" + raw = _twelve(celsius) + data = [sensor & 0xFF, + (0x00 if battery_low else 0x80) | ((raw >> 8) & 0x0F), + raw & 0xFF] + data.append(crc8(data)) + return "".join(format(byte, "08b") for byte in data) + + +# --------------------------------------------------------------------------- +# Turning bits back into something a receiver would have heard +# --------------------------------------------------------------------------- + +# The timings, in microseconds. The newer sensors vary the pulse; the two +# older ones keep the pulse and vary the gap. Neither is read against these +# numbers -- the slicer compares a pulse with its own gap, or a gap with the +# other gaps -- so they are here to transmit with, not to decode by. +PWM_TIMING = {"sync_mark": 600.0, "sync_space": 600.0, "syncs": 4, + "one": (400.0, 220.0), "zero": (220.0, 400.0)} +PPM_TIMING = {"sync_mark": 500.0, "sync_space": 1000.0, "syncs": 1, + "one": (500.0, 2000.0), "zero": (500.0, 1000.0)} +TIMINGS = {"pwm": PWM_TIMING, "ppm": PPM_TIMING} + + +def pulse_train(bits: str, coding: str = "pwm") -> tuple[list, list]: + """One message as the marks and spaces a transmitter would key out.""" + timing = TIMINGS[coding] + marks, spaces = [], [] + for _ in range(timing["syncs"]): + marks.append(timing["sync_mark"]) + spaces.append(timing["sync_space"]) + for bit in bits: + mark, space = timing["one" if bit == "1" else "zero"] + marks.append(mark) + spaces.append(space) + if coding == "ppm": + # In pulse-position coding the bit *is* the gap, so the last gap has + # to be closed by a pulse or there is no gap there to read. That + # trailing pulse carries no bit of its own. + marks.append(timing["sync_mark"]) + return marks, spaces + + +def modulate(messages, sample_rate: float = 1_024_000.0, + coding: str = "pwm", repeats: int = 3, offset: float = 0.0, + amplitude: float = 1.0, noise: float = 0.0, seed: int = 0, + lead_us: float = 4_000.0, gap_us: float = 8_000.0) -> np.ndarray: + """What a receiver would have heard, for one or more messages. + + Each message is sent ``repeats`` times with a short silence between the + copies and a longer one after them, which is what the real sensors do and + is why the two thinly-checked models can be believed at all. + """ + per_us = sample_rate / 1e6 + parts = [np.zeros(int(lead_us * per_us), dtype=np.float32)] + for bits in ([messages] if isinstance(messages, str) else messages): + marks, spaces = pulse_train(bits, coding) + for _ in range(max(1, repeats)): + for mark, space in zip(marks, spaces + [0.0] * len(marks)): + parts.append(np.full(int(round(mark * per_us)), amplitude, + dtype=np.float32)) + if space: + parts.append(np.zeros(int(round(space * per_us)), + dtype=np.float32)) + parts.append(np.zeros(int(gap_us * per_us), dtype=np.float32)) + parts.append(np.zeros(int(gap_us * per_us * 2), dtype=np.float32)) + envelope = np.concatenate(parts) + return _to_air(envelope, sample_rate, offset, noise, seed) + + +def _to_air(envelope: np.ndarray, sample_rate: float, offset: float, + noise: float, seed: int) -> np.ndarray: + """An envelope, put where on the band the receiver expects to find it.""" + signal = envelope.astype(np.complex64) + if offset: + turn = np.exp(2j * math.pi * offset + * np.arange(signal.size, dtype=np.float64) / sample_rate) + signal = signal * turn.astype(np.complex64) + if noise: + rng = np.random.default_rng(seed) + signal = signal + noise * ( + rng.standard_normal(signal.size) + + 1j * rng.standard_normal(signal.size)).astype(np.complex64) + return signal.astype(np.complex64) + + +# --------------------------------------------------------------------------- +# Sensors that are not there +# --------------------------------------------------------------------------- + +@dataclass +class VirtualSensor: + """A sensor on a fence post that does not exist. + + It has an identity, a model, a channel switch and an opinion about the + weather, and it broadcasts all of that exactly as a real one does -- + through the real encoders, the real modulator, the real slicer and the + real decoder. Nothing downstream can tell the difference, which is the + point: everything downstream is what is being exercised. + """ + + family: str = "tower" + sensor: int = 0x1234 + channel: str = "A" + every: float = 16.0 # seconds between transmissions + phase: float = 0.0 # so they do not all speak at once + strength: float = 1.0 + celsius: float = 18.0 + humidity: float = 55.0 + kph: float = 8.0 + rain_counter: int = 0 + strikes: int = 0 + storm_miles: int = 31 + battery_low: bool = False + label: str = "" # what a person would call it + clock: float = 0.0 + turn: int = 0 + + @property + def coding(self) -> str: + return "ppm" if self.family in ("609", "606") else "pwm" + + def messages(self) -> list[str]: + """What it says this time round. + + The 5-in-1 alternates, because it has more to say than fits in one + message; everything else says the same thing every time. + """ + self.turn += 1 + if self.family == "tower": + return [tower_frame(self.sensor, self.celsius, int(self.humidity), + self.channel, self.battery_low)] + if self.family == "5n1": + if self.turn % 2: + return [five_in_one_wind_rain( + self.sensor, self.kph, (self.turn * 23.0) % 360.0, + self.rain_counter, self.channel, self.battery_low)] + return [five_in_one_weather(self.sensor, self.kph, self.celsius, + int(self.humidity), self.channel, + self.battery_low)] + if self.family == "6045": + return [lightning_frame(self.sensor, self.celsius, + int(self.humidity), self.strikes, + self.storm_miles, self.channel, + self.battery_low)] + if self.family == "609": + return [frame_609(self.sensor, self.celsius, int(self.humidity), + self.battery_low)] + return [frame_606(self.sensor, self.celsius, self.battery_low)] + + def advance(self, seconds: float, rng) -> None: + """Let the weather move on, slowly and within reason. + + Slowly matters. A garden that swings six degrees in a minute would + exercise everything downstream perfectly well and would look, to + anyone running ``--simulate`` to see what the program does, like a + program that cannot read a thermometer. So the rates here are + roughly a real afternoon's: about a degree an hour of drift, wind + that gusts around a mean, and rain that falls in tenths of a + millimetre at a time. Everything stays inside what the hardware + could report, so the decoder's own plausibility check is never the + thing that rejects it. + """ + self.clock += seconds + turn = math.sin(self.clock / 1800.0) # half an hour a swing + self.celsius = min(45.0, max(-30.0, self.celsius + + seconds * (0.0004 * turn + + rng.normal(0.0, 0.0008)))) + self.humidity = min(99.0, max(5.0, self.humidity + + seconds * (-0.0016 * turn + + rng.normal(0.0, 0.004)))) + gust = 1.0 + 0.5 * math.sin(self.clock / 41.0) + self.kph = min(90.0, max(0.0, self.kph + + seconds * 0.08 * (gust * 11.0 - self.kph))) + if self.family == "5n1" and rng.random() < 0.01 * seconds: + self.rain_counter += 1 + if self.family == "6045": + if rng.random() < 0.006 * seconds: + self.strikes = (self.strikes + 1) & 0x7F + self.storm_miles = int(rng.integers(1, 25)) + elif rng.random() < 0.002 * seconds: + self.storm_miles = 31 + + +def default_sensors() -> list[VirtualSensor]: + """One of each, in a garden that does not exist. + + The transmission periods are deliberately awkward numbers. Two sensors + on exactly sixteen seconds would transmit over the top of each other + every time they lined up and never come unstuck again, which would be + both unrealistic and unfair: real ones are timed by a cheap oscillator in + the cold and drift past each other within a few minutes. Here they are + simply given periods with no common factor, which has the same effect and + keeps the garden reproducible from a seed. + """ + return [ + VirtualSensor(family="tower", sensor=0x1A2B, channel="A", every=16.1, + phase=1.0, celsius=17.4, humidity=62, + label="back fence"), + VirtualSensor(family="tower", sensor=0x0C41, channel="B", every=16.7, + phase=6.5, celsius=20.9, humidity=44, strength=0.55, + label="greenhouse"), + VirtualSensor(family="5n1", sensor=0x0777, channel="A", every=18.3, + phase=3.0, celsius=16.8, humidity=71, kph=11.0, + rain_counter=1284, label="mast"), + VirtualSensor(family="6045", sensor=0x0311, channel="C", every=24.9, + phase=9.0, celsius=19.1, humidity=58, strikes=3, + storm_miles=12, label="lightning"), + VirtualSensor(family="609", sensor=0x5C, every=30.2, phase=12.0, + celsius=4.2, humidity=80, battery_low=True, + label="shed"), + VirtualSensor(family="606", sensor=0x93, every=32.6, phase=15.0, + celsius=-3.5, strength=0.7, label="freezer"), + ] + + +class SimulatedSensors: + """A receiver-shaped source of weather that is not happening. + + It answers ``read_samples`` the way the dongle does and hands back the + same on-off-keyed bursts a garden full of sensors would, so the whole + section -- the slicer, every decoder, the naming, the log, the report and + the export -- can be run through without an aerial, a sensor or a garden. + """ + + def __init__(self, sensors=None, sample_rate: float = 1_024_000.0, + offset: float = 250_000.0, noise: float = 0.02, + seed: int = 0, realtime: bool = False): + self.sensors = list(sensors) if sensors is not None \ + else default_sensors() + self.sample_rate = float(sample_rate) + self.offset = float(offset) + self.noise = noise + self.rng = np.random.default_rng(seed) + self.frequency = ACURITE_HZ + self.clock = 0.0 + # A block is a second of samples but takes longer than a second to + # slice, so sensors advanced by the length of the block fall behind + # the clock their readings are stamped with. Listening for real, the + # weather moves by the time that actually passed; in a test, by the + # block, so the same seed gives the same garden twice. + self.realtime = realtime + self._last = None + + # -- the shape of a device ------------------------------------------- + def open(self): + return self + + def close(self) -> None: + return None + + def tune(self, hz: float, settle: bool = True) -> int: + self.frequency = hz + return int(hz) + + def read_samples(self, count: int, flush: bool = False) -> np.ndarray: + """One block of garden: whoever is due to speak, speaks.""" + import time as _time + + seconds = count / self.sample_rate + if self.realtime: + # A dongle hands back a second of samples once a second has + # passed, and this stands in for a dongle, so it waits. Without + # the wait a block comes back the moment it is asked for, the + # garden lives several times faster than the clock the readings + # are stamped with, and anything measured against wall time -- + # how long to listen for, how large a capture will be -- comes + # out wrong by whatever the machine happens to be worth. + now = _time.monotonic() + if self._last is not None: + behind = seconds - (now - self._last) + if behind > 0: + _time.sleep(behind) + now = _time.monotonic() + seconds = max(0.0, now - self._last) + self._last = now + block = np.zeros(count, dtype=np.complex64) + began, ended = self.clock, self.clock + seconds + for sensor in self.sensors: + for at in _due(sensor, began, ended): + burst = modulate(sensor.messages(), self.sample_rate, + coding=sensor.coding, offset=self.offset, + amplitude=sensor.strength, lead_us=0.0) + start = int((at - began) * self.sample_rate) + room = min(burst.size, max(0, count - start)) + if room > 0: + block[start:start + room] += burst[:room] + sensor.advance(seconds, self.rng) + self.clock = ended + if self.noise: + block = block + self.noise * ( + self.rng.standard_normal(count) + + 1j * self.rng.standard_normal(count)).astype(np.complex64) + return block.astype(np.complex64) + + def read_seconds(self, seconds: float, flush: bool = False) -> np.ndarray: + return self.read_samples(int(self.sample_rate * seconds)) + + +def _due(sensor: VirtualSensor, began: float, ended: float) -> list[float]: + """Every moment in a block at which one sensor is due to transmit.""" + if sensor.every <= 0 or ended <= began: + return [] + first = math.ceil((began - sensor.phase) / sensor.every) + out = [] + at = sensor.phase + first * sensor.every + while at < ended: + if at >= began: + out.append(at) + at += sensor.every + return out diff --git a/bandsaunter/aircraft.py b/bandsaunter/aircraft.py index e233f83..11d7c32 100644 --- a/bandsaunter/aircraft.py +++ b/bandsaunter/aircraft.py @@ -28,7 +28,7 @@ from .adsb import ADSB_HZ, SAMPLE_RATE from .flightlog import read_position from .settings import Setting, format_value -__all__ = ["AircraftOptions", "OPTIONS", "listen", "watch", "draw", +__all__ = ["AircraftOptions", "OPTIONS", "defaults", "listen", "watch", "draw", "logs_in", "open_device", "open_log", "pump", "finish", "windowed", "format_option", @@ -122,6 +122,11 @@ class AircraftOptions: window_rings: bool = True theme: str = "night" box_opacity: int = 85 + pulse: bool = True + pulse_rate: float = 2.2 + echo: bool = True + echo_every: float = 3.0 + echo_size: int = 46 map_brightness: int = 70 tile_url: str = "" @@ -452,9 +457,75 @@ OPTIONS: tuple[Setting, ...] = ( O("device", "Receiver", "Receiver", "int", flags=("--speed-unit",), metavar="UNIT", guidance="knots is what aviation uses and what the aircraft actually " "said. mph or kph if that is what means something to you."), + + O("pulse", "Pulsing aircraft", "Effects", "bool", + "swell each aircraft from bright to dim and back, over and over", + "At the top of the swell an aeroplane burns: it is drawn in the peak " + "colours with a halo grown out of its own shape, which is what a " + "phosphor does when the beam sits in one place a little too long. At " + "the bottom it is merely there. Each aircraft is offset by its own " + "address so that a sky full of them swells separately rather than " + "beating as one, which would read as a display flashing rather than " + "as a lot of separate things transmitting. Only aircraft still being " + "heard pulse: one that has gone quiet is fading, and a thing that is " + "fading and beating at once says two contradictory things about " + "itself.", + flags=("--pulse",), off_flags=("--no-pulse",), + guidance="Turn it off for a still picture, or if a moving screen in " + "the corner of the room is a distraction."), + O("pulse_rate", "Pulse takes", "Effects", "float", + "how long one swell from dim to bright and back again takes", + "Seconds of watching rather than of flying, so a pulse looks the " + "same whatever speed an evening is being run through. Long enough " + "and it is a slow breath; short enough and it is a blink, which is " + "the one thing it should not be.", + unit="s", minimum=0.2, maximum=30.0, flags=("--pulse-rate",), + metavar="SECONDS", example="2.2", + guidance="Two or three seconds reads as breathing. Under one is a " + "strobe."), + O("echo", "Echo circles", "Effects", "bool", + "rings travelling outward from each aircraft, growing and dimming", + "What a radar repeater does, and what the eye reads as this thing is " + "transmitting -- which is exactly what an aeroplane on this picture " + "is doing, twice a second, and is how it got on the picture at all. " + "One ring at a time per aircraft: a new one leaves as the last " + "reaches the end of its reach, so the sky has one ring per aeroplane " + "rather than a stack of them to read through.", + flags=("--echo",), off_flags=("--no-echo",), + guidance="Handsome on a quiet sky. Turn it off when a hundred " + "aircraft are overhead."), + O("echo_every", "Echo every", "Effects", "float", + "how long between one ring leaving an aircraft and the next", + "Also how long a ring takes to cross its whole reach, since there is " + "only ever one in flight at a time. Seconds of watching rather than " + "of flying.", + unit="s", minimum=0.3, maximum=60.0, flags=("--echo-every",), + metavar="SECONDS", example="3", + guidance="Longer for a calmer picture, shorter for a busier one."), + O("echo_size", "Echo reaches", "Effects", "int", + "how far a ring gets from its aircraft before it has faded away", + "In pixels rather than miles, because it is a mark on a picture " + "rather than a distance in the sky: a ring measured in miles would " + "be a claim about range, and this is not one. Bigger than the space " + "between two aircraft and the rings cross each other; smaller than " + "the aeroplane and there is nothing to see.", + unit="px", minimum=6, maximum=400, flags=("--echo-size",), + metavar="PIXELS", example="46", + guidance="About forty on a nine-hundred-pixel picture. Scale it up " + "with the width."), ) -OPTION_GROUPS = ("Receiver", "Listening", "Aircraft", "Animation", "The map", "Labels") +OPTION_GROUPS = ("Receiver", "Listening", "Aircraft", "Animation", "The map", + "Labels", "Effects") + + +def defaults() -> AircraftOptions: + """A fresh set, for showing what has been changed from it. + + Named the same as the weather section's, so that the menus can show + either without knowing which they are showing. + """ + return AircraftOptions() def in_group(group: str) -> list[Setting]: @@ -768,7 +839,10 @@ def watch(console, options: AircraftOptions, output_dir: str, fade=max(0.0, options.fade), airports=options.airports, rings=options.window_rings, - box_opacity=max(0, options.box_opacity) / 100.0) + box_opacity=max(0, options.box_opacity) / 100.0, + pulse=options.pulse_rate if options.pulse else 0.0, + echo=options.echo_every if options.echo else 0.0, + echo_reach=max(1, options.echo_size)) sky.started = started sky.log_name = log.path.name if log is not None else "" if options.simulate: @@ -980,6 +1054,9 @@ def draw(console, options: AircraftOptions, tracks, out_path, book=None): airports=options.airports, rings=options.rings, box_opacity=max(0, options.box_opacity) / 100.0, + pulse=options.pulse_rate if options.pulse else 0.0, + echo=options.echo_every if options.echo else 0.0, + echo_reach=max(1, options.echo_size), radius_nm=radius_in_nm(options), centre=read_position(options.location)) except (OSError, RuntimeError, ValueError) as exc: diff --git a/bandsaunter/aprs.py b/bandsaunter/aprs.py new file mode 100644 index 0000000..a8f8bee --- /dev/null +++ b/bandsaunter/aprs.py @@ -0,0 +1,1724 @@ +"""Listening to APRS, from either front end. + +One channel, one frequency, everybody: 144.390 MHz in North America and a +different number in every other region, carrying position reports, weather, +messages, objects and telemetry from every amateur station within earshot and +every digipeater repeating them onward. Unlike the aircraft and the weather +sensors, this is a conversation rather than a broadcast -- stations address +each other, acknowledge each other, and relay for each other -- so what is +worth showing is not only who is out there but what has been said. + +It has a channel to itself for the same reason the other two modes do: a scan +stops on a signal, records it and moves on, and APRS is a two-second +transmission every few minutes from a hundred different stations. A sweep +catches whichever one happened to key up while the sweep was pointed there. +Parking on the channel catches all of them. + +Nothing here has to be named. A weather sensor broadcasts a number out of a +hat and needs a person to say which shed it is on; an APRS station broadcasts +its callsign, which is issued by a government and is already the name. That +is the one way this section is simpler than the last one. +""" + +from __future__ import annotations + +import math +import time +from dataclasses import asdict, dataclass, field +from datetime import datetime +from pathlib import Path + +import yaml + +from . import ax25, packets +from .settings import Setting, format_value + +__all__ = ["AprsOptions", "OPTIONS", "OPTION_GROUPS", "defaults", "in_group", + "open_book", "who_text", + "channel_text", "find_channel", "report_channels", "Found", + "use_region", "receiver_at", "coordinates", + "map_unit", "radius_in_nm", "base_call", "licensee", + "by_key", "format_option", "describe", "summarise", "Station", + "Net", "Heard", "listen", "open_device", "open_log", "pump", + "finish", "report", "watch", "windowed", "blip_for", + "station_lines", "station_shade", "STATION_SHADE", + "load_options", "save_options", "options_path", + "logs_in", "channel_named", "distance_km", "bearing_deg"] + + +# --------------------------------------------------------------------------- +# The options +# --------------------------------------------------------------------------- + +@dataclass +class AprsOptions: + """Everything the APRS mode can be told, in one place.""" + + # -- receiver ------------------------------------------------------- + device: int = 0 + gain: str = "auto" + rate: float = 240_000.0 + frequency: float = ax25.APRS_HZ + region: str = "north-america" + simulate: bool = False + + # -- listening ------------------------------------------------------ + seconds: float = 0.0 + log: bool = True + packets_seen: bool = False # a line per packet rather than a table + hold: float = 3600.0 + location: str = "" # where the aerial is, for distances + + # -- what to show --------------------------------------------------- + units: str = "metric" + unparsed: bool = True + lookup: bool = True # who each callsign is licensed to + digipeated: bool = True # count frames that reached here relayed + + # -- the map -------------------------------------------------------- + radius: float = 50.0 # how far the window reaches, in map_unit + theme: str = "night" + map_brightness: int = 70 + basemap: bool = True + window_rings: bool = True + box_opacity: int = 85 + trails: bool = True + tile_url: str = "" + + # -- afterwards ----------------------------------------------------- + report: bool = True + csv: bool = False + kml: bool = False + + @property + def imperial(self) -> bool: + return str(self.units).lower().startswith("imp") + + def validate(self) -> list[str]: + out = [] + if self.rate < 96_000: + out.append("the sample rate must be at least 96 kS/s to hold a " + "16 kHz channel with room to filter it") + if self.seconds < 0: + out.append("times cannot be negative") + if self.hold <= 0: + out.append("a station must stay on the display for some time") + if not 1e6 < self.frequency < 2e9: + out.append("that is not a frequency a receiver can be tuned to") + return out + + def to_dict(self) -> dict: + return asdict(self) + + +def defaults() -> AprsOptions: + return AprsOptions() + + +def channel_named(name: str) -> float: + """The APRS channel for a region, or the default. + + A number per region by agreement rather than regulation, which is why + this is a list rather than a constant and why getting it wrong means + hearing nothing at all rather than hearing it badly. + """ + wanted = (name or "").strip().lower() + for key, hz, _where in ax25.APRS_CHANNELS: + if wanted == key: + return hz + return ax25.APRS_HZ + + +O = Setting +_REGIONS = tuple(key for key, _hz, _where in ax25.APRS_CHANNELS) + +OPTIONS: tuple[Setting, ...] = ( + O("device", "Receiver", "Receiver", "int", + "which receiver to use, when more than one is plugged in", + "The index shown by `bandsaunter devices`. Zero unless you have " + "several dongles.", + minimum=0, flags=("--device",), example="0"), + O("gain", "Gain", "Receiver", "gain", + "tuner gain in dB, or automatic", + "APRS is a two-second burst from a handheld that may be a street away " + "or a hilltop digipeater thirty miles off, and the automatic gain " + "control copes with both. A fixed gain makes the signal figures " + "comparable between one run and the next, which matters if you are " + "using them to judge an aerial.", + flags=("--gain",), example="auto"), + O("rate", "Sample rate", "Receiver", "float", + "how fast to sample; 96 kS/s is the least that holds the channel", + "A 16 kHz FM channel needs a few times that to filter cleanly. The " + "default leaves room and costs little, since everything after the " + "filter runs at audio rates.", + unit="Hz", minimum=96_000.0, flags=("--rate",), example="240000"), + O("region", "Region", "Receiver", "choice", + "which APRS channel to listen on", + "The frequency is agreed between amateurs rather than allocated, so " + "it differs by region and there is no way to discover it from the " + "air: on the wrong one you hear silence, not a bad signal. North " + "America uses 144.390, Europe 144.800, Australia 145.175.", + choices=_REGIONS, flags=("--region",), example="north-america"), + O("frequency", "Listen on", "Receiver", "float", + "the exact frequency, if the region's channel is not what you want", + "Set by the region unless given here. Worth setting by hand for a " + "local packet network on another channel, or for the 9600 baud links " + "some areas run alongside the main one.", + unit="Hz", minimum=1e6, maximum=2e9, flags=("--frequency", "--freq"), + example="144390000"), + O("simulate", "Invent a channel", "Receiver", "bool", + "put imaginary stations on an imaginary band", + "A dozen stations that are not there -- cars moving, a weather " + "station, a digipeater, objects, messages passing between them -- " + "keyed as real AFSK through the real demodulator and the real " + "parsers. Nothing touches the receiver.", + flags=("--simulate",), off_flags=("--no-simulate",), + guidance="Turn this on to see what the whole thing does without an " + "aerial. Turn it off to hear real stations."), + + # -- listening ------------------------------------------------------ + O("seconds", "Listen for", "Listening", "float", + "how long to listen before stopping (0 = until interrupted)", + "A station beacons every few minutes, and a digipeater relays " + "everything it hears, so an hour gives a fair picture of a region and " + "a minute gives almost none. Everything heard is on the disk as it " + "arrives, so stopping never loses anything.", + unit="s", minimum=0.0, flags=("--seconds",), example="3600"), + O("log", "Write a log", "Listening", "bool", + "write every packet down as it arrives", + "One line of JSON per packet, holding the raw frame as well as what " + "was made of it. It is what the report, the spreadsheet and the map " + "are made from afterwards, and what lets a later version of this " + "program read a format this one cannot.", + flags=("--log",), off_flags=("--no-log",)), + O("packets_seen", "Print every packet", "Listening", "bool", + "one line per packet instead of a table that updates in place", + "The table is easier to watch and useless in a pipe or a file. Turn " + "this on for a plain stream of lines that can be piped somewhere, or " + "to watch individual packets arrive.", + flags=("--packets",), off_flags=("--no-packets",)), + O("hold", "Keep on screen for", "Listening", "float", + "how long a station stays on the display after its last packet", + "An hour by default. Stations beacon every few minutes when moving " + "and every half hour when not, so a short hold makes a fixed station " + "flicker in and out while a long one keeps a car on the screen after " + "it has driven out of range.", + unit="s", minimum=1.0, flags=("--hold",), example="3600"), + O("location", "Receiver at", "Listening", "text", + "where the aerial is, as latitude,longitude (blank = no distances)", + "Puts a red flag on the map where you are, draws the range rings " + "round it, and gives every station a distance and a bearing. Left " + "blank, whatever the aircraft side was told is used instead -- one " + "aerial on one roof does not move because the receiver was pointed at " + "a different band -- and if neither has been told, positions are " + "still recorded and drawn and there is simply nothing to measure them " + "from.", + flags=("--at",), example="47.55,-122.30", metavar="LAT,LON"), + + # -- what to show --------------------------------------------------- + O("lookup", "Look up callsigns", "Showing", "bool", + "find out who each station is licensed to, and where", + "An APRS callsign is issued by a government, so unlike a weather " + "sensor it can be looked up: the name on the licence, the town, and " + "the licensed position, which is a street address where a beacon " + "only gives a grid square. Answers are kept in " + "~/.cache/bandsaunter/callsigns.json and shared with every other " + "part of this program that looks a callsign up, so the same net " + "logged night after night is asked about once a month. The lookup " + "never holds up the display -- it happens on its own thread and the " + "name appears when it arrives. The databases are United States " + "registers, so stations elsewhere come back unlisted; the country " + "and district shown beside them come from the callsign's own " + "structure and need no network at all.", + flags=("--lookup",), off_flags=("--no-lookup",)), + O("units", "Show readings in", "Showing", "choice", + "metric or imperial, for the display and the export", + "Only what is shown. APRS is imperial almost throughout -- knots, " + "feet, miles, Fahrenheit -- and all of it is converted once on the " + "way in, so the log holds one kind of number and this can be changed " + "afterwards on an old log.", + choices=("metric", "imperial"), flags=("--units",), example="metric"), + O("unparsed", "Show what cannot be read", "Showing", "bool", + "list packets whose format this does not understand", + "APRS has about twenty formats and a long tail of things one radio " + "manufacturer did once. A packet that defeats every parser here still " + "arrived, still came from a real station, and still has its text; " + "showing it is how the next format gets added.", + flags=("--unparsed",), off_flags=("--no-unparsed",)), + O("digipeated", "Include relayed packets", "Showing", "bool", + "count packets that reached here through a digipeater", + "Most of what a receiver hears is not the original transmission but a " + "hilltop repeating it, which is the whole design. Turning this off " + "leaves only what was heard directly, which is a much shorter list " + "and is the honest measure of what the aerial can reach.", + flags=("--digipeated",), off_flags=("--direct-only",), + guidance="Leave it on for a picture of the network; turn it off to " + "find out what you can actually hear."), + + # -- the map -------------------------------------------------------- + O("radius", "Map reaches", "The map", "float", + "how far around the aerial the window reaches, in the unit being shown", + "Which is also the zoom. Fifty kilometres is a region; five is a town " + "and a digipeater at the edge of it falls off the picture. Stations " + "outside it are still heard, still logged and still on the map drawn " + "afterwards \u2014 they are simply off the edge of the window. " + "In whatever unit Show readings in is set to: kilometres in metric, " + "statute miles in imperial. No suffix is printed against it here " + "because the suffix is the other setting's to give, and a number " + "labelled km while the rings beside it are drawn in miles would be " + "worse than one labelled nothing.", + minimum=1.0, maximum=2000.0, flags=("--radius",), + example="50"), + O("theme", "Theme", "The map", "choice", + "how the window looks", + "The same five the aircraft map has: night, the default, with a " + "blue-grey ground; and the vector-display themes digital, phosphor, " + "amber and red, which draw thin bright lines with a halo on black.", + choices=("night", "digital", "phosphor", "amber", "red"), + flags=("--theme",), example="night"), + O("map_brightness", "Map brightness", "The map", "int", + "how bright the ground under the stations is (10-100)", + "The map is there to say where things are, not to be looked at, so it " + "sits well back. Turned up it is a photograph of a county with marks " + "on it; turned down it is a hint of coastline.", + unit="%", minimum=10, maximum=100, flags=("--map-brightness",), + example="70"), + O("basemap", "Draw a real map", "The map", "bool", + "fetch map tiles to draw under the stations", + "Coastlines, roads and town names from OpenStreetMap, cached on disk " + "after the first time an area is drawn. Turned off, the window is a " + "graticule and the stations on it, which is enough to see where " + "things are relative to each other and not where they are.", + flags=("--basemap",), off_flags=("--no-basemap",)), + O("window_rings", "Range rings", "The map", "bool", + "faint discs at a quarter, a half and three quarters of the radius", + "Labelled with the distance, concentric on the aerial, to give a " + "sense of how far off things are without measuring. Needs somewhere " + "to be concentric about, so it does nothing until the receiver's " + "position is set.", + flags=("--window-rings",), off_flags=("--no-window-rings",)), + O("box_opacity", "Box translucency", "The map", "int", + "how solid the card behind each information box is (0-100)", + "Nothing at all puts the words straight on the map, which reads well " + "over water and badly over a city.", + unit="%", minimum=0, maximum=100, flags=("--box-opacity",), + example="85"), + O("trails", "Leave a trail", "The map", "bool", + "draw the path behind anything that moved", + "A line through every position a station has reported. Fixed stations " + "leave none, because they have not been anywhere.", + flags=("--trails",), off_flags=("--no-trails",)), + O("tile_url", "Map tiles from", "The map", "text", + "where map tiles come from ({z}/{x}/{y}.png); blank is OpenStreetMap", + "Worth setting for a local tile server, or for a style that suits the " + "theme better than the default one does.", + flags=("--tiles",), example="", metavar="URL"), + + # -- afterwards ----------------------------------------------------- + O("report", "Report at the end", "Afterwards", "bool", + "print what each station said when the listening stops", + "A table of stations heard with where they were, how far off, how " + "often they spoke and how well they came in, then the messages that " + "passed between them.", + flags=("--report",), off_flags=("--no-report",)), + O("csv", "Also write a spreadsheet", "Afterwards", "bool", + "write the packets as CSV beside the log", + "A row per packet with the position, the speed and the weather in " + "their own columns, which is the shape anything that draws a graph " + "or a track wants.", + flags=("--csv",), off_flags=("--no-csv",)), + O("kml", "Also write a map", "Afterwards", "bool", + "write the stations and their tracks for Google Earth", + "A placemark per station where it was last heard, and a line for " + "anything that moved. The same writer the callsign map uses.", + flags=("--kml",), off_flags=("--no-kml",)), +) + +OPTION_GROUPS = ("Receiver", "Listening", "Showing", "The map", "Afterwards") + + +def in_group(group: str) -> list[Setting]: + return [o for o in OPTIONS if o.group == group] + + +def by_key(key: str) -> Setting | None: + return next((o for o in OPTIONS if o.key == key), None) + + +_ZERO_MEANS = {"seconds": "until stopped"} + + +def format_option(option: Setting, value) -> str: + if not value and option.key in _ZERO_MEANS: + return _ZERO_MEANS[option.key] + return format_value(option, value) + + +def describe(options: AprsOptions) -> str: + how_long = ("until stopped" if not options.seconds + else f"{options.seconds:g} s") + where = "simulated" if options.simulate else channel_text(options) + return (f"{where}, {how_long}, " + f"{'everything' if options.digipeated else 'direct only'}, " + f"{options.units}") + + +def use_region(options: AprsOptions, name: str) -> AprsOptions: + """Point the receiver at a region's channel: both settings, together. + + The region and the frequency are separate settings because somebody may + want a channel that is not any region's, and that means they can be made + to disagree. Everywhere a region is chosen goes through here, so they + cannot be. + """ + options.region = name + options.frequency = channel_named(name) + return options + + +def channel_text(options: AprsOptions) -> str: + """The channel, named as well as numbered. + + Both, always, because the number alone does not say whether it is the + right one and the name alone does not say what will be tuned. This is + the setting that decides whether anything is heard at all, so it is + never shown as half of itself. + """ + megahertz = f"{options.frequency / 1e6:g} MHz" + for key, hz, _where in ax25.APRS_CHANNELS: + if abs(hz - options.frequency) < 1.0: + return f"{key} {megahertz}" + return f"{megahertz} (not a region's channel)" + + +def summarise(options: AprsOptions) -> str: + return describe(options) + + +def options_path(directory=None) -> Path: + from .config import DEFAULT_CONFIG_DIR + + return Path(directory or DEFAULT_CONFIG_DIR) / "aprs.yaml" + + +def load_options(directory=None) -> AprsOptions: + """The saved options, or the defaults. A broken file is not an error.""" + options = AprsOptions() + try: + body = yaml.safe_load(options_path(directory).read_text()) or {} + except (OSError, ValueError, yaml.YAMLError): + return options + if not isinstance(body, dict): + return options + known = set(options.__dict__) + for key, value in body.items(): + if key in known and value is not None: + try: + setattr(options, key, type(getattr(options, key))(value)) + except (TypeError, ValueError): + pass + return options + + +def save_options(options: AprsOptions, directory=None) -> Path: + path = options_path(directory) + path.parent.mkdir(parents=True, exist_ok=True) + with open(path, "w") as fh: + yaml.safe_dump(options.to_dict(), fh, sort_keys=False, + default_flow_style=False) + return path + + +def logs_in(directory) -> list[Path]: + """Every APRS log in a directory, newest first.""" + try: + found = list(Path(directory).expanduser().glob("aprs_*.jsonl")) + except OSError: + return [] + return sorted(found, key=lambda p: p.stat().st_mtime, reverse=True) + + +def coordinates(text: str) -> tuple[float, float] | None: + try: + lat, lon = (float(x) for x in str(text).split(",", 1)) + except (TypeError, ValueError): + return None + return (lat, lon) if abs(lat) <= 90 and abs(lon) <= 180 else None + + +def map_unit(options: AprsOptions) -> str: + """Which unit the window measures in: statute miles or kilometres. + + The same setting that picks the unit for every reading in the tables + picks it here. A window that said kilometres round its rings while the + table beside it said miles would be two answers to one question. + """ + return "mph" if options.imperial else "kph" + + +def radius_in_nm(options: AprsOptions) -> float: + """How far the window reaches, as the drawing wants it. + + The setting is in whatever unit is being shown -- kilometres in metric, + statute miles in imperial -- so that the number typed in and the number + written beside the outermost ring are the same number. Everything + inside the drawing is in nautical miles, that being a minute of + latitude and so the unit the geometry is already in. + """ + from .flightlog import speed_unit + + return max(1.0, float(options.radius)) / speed_unit(map_unit(options))[3] + + +def base_call(call: str) -> str: + """The licensed callsign inside an APRS one. + + The rules are shared with every other mode that hears callsigns -- an + SSID here, a rover suffix on FT8, a guest prefix anywhere -- so they + live in one place rather than being written out per band and getting + quietly different. + """ + from .callsign import licensed_call + + return licensed_call(call) + + +def licensee(net: "Net", call: str): + """Who a callsign belongs to, as far as is known right now. + + Never waits. The lookup runs on its own thread and the answer appears + in a later frame of the display, which redraws twice a second anyway; + a table that stopped for a network request would stop for every new + station on a busy channel. + """ + if net is None or net.book is None: + return None + bare = base_call(call) + return net.book.get(bare) if bare else None + + +def _keep_lookups(console, net) -> None: + """Write the callsign answers to the disk before the run ends. + + Without this the lookups happen and are then thrown away, so every + evening asks the register about the same net again -- which is the one + thing caching them was for, and is invisible from inside a single run + because the answers are all there in memory while it lasts. + + Outstanding lookups are given a moment to land first. A station heard + in the last second of a run is still worth knowing tomorrow. + """ + book = getattr(net, "book", None) + if book is None: + return + try: + book.wait(3.0) + book.save() + except Exception: + pass # a cache that cannot be written is not a failed run + + +def who_text(who) -> str: + """One line about whose licence a callsign is, for a table cell. + + The name and the town when the register knows them; otherwise the + country and district the callsign itself implies, which is worth + printing because it needs no network and is true everywhere -- a + station in Australia will never be in a United States register, and + leaving its cell blank would read as a lookup that failed rather than + as a database that was never going to have it. + """ + if who is None: + return "" + if who.name: + where = who.location or who.country + return f"{who.name}" + (f" — {where}" if where else "") + if who.status == "pending": + return "…" + bits = [b for b in (who.country, who.district) if b] + return " ".join(bits[:1]) if bits else "" + + +def open_book(options: AprsOptions): + """The callsign register, or nothing if lookups are turned off. + + Offline rather than absent when they are off: an offline book still + answers the country and the licensing district, which come out of the + callsign's own structure and need no network, and which are most of + what there is to say about a station outside the United States anyway. + """ + from .callsign import CallsignBook + + return CallsignBook(online=bool(options.lookup)) + + +def receiver_at(options: AprsOptions) -> tuple[float, float] | None: + """Where the aerial is standing, from here or from the aircraft settings. + + One aerial, one roof, one position. Which band it is pointed at today + does not move it, so somebody who told the aircraft side where they are + should not have to tell this side the same thing again -- and, having + already done it once, would reasonably expect the flag to be there. + + This section's own setting wins where it has one, because two receivers + in two places is a thing that happens and is exactly what the separate + setting is for. + """ + here = coordinates(options.location) + if here is not None: + return here + from . import aircraft as air + + return coordinates(air.load_options().location) + + +# --------------------------------------------------------------------------- +# How far off, and which way +# --------------------------------------------------------------------------- + +EARTH_KM = 6371.0088 + + +def distance_km(from_: tuple, to: tuple) -> float: + """Great-circle distance, by the haversine, in kilometres.""" + lat1, lon1 = math.radians(from_[0]), math.radians(from_[1]) + lat2, lon2 = math.radians(to[0]), math.radians(to[1]) + a = (math.sin((lat2 - lat1) / 2) ** 2 + + math.cos(lat1) * math.cos(lat2) * math.sin((lon2 - lon1) / 2) ** 2) + return 2.0 * EARTH_KM * math.asin(min(1.0, math.sqrt(a))) + + +def bearing_deg(from_: tuple, to: tuple) -> float: + """Initial bearing from one place to another, in degrees true.""" + lat1, lon1 = math.radians(from_[0]), math.radians(from_[1]) + lat2, lon2 = math.radians(to[0]), math.radians(to[1]) + east = math.sin(lon2 - lon1) * math.cos(lat2) + north = (math.cos(lat1) * math.sin(lat2) + - math.sin(lat1) * math.cos(lat2) * math.cos(lon2 - lon1)) + return math.degrees(math.atan2(east, north)) % 360.0 + + +# --------------------------------------------------------------------------- +# Who is out there, and what they said +# --------------------------------------------------------------------------- + +# How much of a moving station's track to keep in memory. A car beaconing +# every twenty seconds for an evening is a few hundred points; this is a +# ceiling rather than a target, and the log has every one of them regardless. +TRACK_POINTS = 600 + + +@dataclass +class Station: + """One callsign, and the latest of everything it has said. + + The latest of each *kind* of thing, not the latest packet: a station may + send its position, then a status, then a weather report, and a display + showing only the newest packet would lose two of the three every time. + """ + + call: str = "" + kind: str = "" # what it mostly sends + symbol: str = "" + position: object = None # a packets.Position + course: float | None = None + speed: float | None = None + altitude: float | None = None + weather: dict = field(default_factory=dict) + status: str = "" + comment: str = "" + path: tuple = () + first: float = 0.0 + last: float = 0.0 + packets: int = 0 + direct: int = 0 # heard without a digipeater in between + snr: float = 0.0 + best_snr: float = 0.0 + kinds: set = field(default_factory=set) + track: list = field(default_factory=list) # (when, lat, lon) + object_of: str = "" # the station that placed this, if any + + @property + def moving(self) -> bool: + return bool(self.speed) and self.speed > 1.0 + + @property + def gap(self) -> float: + """The average wait between packets, in seconds, or zero.""" + return (self.last - self.first) / (self.packets - 1) \ + if self.packets > 1 else 0.0 + + def add(self, packet) -> None: + self.kind = packet.kind if packet.kind != "unparsed" else self.kind + self.kinds.add(packet.kind) + self.first = self.first or packet.at + self.last = max(self.last, packet.at) + self.packets += 1 + if not any("*" in hop for hop in packet.path): + self.direct += 1 + if packet.snr: + self.snr = packet.snr + self.best_snr = max(self.best_snr, packet.snr) + if packet.path: + self.path = packet.path + if packet.position is not None: + self.position = packet.position + self.symbol = packet.position.symbol or self.symbol + here = (packet.at, packet.position.latitude, + packet.position.longitude) + if not self.track or self.track[-1][1:] != here[1:]: + self.track.append(here) + del self.track[:-TRACK_POINTS] + for name in ("course", "speed", "altitude"): + value = getattr(packet, name) + if value is not None: + setattr(self, name, value) + if packet.weather: + self.weather.update(packet.weather) + if packet.status: + self.status = packet.status + if packet.comment: + self.comment = packet.comment + + def away(self, home) -> tuple[float, float] | None: + """How far off and in which direction, if there is a home to measure + from and a position to measure to.""" + if home is None or self.position is None: + return None + there = (self.position.latitude, self.position.longitude) + return distance_km(home, there), bearing_deg(home, there) + + +class Net: + """Every station heard, and every message that passed between them. + + The messages are kept apart from the stations on purpose. A message is + an event between two callsigns rather than a property of either, and a + conversation read in order is the thing somebody actually wants to see. + """ + + def __init__(self, book=None): + self.stations: dict[str, Station] = {} + self.messages: list = [] + self.packets = 0 + self.unparsed = 0 + # Where "who is this" is answered from. Shared with every other + # part of this program that resolves a callsign, and with every + # previous evening, because it is the one on the disk. + self.book = book + + def __len__(self) -> int: + return len(self.stations) + + def add(self, packet) -> Station: + name = packet.station + station = self.stations.get(name) + if station is None: + station = self.stations[name] = Station(call=name) + if packet.name: + station.object_of = packet.source + # Asked once, when the station is first heard, and never again: + # the book remembers, on the disk, for a month. An object + # placed by somebody is not itself licensed to anybody, so only + # real stations are looked up. + elif self.book is not None: + bare = base_call(name) + if bare: + self.book.get(bare) + station.add(packet) + self.packets += 1 + if packet.kind == "unparsed": + self.unparsed += 1 + if packet.message is not None: + self.messages.append(packet) + return station + + def showing(self, hold: float, now: float | None = None) -> list[Station]: + now = time.time() if now is None else now + alive = [s for s in self.stations.values() if now - s.last <= hold] + return sorted(alive, key=lambda s: (-s.last, s.call)) + + def all(self) -> list[Station]: + return sorted(self.stations.values(), key=lambda s: (s.first, s.call)) + + @classmethod + def of(cls, heard) -> "Net": + net = cls() + for packet in heard: + net.add(packet) + return net + + +# --------------------------------------------------------------------------- +# Listening +# --------------------------------------------------------------------------- + +@dataclass +class Heard: + """What one listening session came to.""" + + packets: int = 0 + net: object = None + log_path: Path | None = None + csv_path: Path | None = None + kml_path: Path | None = None + + @property + def stations(self) -> int: + return len(self.net) if self.net is not None else 0 + + +def open_device(console, options: AprsOptions): + """The receiver, or an invented channel, or None if neither can be had.""" + from rich.panel import Panel + from rich.text import Text + + from .device import RtlSdrDevice, RtlSdrError + + if options.simulate: + console.print("[yellow]simulated: these stations are not there." + "[/yellow]") + return SimulatedChannel(sample_rate=options.rate, realtime=True).open() + try: + device = RtlSdrDevice(index=options.device, + sample_rate=int(options.rate), + gain=options.gain, + agc=options.gain == "auto") + device.open() + except RtlSdrError as exc: + console.print(Panel(Text(str(exc)), + title="[red]cannot open the receiver", + border_style="red")) + return None + return device + + +def open_log(console, options: AprsOptions, output_dir: str, + started: float, log_path=None): + """The packet log, or None if it was not wanted or cannot be written.""" + from .aprslog import AprsLog + + if not options.log: + return None + stamp = datetime.fromtimestamp(started).strftime("%Y-%m-%d_%H_%M_%S") + where = Path(log_path).expanduser() if log_path else \ + Path(output_dir).expanduser() / f"aprs_{stamp}.jsonl" + try: + return AprsLog(where, frequency=options.frequency, + sample_rate=options.rate, + receiver="simulated" if options.simulate else + f"device {options.device}", started=started) + except OSError as exc: + console.print(f"[red]cannot write {where}: {exc}[/red]") + return None + + +def make_receiver(options: AprsOptions): + """The FM demodulator and the packet receiver, wired together. + + De-emphasis is turned off, which matters and is easy to miss. Voice FM + lifts the high frequencies on transmit and drops them again on receive; + packet radio takes its audio from the discriminator *before* that + happens, because dropping the highs would take four decibels off the + 2200 Hz tone and leave the 1200 Hz one alone -- which is exactly the + difference the demodulator here is measuring. + """ + from .demod import make_demodulator + + demod = make_demodulator("nfm", int(options.rate), bandwidth=16_000.0, + audio_rate=22_050, deemphasis=None) + return demod, ax25.Receiver(float(demod.audio_rate)) + + +def pump(device, options: AprsOptions, net: Net, log, started: float, + on_block=None, on_packet=None, on_samples=None, + stopping=None) -> int: + """Read the receiver until it stops, or until told to. + + The one loop both front ends are driven from, so that what is written to + the log cannot depend on which one you happened to be looking at. + """ + demod, receiver = make_receiver(options) + total = 0 + device.tune(options.frequency) + block = int(options.rate) # a second at a time + while stopping is None or not stopping(): + at = time.time() + samples = device.read_samples(block) + if samples is None or samples.size == 0: + break + if on_samples is not None: + on_samples(samples, at) + audio, iq = demod.step(samples) + if audio.size == 0: + continue + for frame in receiver.feed(audio, when=at, snr=_level(iq)): + packet = packets.parse(frame, now=at) + if packet.kind == "unparsed" and not options.unparsed: + continue + if not options.digipeated and frame.heard_through: + continue + net.add(packet) + total += 1 + if log is not None: + log.append(packet, frame) + if on_packet is not None: + on_packet(packet, frame) + if on_block is not None: + on_block(total) + if options.seconds and time.time() - started >= options.seconds: + break + return total + + +def _level(iq) -> float: + """How far the channel stood above its own quiet level, in decibels. + + A ratio off one receiver in one second, and not a power at the aerial -- + a dongle has no reference level and, on automatic gain, no fixed gain + either. Enough to compare two stations at the same moment and to point + an aerial, which is what it is for. + """ + import numpy as np + + if iq is None or iq.size < 64: + return 0.0 + magnitude = np.abs(iq) + quiet = float(np.percentile(magnitude, 20)) + loud = float(np.percentile(magnitude, 95)) + if quiet <= 0.0 or loud <= quiet: + return 0.0 + return 20.0 * math.log10(loud / quiet) + + +# --------------------------------------------------------------------------- +# A channel that is not there +# --------------------------------------------------------------------------- + +@dataclass +class VirtualStation: + """An amateur station that does not exist, doing what one does. + + It has a callsign, a place to be, possibly a speed to get somewhere at, + and a reason to transmit. What it sends goes out through the real + encoders, is keyed as real Bell 202 audio, is modulated onto a real FM + carrier and comes back through the real demodulator and the real + parsers -- so nothing downstream can tell the difference, which is the + point, because everything downstream is what is being exercised. + """ + + call: str = "N0CALL" + kind: str = "beacon" # beacon, car, weather, object, message + latitude: float = 47.55 + longitude: float = -122.30 + course: float = 90.0 + speed: float = 0.0 # km/h + symbol: str = "/-" + comment: str = "" + every: float = 120.0 + phase: float = 0.0 + strength: float = 1.0 + path: tuple = ("WIDE1-1", "WIDE2-1") + talks_to: str = "" + turn: int = 0 + + def frame(self, at: float) -> bytes: + """One transmission, as the bytes that go on the air.""" + self.turn += 1 + stamp = time.strftime("%d%H%Mz", time.gmtime(at)) + if self.kind == "weather": + info = packets.weather_report( + self.latitude, self.longitude, timestamp=stamp, + wind_from=(self.turn * 37) % 360, wind=11.0 + self.turn % 7, + gust=18.0, temperature=14.5 + (self.turn % 5) * 0.4, + rain_hour=0.0, humidity=62, pressure=1013.2) + return ax25.frame_bytes(self.call, "APRS", info, self.path) + if self.kind == "message" and self.talks_to: + info = packets.message_text(self.talks_to, + f"testing {self.turn}", + number=str(self.turn)) + return ax25.frame_bytes(self.call, "APRS", info, self.path) + if self.kind == "object": + info = packets.object_report(f"EVENT{self.turn % 3}", + self.latitude + 0.02, + self.longitude - 0.02, + symbol="/:", timestamp=stamp) + return ax25.frame_bytes(self.call, "APRS", info, self.path) + if self.kind == "car": + # A mobile, which in the real world means Mic-E. + destination, info = packets.mic_e_report( + self.latitude, self.longitude, course=self.course, + speed=self.speed, symbol=self.symbol, status="en route", + comment=self.comment) + return ax25.frame_bytes(self.call, destination, info, self.path) + info = packets.position_report(self.latitude, self.longitude, + self.symbol, self.comment, + timestamp=stamp) + return ax25.frame_bytes(self.call, "APRS", info, self.path) + + def advance(self, seconds: float) -> None: + if self.speed <= 0.0: + return + north = math.cos(math.radians(self.course)) * self.speed * seconds / 3600.0 + east = math.sin(math.radians(self.course)) * self.speed * seconds / 3600.0 + self.latitude += north / 111.32 + self.longitude += east / (111.32 * max(0.2, math.cos( + math.radians(self.latitude)))) + self.course = (self.course + math.sin(self.turn * 0.7) * 3.0) % 360.0 + + +def default_stations() -> list[VirtualStation]: + """A channel's worth of them, in a region that does not exist.""" + return [ + VirtualStation(call="W1AW-1", kind="beacon", latitude=47.6062, + longitude=-122.3321, symbol="/#", every=600.0, + phase=5.0, comment="digipeater", path=()), + VirtualStation(call="KU0W-9", kind="car", latitude=47.5480, + longitude=-122.2960, course=70.0, speed=54.0, + symbol="/>", every=45.0, phase=12.0, + comment="mobile"), + VirtualStation(call="N0ABC-7", kind="car", latitude=47.6200, + longitude=-122.3500, course=200.0, speed=32.0, + symbol="/b", every=60.0, phase=27.0, strength=0.5), + VirtualStation(call="KB1XYZ", kind="weather", latitude=47.5900, + longitude=-122.2700, every=300.0, phase=40.0), + VirtualStation(call="W7ABC-10", kind="object", latitude=47.5700, + longitude=-122.3100, every=420.0, phase=70.0), + VirtualStation(call="KU0W-9", kind="message", talks_to="W1AW-1", + latitude=47.5480, longitude=-122.2960, every=240.0, + phase=95.0), + VirtualStation(call="VE3XYZ-5", kind="beacon", latitude=47.4900, + longitude=-122.4100, symbol="\\k", every=900.0, + phase=130.0, strength=0.35, comment="portable"), + ] + + +class SimulatedChannel: + """A receiver-shaped source of an amateur band that is not busy. + + Answers ``read_samples`` the way the dongle does, with real AFSK on a + real FM carrier, so the whole section -- the demodulator, the framer, + every parser, the log, the report and the map -- runs through without an + aerial or a licence. + """ + + def __init__(self, stations=None, sample_rate: float = 240_000.0, + noise: float = 0.05, seed: int = 0, realtime: bool = False, + deviation: float = 3_000.0): + import numpy as np + + self.stations = list(stations) if stations is not None \ + else default_stations() + self.sample_rate = float(sample_rate) + self.noise = noise + self.deviation = deviation + self.rng = np.random.default_rng(seed) + self.carrier = ax25.APRS_HZ # where these invented stations are + self.frequency = ax25.APRS_HZ # where the receiver is pointed + self.clock = 0.0 + self.realtime = realtime + self._last = None + # Whatever of a transmission did not fit in the block it began in. + # A packet is most of a second and a block is about one, so a + # transmission straddling the boundary is not an edge case -- it is + # most of them, and dropping the remainder means the receiver sees + # the first tenth of a frame and nothing decodes at all. This was + # invisible for a while because a simulated clock that advances in + # exact seconds puts every transmission at the start of a block. + self._pending = np.zeros(0, dtype=np.complex64) + + def open(self): + return self + + def close(self) -> None: + return None + + def tune(self, hz: float, settle: bool = True) -> int: + self.frequency = hz + return int(hz) + + def read_samples(self, count: int, flush: bool = False): + import numpy as np + + seconds = count / self.sample_rate + if self.realtime: + now = time.monotonic() + if self._last is not None: + behind = seconds - (now - self._last) + if behind > 0: + time.sleep(behind) + now = time.monotonic() + seconds = max(0.0, now - self._last) + self._last = now + block = np.zeros(count, dtype=np.complex64) + if self._pending.size: + take = min(self._pending.size, count) + block[:take] += self._pending[:take] + self._pending = self._pending[take:] + began, ended = self.clock, self.clock + seconds + # Only what this channel actually carries. A simulated band that + # answered the same on every frequency would let the channel search + # below pass without ever having searched anything. + on_channel = abs(self.frequency - self.carrier) <= 10_000.0 + for station in self.stations: + for at in (_due(station, began, ended) if on_channel else ()): + burst = self._keyed(station) + start = int((at - began) * self.sample_rate) + if start >= count: + self._hold(burst, start - count) + continue + room = min(burst.size, count - start) + block[start:start + room] += burst[:room] + if burst.size > room: + self._hold(burst[room:], 0) + station.advance(seconds) + self.clock = ended + if self.noise: + block = block + self.noise * ( + self.rng.standard_normal(count) + + 1j * self.rng.standard_normal(count)).astype(np.complex64) + return block.astype(np.complex64) + + def _hold(self, samples, offset: int) -> None: + """Keep the rest of a transmission for the block it runs into.""" + import numpy as np + + need = offset + samples.size + if self._pending.size < need: + self._pending = np.concatenate( + (self._pending, + np.zeros(need - self._pending.size, dtype=np.complex64))) + self._pending[offset:need] += samples + + def _keyed(self, station: VirtualStation): + """One transmission as a receiver would see it: AFSK inside FM.""" + import numpy as np + + audio = ax25.modulate(station.frame(self.clock), self.sample_rate, + amplitude=0.7, quiet_ms=10.0) + turn = 2.0 * math.pi * self.deviation / self.sample_rate + phase = np.cumsum(audio.astype(np.float64)) * turn + return (station.strength * np.exp(1j * phase)).astype(np.complex64) + + def read_seconds(self, seconds: float, flush: bool = False): + return self.read_samples(int(self.sample_rate * seconds)) + + +def _due(station: VirtualStation, began: float, ended: float) -> list[float]: + """Every moment in a block at which one station is due to transmit.""" + if station.every <= 0 or ended <= began: + return [] + first = math.ceil((began - station.phase) / station.every) + out, at = [], station.phase + first * station.every + while at < ended: + if at >= began: + out.append(at) + at += station.every + return out + + +# --------------------------------------------------------------------------- +# A listening session, start to finish +# --------------------------------------------------------------------------- + +def listen(console, options: AprsOptions, output_dir: str, + log_path=None, device=None) -> Heard: + """Park on the APRS channel and write down everything that passes. + + Everything heard goes into the log as it arrives, because a packet is a + two-second transmission and then gone: the screen is for the person + watching, and the log is for the report, the map and everything + afterwards. + """ + heard = Heard() + device = open_device(console, options) if device is None else device + if device is None: + return heard + + started = time.time() + log = open_log(console, options, output_dir, started, log_path) + net = Net(book=open_book(options)) + heard.net = net + _say_where(console, options, receiver_at(options)) + console.print(f"[grey62]listening on {options.frequency / 1e6:g} MHz at " + f"{options.rate / 1e6:g} MS/s — control-C to stop" + "[/grey62]") + if log is not None: + console.print(f"[grey62]writing {log.path}[/grey62]") + + display, live = _open_display(console, options, started) + + def on_block(total: int) -> None: + if live is not None: + display.update(net, total, + log.path if log is not None else None) + live.update(display.render()) + + def on_packet(packet, frame) -> None: + if options.packets_seen or live is None: + console.print(f"[cyan]{packet.station}[/cyan] " + f"{packet.describe(options.imperial)}", + highlight=False) + + total = 0 + try: + total = pump(device, options, net, log, started, + on_block=on_block, on_packet=on_packet) + except KeyboardInterrupt: + pass + finally: + if live is not None: + live.stop() + console.print("[grey62]stopped listening[/grey62]") + device.close() + if log is not None: + log.close() + heard.log_path = log.path + heard.packets = total + return finish(console, options, output_dir, heard) + + +def _say_where(console, options: AprsOptions, home) -> None: + """Where the aerial is taken to be, and where that came from. + + Said rather than assumed, because a position inherited from the aircraft + settings is a convenience right up until somebody has moved and only + changed one of them. + """ + if home is None: + console.print("[grey62]no receiver position set, so no flag, no " + "rings and no distances — `--at LAT,LON` gives all " + "three[/grey62]") + return + if not coordinates(options.location): + console.print(f"[grey62]receiver at {home[0]:.4f},{home[1]:.4f}, " + f"from the aircraft settings[/grey62]") + + +def _open_display(console, options: AprsOptions, started: float): + """A live table where there is a terminal to draw it on, else nothing.""" + if options.packets_seen or not getattr(console, "is_terminal", False): + return None, None + from rich.live import Live + + from .ui import AprsDisplay + + display = AprsDisplay(console, hold=options.hold, + imperial=options.imperial, + frequency=options.frequency, + home=receiver_at(options)) + display.started = started + live = Live(display.render(), console=console, refresh_per_second=2, + screen=False, transient=False, vertical_overflow="crop") + live.start() + return display, live + + +def finish(console, options: AprsOptions, output_dir: str, + heard: Heard) -> Heard: + """Say what was heard, and leave the files behind.""" + net = heard.net + # Before the early return: a run that heard one station and no more + # still learned who that station was, and throwing the answer away + # because the evening was quiet would mean asking again tomorrow. The + # book writes only if it has something new. + _keep_lookups(console, net) + if net is None or not len(net): + console.print("[yellow]nothing heard. APRS is a two-second " + "transmission every few minutes, so give it a while " + "— and check the region: on the wrong channel there " + "is silence rather than a bad signal.[/yellow]") + return heard + + if options.report: + report(console, net, options) + if heard.log_path is not None: + from .aprslog import read_logs, write_csv, write_kml + + if options.csv: + heard.csv_path = _write(console, write_csv, + heard.log_path.with_suffix(".csv"), + read_logs([heard.log_path]), options) + if options.kml: + heard.kml_path = _write(console, write_kml, + heard.log_path.with_suffix(".kml"), + read_logs([heard.log_path]), options) + console.print(f"[grey62]read it back: bandsaunter packets " + f"{heard.log_path}[/grey62]") + return heard + + +def _write(console, writer, where, heard, options): + try: + written = writer(where, heard, options.imperial) + except OSError as exc: + console.print(f"[red]cannot write {where}: {exc}[/red]") + return None + console.print(f"[green]wrote {written}[/green]") + return written + + +# --------------------------------------------------------------------------- +# What the evening came to +# --------------------------------------------------------------------------- + +SIGNAL_FAIR = 8.0 +SIGNAL_GOOD = 16.0 + + +def signal_text(level: float) -> str: + """One station's strength, coloured so an aerial can be aimed by it.""" + if not level: + return "" + colour = ("red" if level < SIGNAL_FAIR else + "yellow" if level < SIGNAL_GOOD else "green") + return f"[{colour}]{level:.0f} dB[/{colour}]" + + +def report(console, net: Net, options: AprsOptions) -> None: + """Three tables: who was heard, what they said, and what passed between. + + Split because they answer different questions. The first is about the + band and the aerial; the second is about the stations; the third is the + only part of APRS that is a conversation rather than a broadcast, and + reading it in order is the point of it. + """ + from rich.table import Table + + from .packets import height_text + from .acurite import Measure, format_measure + + home = receiver_at(options) + stations = sorted(net.all(), key=lambda s: (-s.packets, s.call)) + + t = Table(box=None, header_style="bold", pad_edge=False, + title="[bold]stations heard[/bold]", title_justify="left") + t.add_column("station", overflow="fold") + named = options.lookup and net.book is not None + if named: + t.add_column("licensed to", style="grey62", overflow="fold") + t.add_column("what", style="grey62", overflow="fold") + t.add_column("position", style="grey62", no_wrap=True) + if home is not None: + t.add_column("away", justify="right", no_wrap=True) + t.add_column("pkts", justify="right") + t.add_column("direct", justify="right", style="grey62") + t.add_column("signal", justify="right") + t.add_column("last heard", style="grey62", no_wrap=True) + for station in stations: + row = [station.call + (" (object)" if station.object_of else "")] + if named: + row.append(who_text(licensee(net, station.call)) + if not station.object_of else "") + row += [station.symbol or station.kind, + station.position.describe() if station.position else ""] + if home is not None: + away = station.away(home) + row.append("" if away is None else + f"{format_measure(Measure('', away[0], 'km'), options.imperial)}" + f" {_point(away[1])}") + row += [f"{station.packets:,}", f"{station.direct:,}", + signal_text(station.snr), + datetime.fromtimestamp(station.last).strftime("%H:%M:%S") + if station.last else ""] + t.add_row(*row) + console.print(t) + + said = [s for s in stations if s.status or s.comment or s.weather + or s.speed or s.altitude is not None] + if said: + w = Table(box=None, header_style="bold", pad_edge=False, + title="\n[bold]what they said[/bold]", title_justify="left") + w.add_column("station", overflow="fold") + w.add_column("moving", style="grey62", no_wrap=True) + w.add_column("and", overflow="fold") + for station in said: + moving = "" + if station.speed: + moving = (f"{_point(station.course or 0.0)} " + + format_measure(Measure("", station.speed, "km/h"), + options.imperial)) + if station.altitude is not None: + moving += (" " if moving else "") + height_text( + station.altitude, options.imperial) + words = " ".join( + [f"{name} {format_measure(m, options.imperial)}" + for name, m in station.weather.items()] + + [x for x in (station.status, station.comment) if x]) + w.add_row(station.call, moving, words) + console.print(w) + + if net.messages: + m = Table(box=None, header_style="bold", pad_edge=False, + title="\n[bold]what passed between them[/bold]", + title_justify="left") + m.add_column("when", style="grey62", no_wrap=True) + m.add_column("from", no_wrap=True) + m.add_column("message", overflow="fold") + for packet in net.messages[-40:]: + m.add_row(datetime.fromtimestamp(packet.at).strftime("%H:%M:%S"), + packet.source, packet.message.describe()) + console.print(m) + + if net.unparsed: + console.print(f"[grey62]{net.unparsed:,} packet" + f"{'s' if net.unparsed != 1 else ''} in a format this " + f"cannot read; their text is in the log[/grey62]") + + +def _point(degrees: float) -> str: + from .acurite import compass + + return compass(degrees) + + +# --------------------------------------------------------------------------- +# Finding the channel +# --------------------------------------------------------------------------- + +@dataclass +class Found: + """What one region's channel had on it while it was listened to.""" + + region: str = "" + frequency: float = 0.0 + where: str = "" + packets: int = 0 + stations: int = 0 + best: float = 0.0 # the strongest packet, in dB + + @property + def busy(self) -> bool: + return self.packets > 0 + + +def find_channel(console, options: AprsOptions, seconds: float = 20.0, + device=None) -> list[Found]: + """Listen on each region's channel in turn and say where the traffic is. + + The one question about APRS that cannot be answered from the air on any + single frequency, because the answer *is* a frequency. The channel is + agreed between amateurs rather than allocated, so being on the wrong one + sounds exactly like having no aerial: silence. Somebody who has just + plugged a dongle in has no way to tell those two apart, and that is worth + a minute of listening rather than an evening of doubt. + + A minute, and not longer, is the honest cost. A quiet channel after + twenty seconds is not proof of an empty one -- a fixed station beacons + every half hour -- so what this finds is traffic, and what it fails to + find is only the absence of traffic in twenty seconds. The table says + so. + """ + from dataclasses import replace + + mine = device is None + device = open_device(console, options) if device is None else device + if device is None: + return [] + out: list[Found] = [] + try: + for region, hz, where in ax25.APRS_CHANNELS: + trial = replace(options, frequency=hz, seconds=seconds, + log=False, report=False, csv=False, kml=False) + net = Net() + console.print(f"[grey62]{region:<14} {hz / 1e6:>8.3f} MHz " + f"listening for {seconds:g} s…[/grey62]", + highlight=False) + best = [0.0] + + def loudest(packet, _frame, best=best) -> None: + best[0] = max(best[0], packet.snr) + + try: + pump(device, trial, net, None, time.time(), + on_packet=loudest) + except KeyboardInterrupt: + console.print("[grey62]stopped[/grey62]") + break + out.append(Found(region=region, frequency=hz, where=where, + packets=net.packets, stations=len(net), + best=best[0])) + finally: + if mine: + device.close() + return out + + +def report_channels(console, found: list[Found], seconds: float) -> Found | None: + """The table, and whichever channel had the most on it.""" + from rich.table import Table + + if not found: + return None + t = Table(box=None, header_style="bold", pad_edge=False, + title="[bold]what was on each channel[/bold]", + title_justify="left") + t.add_column("region") + t.add_column("frequency", justify="right", style="grey62") + t.add_column("packets", justify="right") + t.add_column("stations", justify="right") + t.add_column("strongest", justify="right") + t.add_column("used in", style="grey62", overflow="fold") + best = max(found, key=lambda f: (f.packets, f.best)) + for one in found: + style = "bold green" if one is best and one.busy else \ + ("white" if one.busy else "grey62") + t.add_row(f"[{style}]{one.region}[/{style}]", + f"{one.frequency / 1e6:.3f} MHz", + f"{one.packets:,}" if one.packets else "—", + f"{one.stations:,}" if one.stations else "—", + signal_text(one.best) or "—", one.where) + console.print(t) + if not best.busy: + console.print(f"[yellow]nothing on any of them in {seconds:g} " + f"seconds each. That is not proof of an empty band: a " + f"fixed station beacons every half hour, so a quiet " + f"channel may simply not have been spoken on yet. Try " + f"a longer listen, or check the aerial — a quarter-wave " + f"whip for 144 MHz is 49 cm.[/yellow]") + return None + console.print(f"[green]{best.region} had the most on it[/green] " + f"[grey62]— {best.packets:,} packet" + f"{'s' if best.packets != 1 else ''} from {best.stations} " + f"station{'s' if best.stations != 1 else ''}[/grey62]") + return best + + +# --------------------------------------------------------------------------- +# The window +# --------------------------------------------------------------------------- +# +# The same window the aircraft use, given marks that are not aeroplanes. It +# already knows how to fetch a map, place an information box where it does +# not cover anything, glide it when its owner moves, draw range rings and put +# a flag where the aerial is -- and none of that has anything to do with +# aviation. What it did not know is that a mark might not fade, might not +# point anywhere, and might be coloured by what it is rather than by how high +# it is; those are three small hooks rather than a second window. + +# Where each sort of station sits on the altitude ramp. The ramp because it +# is the one set of colours every theme defines: on the default map it runs +# warm to cold, and on the phosphor themes it runs dim to bright, so a +# digipeater stays distinguishable from a car on all five without a single +# colour being named here. +STATION_SHADE = { + "moving": 0.92, # the brightest thing: it is going somewhere + "digipeater": 0.74, # the infrastructure holding the net up + "weather": 0.50, + "object": 0.34, + "message": 0.34, + "fixed": 0.12, # a house that beacons twice an hour +} + +# Symbols that mean a station is part of the network rather than on it. +DIGI_SYMBOLS = ("digipeater", "numbered digipeater", "gateway", + "gateway station", "network node", "node", "repeater", + "mic-repeater") + + +def station_shade(station: Station) -> str: + """Which sort of thing this is, for choosing a colour and a shape.""" + if station.object_of: + return "object" + if station.symbol in DIGI_SYMBOLS: + return "digipeater" + if station.weather: + return "weather" + if station.moving: + return "moving" + return "fixed" + + +def blip_for(station: Station, home=None, imperial: bool = False, + who=None): + """One station as the window wants it: a place, a shape and a box.""" + from .flightmap import RAMP, RAMP_STEPS + from .livemap import Blip + + sort = station_shade(station) + shade = STATION_SHADE.get(sort, 0.5) + place = station.position + return Blip( + # The callsign goes in the identity field and the name field is left + # empty, because the box heading prints both and an APRS station has + # only the one name. An aeroplane has two -- a flight number and a + # 24-bit address -- and the heading is built for that. + icao=station.call, callsign="", + latitude=place.latitude if place else 0.0, + longitude=place.longitude if place else 0.0, + altitude_ft=int(round((station.altitude or 0.0) / 0.3048)), + ground_speed_kt=(station.speed or 0.0) / 1.852, + track_deg=station.course or 0.0, + messages=station.packets, + first_seen=station.first, last_seen=station.last, + shape="vehicle" if station.moving else "station", + colour_index=RAMP + int(round(shade * (RAMP_STEPS - 1))), + details=tuple(station_lines(station, home, imperial, who))) + + +def station_lines(station: Station, home=None, + imperial: bool = False, + who=None) -> list[tuple[str, str, str]]: + """What the information box says about one station. + + The same shape the aircraft boxes use -- a label, a value and a country + flag that is always empty here, because a callsign already says which + country and saying it twice would cost a line that the weather wants. + + ``who`` is the licence record, where one has arrived. It goes at the + top, because a name is what somebody reads a callsign to find out. + """ + from .acurite import Measure, compass, format_measure + from .packets import height_text + + out: list[tuple[str, str, str]] = [] + if who is not None: + if who.name: + out.append(("licensed to", who.name, "")) + where = who.location or who.country + if where: + out.append(("at", where, "")) + if station.object_of: + out.append(("placed by", station.object_of, "")) + if station.symbol: + out.append(("symbol", station.symbol, "")) + if station.position is not None: + # Where it actually said it was, in figures. A mark on a map shows + # where a station is and a number is what gets written down, read + # out over the air, or typed into something else -- and where the + # station blanked its minutes, the figures are the only place the + # vagueness shows. + out.append(("position", station.position.describe(), "")) + away = station.away(home) + if away is not None: + out.append(("away", f"{format_measure(Measure('', away[0], 'km'), imperial)}" + f" {compass(away[1])}", "")) + if station.moving: + out.append(("moving", f"{compass(station.course or 0.0)} " + f"{format_measure(Measure('', station.speed, 'km/h'), imperial)}", + "")) + if station.altitude is not None: + out.append(("altitude", height_text(station.altitude, imperial), "")) + for name, measure in list(station.weather.items())[:6]: + out.append((name, format_measure(measure, imperial), "")) + for label, words in (("status", station.status), + ("says", station.comment)): + if words: + out.append((label, words[:44], "")) + if station.path: + out.append(("via", ",".join(station.path)[:40], "")) + heard = f"{station.packets:,}" + if station.direct and station.direct < station.packets: + heard += f" ({station.direct:,} direct)" + out.append(("packets", heard, "")) + if station.snr: + out.append(("signal", f"{station.snr:.0f} dB", "")) + return out + + +def watch(console, options: AprsOptions, output_dir: str, + log_path=None, device=None) -> Heard: + """The same listening, in a window with a map in it. + + The receiver runs on its own thread and the window paints from a copy of + what it found, so a slow repaint can never cost a packet and a slow + network can never stop the picture moving. Everything else -- the log, + the report, the spreadsheet, the map written at the end -- is exactly + what listening without a window does, because it is the same code. + """ + import threading + + from rich.panel import Panel + from rich.text import Text + + from . import livemap + from .flightmap import set_theme + + heard = Heard() + if not livemap.available(): + console.print(Panel(Text(livemap.MISSING_QT), + title="[yellow]no window to open", + border_style="yellow")) + return heard + device = open_device(console, options) if device is None else device + if device is None: + return heard + + started = time.time() + log = open_log(console, options, output_dir, started, log_path) + net = Net(book=open_book(options)) + heard.net = net + home = receiver_at(options) + # Set before the window is built, because the window reads its colours + # out of the palette this writes. + set_theme(options.theme) + sky = livemap.Sky( + unit=map_unit(options), hold=options.hold, home=home, + radius_nm=radius_in_nm(options), + brightness=max(10, options.map_brightness) / 100.0, + fade=0.0, airports=False, rings=options.window_rings, + box_opacity=max(0, options.box_opacity) / 100.0, + # Stations accumulate: see Sky.fades. An evening of listening fills + # a map, which is the whole point of pointing a window at this band. + fades=False, + channel=f"{options.frequency / 1e6:g} MHz", + subject="on the map", counted="packets", + waiting=(f"listening on {options.frequency / 1e6:g} MHz\n\n" + "nothing placed yet — a station appears once it has said\n" + "where it is, which most do every few minutes")) + sky.started = started + sky.log_name = log.path.name if log is not None else "" + if options.simulate: + sky.note = "simulated" + + def on_block(total: int) -> None: + sky.update([blip_for(station, home, options.imperial, + licensee(net, station.call)) + for station in net.all() if station.position is not None], + total, len(net)) + + counted = {"packets": 0} + + def listening() -> None: + try: + counted["packets"] = pump(device, options, net, log, started, + on_block=on_block, + stopping=lambda: sky.stopping) + except Exception as exc: # a window must survive it + sky.note = str(exc)[:60] + finally: + sky.finished = True + + threads = [threading.Thread(target=listening, daemon=True, + name="aprs-receiver")] + if options.basemap: + threads.append(threading.Thread( + target=livemap.fetch_ground, args=(sky, options.tile_url), + daemon=True, name="aprs-basemap")) + for thread in threads: + thread.start() + _say_where(console, options, home) + console.print(f"[grey62]listening on {options.frequency / 1e6:g} MHz — " + "close the window to stop[/grey62]") + if log is not None: + console.print(f"[grey62]writing {log.path}[/grey62]") + try: + livemap.show(sky, f"bandsaunter — APRS on " + f"{options.frequency / 1e6:g} MHz") + finally: + sky.stopping = True + for thread in threads: + thread.join(timeout=3.0) + device.close() + if log is not None: + log.close() + heard.log_path = log.path + heard.packets = counted["packets"] + return finish(console, options, output_dir, heard) + + +def windowed() -> bool: + """Whether the realtime window can be opened on this machine.""" + from . import livemap + + return livemap.available() diff --git a/bandsaunter/aprslog.py b/bandsaunter/aprslog.py new file mode 100644 index 0000000..5a55428 --- /dev/null +++ b/bandsaunter/aprslog.py @@ -0,0 +1,345 @@ +"""Writing down what came over the channel, and reading it back. + +One line of JSON per packet, written the moment it arrives. Flushed after +every one, for the reason every log in this program is: a listening session +ends when the operator gets bored and presses control-C, and a log that only +reached the disk on a clean shutdown would be empty exactly when it was most +wanted. + +Each line keeps the whole AX.25 frame in hexadecimal alongside whatever was +made of it. APRS has a long tail of formats -- about twenty in the +specification and more that one manufacturer invented once -- so a packet +this version cannot read is still on the disk in full, and a later version +that can read it can go back through old logs and do so. That is not a +hypothetical: the reason the frame is kept is that the list of formats is +still growing. + +Out of it come two other things. A spreadsheet, because a track and a +temperature are both columns of numbers people want to plot; and a KML file, +because the natural thing to do with a hundred positions is to look at them +on a globe. +""" + +from __future__ import annotations + +import csv +import json +import time +from datetime import datetime +from pathlib import Path + +from .acurite import Measure +from .packets import Message, Packet, Position, Telemetry + +__all__ = ["AprsLog", "read_logs", "write_csv", "write_kml", "logs_in", + "LOG_VERSION"] + +LOG_VERSION = 1 + + +class AprsLog: + """A JSON Lines record of every packet heard, written as it arrives.""" + + def __init__(self, path, receiver: str = "", frequency: float = 0.0, + sample_rate: float = 0.0, started: float = 0.0): + self.path = Path(path) + self.packets = 0 + self.started = started or time.time() + self.path.parent.mkdir(parents=True, exist_ok=True) + self._file = self.path.open("a", encoding="utf8") + self._write({"log": "bandsaunter-aprs", "version": LOG_VERSION, + "started": round(self.started, 3), + "started_local": datetime.fromtimestamp( + self.started).strftime("%Y-%m-%d %H:%M:%S"), + "frequency": frequency, "sample_rate": sample_rate, + "receiver": receiver}) + + def _write(self, body: dict) -> None: + self._file.write(json.dumps(body, separators=(",", ":"), + ensure_ascii=False) + "\n") + self._file.flush() + + def append(self, packet: Packet, frame=None) -> None: + """Record one packet: what arrived, and what was made of it.""" + body: dict = {"t": round(packet.at or time.time(), 3), + "src": packet.source, "dst": packet.destination, + "kind": packet.kind, "info": packet.info} + if packet.path: + body["path"] = list(packet.path) + if frame is not None and frame.raw: + # The frame is the evidence; everything else in the line is an + # opinion about it, and the opinions may improve later. + body["hex"] = frame.raw.hex().upper() + if packet.snr: + body["snr"] = round(packet.snr, 1) + if packet.reported: + body["reported"] = round(packet.reported, 3) + if packet.position is not None: + body["lat"] = round(packet.position.latitude, 6) + body["lon"] = round(packet.position.longitude, 6) + body["sym"] = packet.position.table + packet.position.code + if packet.position.ambiguity: + body["vague"] = packet.position.ambiguity + for name in ("course", "speed", "altitude", "range", "power", + "height", "gain"): + value = getattr(packet, name) + if value is not None: + body[name] = round(float(value), 3) + for name in ("name", "status", "comment", "beam"): + value = getattr(packet, name) + if value: + body[name] = value + if not packet.live: + body["killed"] = True + if packet.weather: + body["wx"] = {n: [m.value, m.unit] + for n, m in packet.weather.items()} + if packet.message is not None: + body["msg"] = {k: v for k, v in vars(packet.message).items() if v} + if packet.telemetry is not None: + body["tlm"] = {"seq": packet.telemetry.sequence, + "a": list(packet.telemetry.analogue), + "d": packet.telemetry.digital} + self.packets += 1 + self._write(body) + + def close(self) -> None: + try: + self._file.close() + except OSError: + pass + + def __enter__(self) -> "AprsLog": + return self + + def __exit__(self, *exc) -> None: + self.close() + + +# --------------------------------------------------------------------------- +# Reading it back +# --------------------------------------------------------------------------- + +def logs_in(directory) -> list[Path]: + try: + found = list(Path(directory).expanduser().glob("aprs_*.jsonl")) + except OSError: + return [] + return sorted(found, key=lambda p: p.stat().st_mtime, reverse=True) + + +def read_logs(paths) -> list[Packet]: + """Every packet in one or more logs, in the order they were heard. + + A line that will not parse is skipped rather than fatal. A log is + appended to while the disk fills and the power goes off, so the last line + of one is quite often half a line. + """ + out: list[Packet] = [] + for path in ([paths] if isinstance(paths, (str, Path)) else paths): + try: + text = Path(path).expanduser().read_text(encoding="utf8") + except OSError: + continue + for line in text.splitlines(): + packet = _packet_from(line) + if packet is not None: + out.append(packet) + out.sort(key=lambda p: p.at) + return out + + +def _packet_from(line: str) -> Packet | None: + line = line.strip() + if not line: + return None + try: + body = json.loads(line) + except ValueError: + return None + if not isinstance(body, dict) or "src" not in body: + return None # the header line, or something else entirely + + packet = Packet(kind=str(body.get("kind", "unparsed")), + source=str(body.get("src", "")), + destination=str(body.get("dst", "")), + path=tuple(body.get("path") or ()), + at=float(body.get("t", 0.0) or 0.0), + reported=float(body.get("reported", 0.0) or 0.0), + snr=float(body.get("snr", 0.0) or 0.0), + info=str(body.get("info", ""))) + if "lat" in body and "lon" in body: + symbol = str(body.get("sym", "/-")) + packet.position = Position(latitude=float(body["lat"]), + longitude=float(body["lon"]), + ambiguity=int(body.get("vague", 0) or 0), + table=symbol[:1] or "/", + code=symbol[1:2] or "-") + for name in ("course", "speed", "altitude", "range", "power", "height", + "gain"): + if name in body: + setattr(packet, name, float(body[name])) + for name in ("name", "status", "comment", "beam"): + if name in body: + setattr(packet, name, str(body[name])) + packet.live = not body.get("killed", False) + for name, value in (body.get("wx") or {}).items(): + if isinstance(value, list) and value: + packet.weather[name] = Measure( + name, float(value[0]), str(value[1]) if len(value) > 1 else "") + if body.get("msg"): + packet.message = Message(**{k: str(v) + for k, v in body["msg"].items() + if k in vars(Message())}) + if body.get("tlm"): + told = body["tlm"] + packet.telemetry = Telemetry(sequence=str(told.get("seq", "")), + analogue=tuple(told.get("a") or ()), + digital=str(told.get("d", ""))) + return packet + + +# --------------------------------------------------------------------------- +# Out to a spreadsheet +# --------------------------------------------------------------------------- + +_IMPERIAL = {"C": "F", "km/h": "mph", "mm": "in", "km": "mi", "m": "ft"} + + +def write_csv(path, heard, imperial: bool = False) -> Path: + """A row per packet, with what it said in columns. + + The union of every weather quantity anything reported, so a channel with + one weather station on it has temperature and pressure columns and every + car leaves them empty. That is the shape a spreadsheet wants. + """ + path = Path(path).expanduser() + path.parent.mkdir(parents=True, exist_ok=True) + quantities: list[str] = [] + for packet in heard: + for name in packet.weather: + if name not in quantities: + quantities.append(name) + + speed_unit = "mph" if imperial else "km/h" + height_unit = "ft" if imperial else "m" + heads = ["time", "unix", "station", "source", "destination", "path", + "kind", "latitude", "longitude", "symbol", "course", + f"speed ({speed_unit})", f"altitude ({height_unit})", + "signal (dB)", "status", "comment"] \ + + [_column(name, heard, imperial) for name in quantities] + with open(path, "w", encoding="utf8", newline="") as fh: + out = csv.writer(fh) + out.writerow(heads) + for packet in heard: + place = packet.position + out.writerow([ + datetime.fromtimestamp(packet.at).isoformat(timespec="seconds") + if packet.at else "", + f"{packet.at:.3f}" if packet.at else "", + packet.station, packet.source, packet.destination, + ",".join(packet.path), packet.kind, + f"{place.latitude:.6f}" if place else "", + f"{place.longitude:.6f}" if place else "", + (place.table + place.code) if place else "", + f"{packet.course:.0f}" if packet.course is not None else "", + _shown(packet.speed, "km/h", imperial), + _shown(packet.altitude, "m", imperial), + f"{packet.snr:.1f}" if packet.snr else "", + packet.status, packet.comment] + + [_shown(packet.weather[name].value, + packet.weather[name].unit, imperial) + if name in packet.weather else "" + for name in quantities]) + return path + + +def _column(name: str, heard, imperial: bool) -> str: + unit = "" + for packet in heard: + if name in packet.weather and packet.weather[name].unit: + unit = packet.weather[name].unit + break + if imperial: + unit = _IMPERIAL.get(unit, unit) + return f"{name} ({unit})" if unit else name + + +def _shown(value, unit: str, imperial: bool) -> str: + """One number, in whichever system was asked for, as a plain figure. + + Plain because a column of "21.5 C" is text and a column of 21.5 is a + temperature, and only one of those can be plotted. + """ + if value is None: + return "" + if imperial: + if unit == "C": + value = value * 9 / 5 + 32 + elif unit in ("km/h", "km"): + value = value / 1.609344 + elif unit == "mm": + value = value / 25.4 + elif unit == "m": + value = value / 0.3048 + return str(round(float(value), 3)) + + +# --------------------------------------------------------------------------- +# Out to a globe +# --------------------------------------------------------------------------- + +def write_kml(path, heard, imperial: bool = False, + title: str = "bandsaunter — APRS stations") -> Path | None: + """A pin where each station was last heard, and a line where it moved. + + Only the stations that said where they were, because a pin at nowhere is + worse than no pin: it puts a station off the west coast of Africa, which + is where nought degrees by nought degrees is and is the reason that bug + has a name. + """ + from xml.sax.saxutils import escape as xml_escape + + tracks: dict[str, list] = {} + latest: dict[str, object] = {} + for packet in heard: + if packet.position is None: + continue + where = (packet.position.longitude, packet.position.latitude, + packet.altitude or 0.0) + line = tracks.setdefault(packet.station, []) + if not line or line[-1][:2] != where[:2]: + line.append(where) + latest[packet.station] = packet + if not latest: + return None + + path = Path(path).expanduser() + path.parent.mkdir(parents=True, exist_ok=True) + out = ['', + '', " ", + f" {xml_escape(title)}", + ' "] + for call, line in sorted(tracks.items()): + packet = latest[call] + told = xml_escape(packet.describe(imperial)) + name = xml_escape(call) + if len(line) > 1: + where = " ".join(f"{lon:.6f},{lat:.6f},{alt:.0f}" + for lon, lat, alt in line) + out += [" ", f" {name}", + f" {told}", + " #track", + " 1", + f" {where}", + " ", " "] + last = line[-1] + out += [" ", f" {name}", + f" {told}", + " " + f"{last[0]:.6f},{last[1]:.6f},{last[2]:.0f}" + "", " "] + out += [" ", "", ""] + path.write_text("\n".join(out), encoding="utf8") + return path diff --git a/bandsaunter/ax25.py b/bandsaunter/ax25.py new file mode 100644 index 0000000..904fb63 --- /dev/null +++ b/bandsaunter/ax25.py @@ -0,0 +1,524 @@ +"""AX.25 over the air: the frames APRS is carried in, and how to recover them. + +Amateur packet radio sends data as HDLC frames over a carrier that is simply +switched between two audio tones inside an ordinary FM transmission -- 1200 Hz +for a mark and 2200 Hz for a space, twelve hundred of them a second, which is +Bell 202 and is what a telephone modem sounded like in 1976. It has stayed +because it works through any FM radio ever made, and because every handheld in +a rucksack is already an AFSK transmitter with a microphone socket. + +Four things happen between the aerial and a frame, and each is a place to get +it wrong. + +**The tones become a soft symbol.** Two correlators, one at each tone, and +the difference between them. A correlator rather than a frequency +discriminator because the tones are less than an octave apart and radio audio +is distorted enough that instantaneous frequency wanders badly; asking which +of the two tones a bit-length window contains more of is a question that +survives a weak signal. + +**The soft symbol becomes bits.** Sampled once a bit, at an instant kept in +the middle of the bit by a phase-locked loop that is nudged at every zero +crossing. The loop is what makes this a receiver rather than a decoder of +recordings: it carries its phase from one block of audio to the next, so a +frame that straddles the boundary is read straight through. + +**The bits become a frame.** NRZI first -- the data is in whether the tone +changed, not which tone it is, which makes the whole thing immune to being +wired up backwards. Then HDLC: frames are delimited by the flag 01111110 and +a zero is stuffed after every five ones so the flag cannot occur inside one. + +**The frame is believed or it is not.** Sixteen bits of CRC, and nothing +without a correct one is reported. That is what makes it safe to run this +over hours of an open squelch: a frame either checks out or it never existed. +""" + +from __future__ import annotations + +import math +from dataclasses import dataclass, field + +import numpy as np + +__all__ = ["Address", "Frame", "Receiver", "fcs", "frame_from", "frame_bytes", + "hdlc_frames", "stuff", "unstuff", "modulate", "nrzi", "un_nrzi", + "MARK_HZ", "SPACE_HZ", "BAUD", "FLAG", "APRS_HZ", "APRS_CHANNELS", + "UI_CONTROL", "NO_LAYER_3", "MAX_FRAME"] + + +# Bell 202, as every VHF packet station on earth sends it. +MARK_HZ = 1200.0 +SPACE_HZ = 2200.0 +BAUD = 1200.0 +FLAG = "01111110" + +# Where APRS lives. One channel per region by agreement rather than by +# regulation, which is why there is a list of them rather than a number. +APRS_HZ = 144_390_000.0 +APRS_CHANNELS = ( + ("north-america", 144_390_000.0, "United States, Canada, Mexico"), + ("europe", 144_800_000.0, "IARU Region 1, including the UK"), + ("australia", 145_175_000.0, "Australia and New Zealand"), + ("japan", 144_640_000.0, "Japan"), + ("brazil", 145_570_000.0, "Brazil"), + ("thailand", 145_525_000.0, "Thailand"), +) + +# An unnumbered information frame with no layer-3 protocol, which is what +# every APRS packet is and very nearly all this will ever see. +UI_CONTROL = 0x03 +NO_LAYER_3 = 0xF0 + +# The longest thing worth believing: eight two-byte-addressed hops, a control +# and protocol byte, and 256 bytes of information. +MAX_FRAME = 8 * 7 + 2 + 2 + 256 + 2 + + +# --------------------------------------------------------------------------- +# What a frame is made of +# --------------------------------------------------------------------------- + +@dataclass(frozen=True) +class Address: + """One callsign in a frame's path, with everything packed around it. + + An AX.25 address is six characters and four bits, and the other four bits + of the last byte carry the parts that matter here: which station this is + when a callsign is not unique -- the SSID, the number after the dash -- + and whether a digipeater has already repeated the frame. + """ + + call: str = "" + ssid: int = 0 + repeated: bool = False # the H bit: a digipeater has used this hop + reserved: int = 0b11 # the two bits nobody uses + command: bool = False # the C bit, meaningful only on the first two + + def __str__(self) -> str: + out = f"{self.call}-{self.ssid}" if self.ssid else self.call + return out + "*" if self.repeated else out + + @property + def plain(self) -> str: + """Without the asterisk, for looking a station up.""" + return f"{self.call}-{self.ssid}" if self.ssid else self.call + + +def address_from(raw: bytes) -> Address: + """Seven bytes into a callsign. Everything is shifted up by one bit.""" + call = "".join(chr(byte >> 1) for byte in raw[:6]).rstrip() + last = raw[6] + return Address(call=call, ssid=(last >> 1) & 0x0F, + repeated=bool(last & 0x80), + reserved=(last >> 5) & 0x03, + command=bool(last & 0x80)) + + +def address_bytes(address: Address, last: bool = False) -> bytes: + """One callsign back into seven bytes, ready to send.""" + call = (address.call.upper() + " ")[:6] + flags = ((address.ssid & 0x0F) << 1) | (0x60 if address.reserved else 0) + if address.repeated: + flags |= 0x80 + return bytes((ord(c) << 1) & 0xFE for c in call) + bytes([flags | int(last)]) + + +@dataclass +class Frame: + """One AX.25 frame whose frame-check sequence was correct.""" + + destination: Address = field(default_factory=Address) + source: Address = field(default_factory=Address) + path: tuple = () + control: int = UI_CONTROL + pid: int = NO_LAYER_3 + info: bytes = b"" + raw: bytes = b"" + at: float = 0.0 # when it arrived, as a clock time + snr: float = 0.0 # how far above the noise, in dB + + @property + def unnumbered_information(self) -> bool: + """Whether this is the frame type APRS uses, and nothing else does. + + The control byte has its two low bits set on an unnumbered frame, and + the rest of it says which sort. APRS is always UI with no layer 3; + anything else on this channel is somebody running a real AX.25 + connection, which is a fair thing to see and not an APRS packet. + """ + return (self.control & 0xEF) == UI_CONTROL and self.pid == NO_LAYER_3 + + @property + def kind(self) -> str: + """What sort of AX.25 frame this is, in words.""" + if not self.control & 0x01: + return "information" + if (self.control & 0x03) == 0x01: + return {0x00: "receive ready", 0x04: "receive not ready", + 0x08: "reject", 0x0C: "selective reject"}.get( + self.control & 0x0C, "supervisory") + return {0x03: "unnumbered information", 0x2F: "set async balanced", + 0x43: "disconnect", 0x0F: "disconnect mode", + 0x63: "unnumbered ack", 0x87: "frame reject"}.get( + self.control & 0xEF, "unnumbered") + + @property + def heard_through(self) -> tuple: + """The digipeaters that actually repeated this, in order. + + The ones with the H bit set, which is how a station says "I passed + this on". The rest of the path is where it has yet to go. + """ + return tuple(hop for hop in self.path if hop.repeated) + + def route(self) -> str: + """The frame's path the way every APRS tool in the world writes it.""" + parts = [self.source.plain, self.destination.plain] + parts += [str(hop) for hop in self.path] + return ">".join(parts[:2]) + ("," + ",".join(parts[2:]) + if len(parts) > 2 else "") + + def text(self) -> str: + """The information field as characters, for reading and for parsing.""" + return self.info.decode("latin-1") + + def describe(self) -> str: + body = self.text() + return f"{self.route()}:{body}" if body else self.route() + + +# --------------------------------------------------------------------------- +# The check that makes any of this safe to run +# --------------------------------------------------------------------------- + +def fcs(data: bytes) -> int: + """The AX.25 frame check: CRC-16/X.25, reflected, inverted at the end.""" + crc = 0xFFFF + for byte in data: + crc ^= byte + for _ in range(8): + crc = (crc >> 1) ^ 0x8408 if crc & 1 else crc >> 1 + return crc ^ 0xFFFF + + +def frame_from(raw: bytes) -> Frame | None: + """One frame out of its bytes, or None if it is not one. + + The check is run first and nothing else is looked at until it passes. + Every field below is read on the strength of sixteen bits of CRC saying + the bytes are what was sent. + """ + if len(raw) < 7 * 2 + 2 + 2: + return None + body, check = raw[:-2], raw[-1] << 8 | raw[-2] + if fcs(body) != check: + return None + + addresses, at = [], 0 + while at + 7 <= len(body) and len(addresses) < 10: + field_ = body[at:at + 7] + addresses.append(address_from(field_)) + at += 7 + if field_[6] & 0x01: # the end-of-address bit + break + else: + return None + if len(addresses) < 2 or at + 2 > len(body): + return None + if not all(_sane_call(a.call) for a in addresses): + return None + return Frame(destination=addresses[0], source=addresses[1], + path=tuple(addresses[2:]), control=body[at], + pid=body[at + 1], info=body[at + 2:], raw=raw) + + +def _sane_call(call: str) -> bool: + """Whether a callsign is one, rather than seven bits of luck. + + Sixteen bits of CRC is a strong check and this is a crowded band; a frame + whose addresses are unprintable is one that passed the check by accident, + and there is no reason to put it on a display. + """ + return bool(call) and all(c.isalnum() or c == "-" for c in call) \ + and call.isascii() and call.upper() == call + + +def frame_bytes(source, destination, info: str | bytes = b"", + path=(), control: int = UI_CONTROL, + pid: int = NO_LAYER_3) -> bytes: + """A complete frame, check included, ready to be keyed out. + + Kept beside the decoder so the two cannot drift apart, and so a test can + put a packet in and take the same one out. Callsigns may be given as + strings -- "W1AW-5", "WIDE2-1*" -- or as addresses. + """ + hops = tuple(_as_address(hop) for hop in path) + out = address_bytes(_as_address(destination)) + out += address_bytes(_as_address(source), last=not hops) + for i, hop in enumerate(hops): + out += address_bytes(hop, last=(i == len(hops) - 1)) + out += bytes([control & 0xFF, pid & 0xFF]) + out += info.encode("latin-1") if isinstance(info, str) else bytes(info) + return out + bytes([fcs(out) & 0xFF, (fcs(out) >> 8) & 0xFF]) + + +def _as_address(value) -> Address: + if isinstance(value, Address): + return value + text = str(value).strip().upper() + repeated = text.endswith("*") + text = text.rstrip("*") + call, _, ssid = text.partition("-") + return Address(call=call, ssid=int(ssid) if ssid.isdigit() else 0, + repeated=repeated) + + +# --------------------------------------------------------------------------- +# HDLC: where one frame ends and the next begins +# --------------------------------------------------------------------------- + +def stuff(bits: str) -> str: + """Insert a zero after every five ones, so no flag can occur inside.""" + out, ones = [], 0 + for bit in bits: + out.append(bit) + if bit == "1": + ones += 1 + if ones == 5: + out.append("0") + ones = 0 + else: + ones = 0 + return "".join(out) + + +def unstuff(bits: str) -> str: + """Take those zeroes back out again.""" + out, ones = [], 0 + for bit in bits: + if ones == 5: + ones = 0 + if bit == "0": + continue # the stuffed bit + out.append(bit) + ones = ones + 1 if bit == "1" else 0 + return "".join(out) + + +def nrzi(bits: str, level: str = "1") -> str: + """Encode: a zero is sent as a change of tone, a one as no change.""" + out = [] + for bit in bits: + if bit == "0": + level = "0" if level == "1" else "1" + out.append(level) + return "".join(out) + + +def un_nrzi(bits: str) -> str: + """Decode the same, which needs no knowledge of which tone is which. + + A one is no change and a zero is a change, so inverting the whole stream + -- swapping mark for space, or wiring a discriminator up backwards -- + decodes to exactly the same data. That is the point of the coding and is + why nothing here ever has to guess at polarity. + """ + return "".join("1" if a == b else "0" for a, b in zip(bits, bits[1:])) + + +def hdlc_frames(bits: str, most: int = 64) -> tuple[list[bytes], int]: + """Split a bit stream at flags and undo the stuffing. + + Returns the frames and how far along the stream was consumed, so a caller + reading a continuous signal knows what it may forget. + """ + frames: list[bytes] = [] + at = bits.find(FLAG) + if at < 0: + return frames, max(0, len(bits) - len(FLAG)) + used = at + while len(frames) < most: + while bits.startswith(FLAG, at): # flags repeat between frames + at += 8 + used = at - 8 # the last flag fully passed + end = bits.find(FLAG, at) + if end < 0: + break # a body still arriving; keep it + body, at = bits[at:end], end + # Everything before this flag has been dealt with. Said here rather + # than at the top of the loop because a caller reading a continuous + # signal trims its buffer by this, and trimming to before a frame + # that has already been reported hands it back again on the next + # block -- every packet counted twice, for ever. + used = end + if len(body) < 8 * 17: # shorter than an empty frame + continue + clean = unstuff(body) + whole = len(clean) - len(clean) % 8 + if not 17 <= whole // 8 <= MAX_FRAME: + continue + # Least significant bit first on the air, which is the one thing + # about AX.25 that catches everybody out. + frames.append(bytes(int(clean[i:i + 8][::-1], 2) + for i in range(0, whole, 8))) + return frames, max(used, 0) + + +# --------------------------------------------------------------------------- +# From audio to frames, without ever stopping +# --------------------------------------------------------------------------- + +# How hard the sampling instant is pulled towards the middle of a bit at each +# zero crossing. Low enough that noise cannot drag it about, high enough to +# pull in within a flag or two of the start of a transmission -- which is +# what the flags at the front of every frame are there to allow. +LOOP_GAIN = 0.15 + +# How much of a bit stream to carry over when no flag has been seen. A frame +# is at most three hundred bytes, so anything older than that has no frame +# in it that has not already been found. +KEEP_BITS = MAX_FRAME * 8 * 2 + + +class Receiver: + """Audio in, frames out, across as many blocks as you care to feed it. + + The state that has to survive a block boundary is the whole point of this + being a class: the tail of the audio, so the correlators see no edge; the + phase of the sampling loop, so a bit is not lost or gained where one + block meets the next; the level the tone was last at, for the NRZI; and + the bits themselves, so a frame that began in one block and ended in + another is read straight through rather than halved. + + A packet takes most of a second at twelve hundred baud and blocks are + about that long, so frames straddling a boundary are not an edge case -- + they are most of them. + """ + + def __init__(self, rate: float, baud: float = BAUD, + mark: float = MARK_HZ, space: float = SPACE_HZ): + self.rate = float(rate) + self.baud = float(baud) + self.window = max(4, int(round(self.rate / self.baud))) + self.frames = 0 + self.bytes_seen = 0 + turn = 2.0 * math.pi * np.arange(self.window) / self.rate + self._mark = (np.cos(mark * turn), np.sin(mark * turn)) + self._space = (np.cos(space * turn), np.sin(space * turn)) + self._tail = np.zeros(self.window - 1, dtype=np.float64) + self._phase = 0.0 + self._was = 0.0 # the last soft sample, for crossings + self._level = "1" # the tone the line was last at + self._bits = "" + + # -- the three stages ------------------------------------------------ + def soft(self, audio: np.ndarray) -> np.ndarray: + """How much more mark than space each moment of audio holds. + + Two correlators a bit long, and the difference of their magnitudes. + Positive is a mark. The tail of the previous block is prepended so + that the first bit of this one is measured against real audio rather + than against the zeroes a convolution would otherwise invent. + """ + x = np.concatenate((self._tail, np.asarray(audio, dtype=np.float64))) + if x.size < self.window: + self._tail = x + return np.zeros(0) + self._tail = x[-(self.window - 1):] if self.window > 1 else x[:0] + x = x - x.mean() + out = [] + for cosine, sine in (self._mark, self._space): + i = np.convolve(x, cosine[::-1], mode="valid") + q = np.convolve(x, sine[::-1], mode="valid") + out.append(np.hypot(i, q)) + return out[0] - out[1] + + def slice(self, soft: np.ndarray) -> str: + """Sample the soft signal once a bit, in the middle of the bit. + + The instant is held there by a loop nudged at every zero crossing: + a crossing is a bit boundary, so it should fall half a bit away from + where the last sample was taken, and any difference is an error to + be taken out gently. + """ + step = self.baud / self.rate + phase, was = self._phase, self._was + out = [] + for value in soft: + phase += step + if (value > 0.0) != (was > 0.0): + error = phase - 0.5 if phase < 1.0 else phase - 1.5 + phase -= error * LOOP_GAIN + if phase >= 1.0: + phase -= 1.0 + out.append("1" if value > 0.0 else "0") + was = value + self._phase, self._was = phase, was + return "".join(out) + + def feed(self, audio, when: float = 0.0, snr: float = 0.0) -> list[Frame]: + """One block of audio. Returns whatever frames finished in it.""" + soft = self.soft(audio) + if soft.size == 0: + return [] + tones = self._level + self.slice(soft) + self._level = tones[-1] + self._bits += un_nrzi(tones) + raw, used = hdlc_frames(self._bits) + self._bits = self._bits[used:][-KEEP_BITS:] + + out = [] + for data in raw: + self.bytes_seen += len(data) + frame = frame_from(data) + if frame is not None: + frame.at, frame.snr = when, snr + self.frames += 1 + out.append(frame) + return out + + def reset(self) -> None: + self._tail = np.zeros(self.window - 1, dtype=np.float64) + self._phase, self._was, self._level, self._bits = 0.0, 0.0, "1", "" + + +# --------------------------------------------------------------------------- +# Keying the same thing out, so the receiver can be held to it +# --------------------------------------------------------------------------- + +def bits_of(frame: bytes, flags: int = 8) -> str: + """One frame as the bits that go on the air: flags, stuffing and all.""" + body = "".join(f"{byte:08b}"[::-1] for byte in frame) + return FLAG * flags + stuff(body) + FLAG * flags + + +def modulate(frames, rate: float = 22_050.0, baud: float = BAUD, + mark: float = MARK_HZ, space: float = SPACE_HZ, + flags: int = 8, amplitude: float = 0.5, noise: float = 0.0, + seed: int = 0, quiet_ms: float = 40.0) -> np.ndarray: + """What a receiver would hear: the audio, not the radio signal. + + The tone is continuous in phase across a bit boundary, because a real + modem's is -- it is one oscillator being retuned, not two being switched + -- and a decoder that only ever saw phase jumps at every bit would be + tested against something nothing transmits. + """ + if isinstance(frames, (bytes, bytearray)): + frames = [frames] + stream = "".join(nrzi(bits_of(bytes(f), flags)) for f in frames) + quiet = int(round(rate * quiet_ms / 1000.0)) + per_bit = rate / baud + total = quiet * 2 + int(round(len(stream) * per_bit)) + out = np.zeros(total, dtype=np.float64) + phase, at = 0.0, float(quiet) + for bit in stream: + tone = mark if bit == "1" else space + n = int(round(at + per_bit)) - int(round(at)) + step = 2.0 * math.pi * tone / rate + out[int(round(at)):int(round(at)) + n] = amplitude * np.sin( + phase + step * np.arange(n)) + phase = (phase + step * n) % (2.0 * math.pi) + at += per_bit + if noise: + out = out + np.random.default_rng(seed).normal(0.0, noise, out.size) + return out.astype(np.float32) diff --git a/bandsaunter/basemap.py b/bandsaunter/basemap.py index ede263e..03ff08f 100644 --- a/bandsaunter/basemap.py +++ b/bandsaunter/basemap.py @@ -39,7 +39,9 @@ from . import __version__ __all__ = ["decode_png", "tile_of", "choose_zoom", "fetch_tile", "mosaic", "ground_under", "TILE_URL", "ATTRIBUTION", "MAX_TILES", "MAX_ZOOM", "cache_dir", "PNGError", "airports_in", "AIRPORTS_URL", - "tile_span"] + "tile_span", "on_disk", "ground_cache_dir", "cache_size", + "prune_cache", "forget_ground", "GROUND_CACHE_VERSION", + "GROUND_CACHE_DAYS", "CACHE_LIMIT_MB"] # The standard OpenStreetMap tiles. Any {z}/{x}/{y} server can be put here # instead; nothing below knows anything about this one in particular. @@ -64,13 +66,27 @@ OVERSAMPLE = 1.4 MIN_ZOOM = 2 TILE_PIXELS = 256 +# What the tile server is told it is talking to. This is not decoration: the +# usage policy of the service the default tiles come from asks for a User-Agent +# that identifies the application and gives somewhere to look it up, so that an +# operator with a question about the traffic has somebody to ask. It pointed +# at a topic listing until the project had an address of its own. USER_AGENT = (f"bandsaunter/{__version__} " - "(+https://github.com/topics/rtl-sdr; aircraft map drawing)") + f"(+https://frostwarning.com/git/dustcouncil/bandsaunter; offline-cached map tiles for a signal scanner)") # Politeness between requests to a volunteer-funded service. Only paid on a # tile that was not already on the disk. FETCH_PAUSE = 0.12 +# What to do about tiles that did not arrive. A server that has just been +# asked for a hundred tiles in a row refuses some of them, and the answer to +# being told to slow down is to slow down and ask again rather than to draw +# the gap. Longer than FETCH_PAUSE because the first pass is what provoked +# it; once, because a tile that fails twice is usually a tile that is not +# there, and the caller asks again later anyway. +RETRY_PAUSE = 0.8 +RETRIES = 1 + # Where to ask what aerodromes are in a piece of the world. The same OSM # data the tiles are drawn from, asked as a question rather than a picture. @@ -89,6 +105,20 @@ AIRPORT_CACHE_VERSION = 2 # list of airstrips. MOST_AIRPORTS = 40 +# The finished map, kept as well as the tiles it was built from. The tiles +# are cached already and always have been, so a second evening on the same +# view costs nothing on the network -- but it still costs decoding forty +# PNGs and resampling a megapixel and a half into the picture's own +# projection, every time the window opens, for an answer that cannot have +# changed. A coastline does not move. +GROUND_CACHE_VERSION = 1 +GROUND_CACHE_DAYS = 30 + +# What the whole cache is allowed to grow to. Tiles are small and a map is +# not, so a cache that only ever grew would quietly become the largest thing +# this program had ever written. +CACHE_LIMIT_MB = 400 + class PNGError(ValueError): """A PNG this decoder cannot read.""" @@ -303,12 +333,55 @@ def fetch_tile(zoom: int, x: int, y: int, url: str = TILE_URL, return body +def on_disk(zoom: int, x: int, y: int, cache: Path | None = None, + **kw) -> bool: + """Whether this tile has been fetched before. + + A read off the disk owes a volunteer-funded server no politeness, and + paying it anyway is how a view whose tiles are all in hand takes half a + minute to redraw: two hundred and twenty tiles at an eighth of a second + each, spent sleeping between files that were already there. + """ + where = (cache if cache is not None else cache_dir()) / str(zoom) / str(x) + try: + return (where / f"{y}.png").exists() + except OSError: + return False + + +def _one_tile(fetch, zoom: int, tx: int, ty: int, **kw): + """One decoded tile, or None if it could not be had or made sense of.""" + body = fetch(zoom, tx, ty, **kw) + if body is None: + return None + try: + tile = decode_png(body) + except (PNGError, zlib.error, ValueError): + return None + if tile.shape[0] != TILE_PIXELS or tile.shape[1] != TILE_PIXELS: + return None + return tile + + def mosaic(south: float, west: float, north: float, east: float, zoom: int, - fetch=fetch_tile, pause: float = FETCH_PAUSE, **kw): + fetch=fetch_tile, pause: float = FETCH_PAUSE, + retries: int = RETRIES, retry_pause: float = RETRY_PAUSE, + cached=on_disk, **kw): """Every tile the box touches, stitched into one image. - Returns the pixels and where their top-left corner sits in the world, in - tile-grid pixels at this zoom, so the resampling below can place them. + Returns the pixels, where their top-left corner sits in the world in + tile-grid pixels at this zoom so the resampling below can place them, + and which tiles actually arrived. + + That last one matters because the canvas starts black, and black is not + a neutral colour here: the brightness is inverted further down, so a + tile that never arrived is drawn as the brightest thing on the map + rather than as a gap. Saying which squares are real lets the map be + drawn without them, and lets the caller know the answer is not final. + + Tiles that did not arrive are asked for again before the map is given up + on, because the usual reason for a hole is a hundred tiles having been + asked for in the preceding second. """ x0, y0 = tile_of(north, west, zoom) x1, y1 = tile_of(south, east, zoom) @@ -317,32 +390,40 @@ def mosaic(south: float, west: float, north: float, east: float, zoom: int, span = 2 ** zoom wide, tall = right - left + 1, bottom - top + 1 if wide <= 0 or tall <= 0 or wide * tall > MAX_TILES * 4: - return None, 0, 0 + return None, 0, 0, np.zeros((0, 0), dtype=bool) canvas = np.zeros((tall * TILE_PIXELS, wide * TILE_PIXELS, 3), dtype=np.uint8) - got = 0 - for row in range(tall): - for column in range(wide): - tx, ty = (left + column) % span, top + row - if not 0 <= ty < span: - continue - body = fetch(zoom, tx, ty, **kw) - if body is None: - continue - try: - tile = decode_png(body) - except (PNGError, zlib.error, ValueError): - continue - if tile.shape[0] != TILE_PIXELS or tile.shape[1] != TILE_PIXELS: + covered = np.zeros((tall, wide), dtype=bool) + # Rows off the top or bottom of the world are left out rather than + # asked for: they are not missing tiles, they are places there is no + # map of, and they stay uncovered so they are drawn as nothing. + todo = [(row, column, (left + column) % span, top + row) + for row in range(tall) for column in range(wide) + if 0 <= top + row < span] + for attempt in range(max(0, retries) + 1): + if not todo: + break + if attempt and retry_pause: + time.sleep(retry_pause) + missed = [] + was_here = {(row, column): bool(pause) and cached(zoom, tx, ty, **kw) + for row, column, tx, ty in todo} + for row, column, tx, ty in todo: + tile = _one_tile(fetch, zoom, tx, ty, **kw) + if tile is None: + missed.append((row, column, tx, ty)) continue canvas[row * TILE_PIXELS:(row + 1) * TILE_PIXELS, column * TILE_PIXELS:(column + 1) * TILE_PIXELS] = tile - got += 1 - if pause: + covered[row, column] = True + # Asked before the fetch rather than after it, because the + # fetch is what puts the tile on the disk. + if pause and not was_here[(row, column)]: time.sleep(pause) - if not got: - return None, 0, 0 - return canvas, left * TILE_PIXELS, top * TILE_PIXELS + todo = missed + if not covered.any(): + return None, 0, 0, covered + return canvas, left * TILE_PIXELS, top * TILE_PIXELS, covered # --------------------------------------------------------------------------- @@ -372,6 +453,150 @@ def _resample(values: np.ndarray, edges: np.ndarray, axis: int) -> np.ndarray: return (total / counts.reshape(shape)).astype(np.float32) +def ground_cache_dir() -> Path: + root = os.environ.get("XDG_CACHE_HOME") or "~/.cache" + return Path(root).expanduser() / "bandsaunter" / "ground" + + +def _ground_key(south, west, north, east, width, height, shades, + zoom) -> str: + """What makes two requests for the ground the same request. + + The box is rounded to three places -- about a hundred metres, which is + finer than a tile and far finer than anything visible -- so that a view + which has drifted by a pixel is answered from the cache rather than + rebuilt. The size is in, because the same box rendered into a bigger + window is a different picture and reusing it would be the blur this + program went to some trouble to avoid. + """ + parts = (f"{round(south, 3):+09.3f}", f"{round(west, 3):+09.3f}", + f"{round(north, 3):+09.3f}", f"{round(east, 3):+09.3f}", + f"{int(width)}x{int(height)}", f"s{int(shades)}", + f"z{'auto' if zoom is None else int(zoom)}") + return "_".join(parts) + + +def _read_ground(path: Path): + """A finished map off the disk, or None if there is not a usable one.""" + try: + with np.load(path) as body: + if int(body["version"]) != GROUND_CACHE_VERSION: + return None + if time.time() - float(body["at"]) > GROUND_CACHE_DAYS * 86_400: + return None + return body["levels"] + except Exception: + # Deliberately everything. A truncated or half-written file is what + # a cache looks like after a power cut, and numpy reports that + # through whichever of half a dozen exceptions the zip reader + # happened to raise. None of them is an error here: they are all a + # cache miss, and the answer to a cache miss is to build the map. + return None + + +def _write_ground(path: Path, levels: np.ndarray) -> None: + try: + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_suffix(".tmp.npz") + np.savez_compressed(tmp, levels=levels, at=time.time(), + version=GROUND_CACHE_VERSION) + tmp.replace(path) + except (OSError, ValueError): + return # an unwritable cache is not a reason to stop + # And keep the whole thing inside its limit. Done here because this is + # the only place that makes the cache bigger by a megabyte at a time -- + # a tile is small and a map is not -- and because a cache nothing ever + # prunes is a disk filling up slowly enough that nobody notices. + try: + prune_cache() + except Exception: + pass + + +def forget_ground() -> int: + """Throw away every finished map. The tiles under them are kept.""" + gone = 0 + try: + for path in ground_cache_dir().glob("*.npz"): + try: + path.unlink() + gone += 1 + except OSError: + pass + except OSError: + pass + return gone + + +def cache_size() -> dict: + """How much of the disk each cache is using, and how many files. + + Worth being able to answer, because everything here grows and nothing + here ever shrank until now. + """ + root = Path(os.environ.get("XDG_CACHE_HOME") or "~/.cache") + root = root.expanduser() / "bandsaunter" + out = {} + for name, pattern in (("tiles", "tiles/**/*.png"), + ("ground", "ground/*.npz"), + ("airports", "airports/*.json")): + total = count = 0 + try: + for path in root.glob(pattern): + try: + total += path.stat().st_size + count += 1 + except OSError: + pass + except OSError: + pass + out[name] = {"bytes": total, "files": count} + out["total"] = {"bytes": sum(v["bytes"] for v in out.values()), + "files": sum(v["files"] for v in out.values())} + return out + + +def prune_cache(limit_mb: float = CACHE_LIMIT_MB) -> int: + """Delete the least recently used files until the cache fits. + + Least recently *used* rather than oldest: a tile fetched a year ago and + looked at last night is the receiver's own neighbourhood, and throwing + that away to keep last week's holiday is the wrong way round. Reading + a file updates its access time on any filesystem not mounted noatime, + and where it does not this falls back to when it was written, which is + the same thing for a cache that is only ever written once. + """ + root = Path(os.environ.get("XDG_CACHE_HOME") or "~/.cache") + root = root.expanduser() / "bandsaunter" + files = [] + for pattern in ("tiles/**/*.png", "ground/*.npz", "airports/*.json"): + try: + for path in root.glob(pattern): + try: + stat = path.stat() + files.append((max(stat.st_atime, stat.st_mtime), + stat.st_size, path)) + except OSError: + pass + except OSError: + pass + total = sum(size for _when, size, _p in files) + limit = limit_mb * 1024 * 1024 + if total <= limit: + return 0 + gone = 0 + for _when, size, path in sorted(files): + if total <= limit: + break + try: + path.unlink() + total -= size + gone += 1 + except OSError: + pass + return gone + + def _airport_cache(box) -> Path: root = os.environ.get("XDG_CACHE_HOME") or "~/.cache" name = "_".join(f"{round(v, 1):+06.1f}" for v in box) @@ -483,7 +708,7 @@ def _ask_overpass(box, url: str, timeout: float) -> dict: def ground_under(south: float, west: float, north: float, east: float, width: int, height: int, shades: int = 32, fetch=fetch_tile, zoom: int | None = None, - **kw) -> np.ndarray | None: + remember: bool = True, **kw): """The map for one picture, as ``shades`` levels of brightness. The tiles are Web Mercator and the picture is not, so every output pixel @@ -494,17 +719,39 @@ def ground_under(south: float, west: float, north: float, east: float, point of putting a coastline under it is that the coastline is where the aircraft was. - Returns None when nothing could be fetched, which the caller draws as the - plain grid it drew before. + A whole map is kept on the disk beside the tiles it was built from, so + that opening the same window tomorrow costs a file read rather than + forty PNG decodes and a resample into this program's own projection. + ``remember=False`` turns that off, which is what the tests want when + they are measuring the building rather than the remembering. + + Returns the levels and whether that is the whole answer. Nothing at all + is None, which the caller draws as the plain grid it drew before; a map + with squares missing from it is returned all the same, because most of a + map is better than none, but it is flagged as not settled so the caller + knows to ask again rather than keeping it for the life of the view. """ if width < 1 or height < 1 or north <= south or east <= west: - return None + return None, True + + # The finished map, if this exact picture has been built before. Only + # a whole one is ever written, so anything found here is settled. + key = _ground_key(south, west, north, east, width, height, shades, zoom) + kept = ground_cache_dir() / f"{key}.npz" if remember else None + if kept is not None: + have = _read_ground(kept) + if have is not None and have.shape == (height, width): + return have, True + if zoom is None: zoom = choose_zoom(south, west, north, east, width=width) - tiles, origin_x, origin_y = mosaic(south, west, north, east, zoom, - fetch=fetch, **kw) + tiles, origin_x, origin_y, covered = mosaic(south, west, north, east, + zoom, fetch=fetch, **kw) if tiles is None: - return None + # Nothing arrived. Settled on purpose: a machine with no network + # must not be made to ask for the same tiles five times a second + # for the rest of the night. + return None, True # The edges of each output pixel rather than its middle, so that what # lands in it can be averaged. Taking the nearest source pixel instead @@ -528,11 +775,39 @@ def ground_under(south: float, west: float, north: float, east: float, # brightest thing on the picture. whole = (0.299 * tiles[:, :, 0] + 0.587 * tiles[:, :, 1] + 0.114 * tiles[:, :, 2]).astype(np.float32) - luma = _resample(_resample(whole, ys, axis=0), xs, axis=1) - low, high = float(luma.min()), float(luma.max()) + + # Averaged over the tiles that are really there rather than over the + # canvas. Both sums come off the same box filter, so dividing one by + # the other is the weighted mean over the real pixels -- which gets the + # cells along the edge of a hole right as well, those being part tile + # and part nothing. + real = np.repeat(np.repeat(covered.astype(np.float32), TILE_PIXELS, + axis=0), TILE_PIXELS, axis=1) + share = _resample(_resample(real, ys, axis=0), xs, axis=1) + luma = _resample(_resample(whole * real, ys, axis=0), xs, axis=1) + here = share > 1e-6 + luma = np.where(here, luma / np.where(here, share, 1.0), 0.0) + + if not here.any(): + return None, True + # The darkest and brightest of what was actually fetched. A hole left + # in the reckoning would put the floor at black, which is darker than + # any real tile: the whole map would be drawn dimmer to make room for + # a square that is not there. + low, high = float(luma[here].min()), float(luma[here].max()) if high - low < 1.0: levels = np.zeros_like(luma) else: levels = 1.0 - (luma - low) / (high - low) - return np.clip((levels * (shades - 1)).round(), 0, - shades - 1).astype(np.uint8) + # And the hole itself is drawn as bare ground rather than as the + # brightest thing on the picture, which is what inverting black gives. + levels = np.where(here, levels, 0.0) + out = np.clip((levels * (shades - 1)).round(), 0, + shades - 1).astype(np.uint8) + settled = bool(covered.all()) + # Only a whole map is kept. Writing one with squares missing would + # cache the hole for a month, and the whole point of treating a partial + # map as provisional is that it gets asked for again. + if kept is not None and settled: + _write_ground(kept, out) + return out, settled diff --git a/bandsaunter/browse.py b/bandsaunter/browse.py index 0399de3..89b7b53 100644 --- a/bandsaunter/browse.py +++ b/bandsaunter/browse.py @@ -38,6 +38,7 @@ from rich.table import Table from rich.text import Text from . import version_notice +from . import __version__ from .bandplan import band_names, fmt_hz, shorten_band from .callsign import CallsignBook, HEADING, find_callsigns from .config import DEFAULT_CONFIG_DIR, load_default, remember_lockouts @@ -666,6 +667,13 @@ STAMP = "%y-%m-%d %I:%M:%S %p" STAMP_WIDTH = 20 +def _reg_when(value: float) -> str: + """When something was looked up, or nothing if it never was.""" + if not value: + return "" + return datetime.fromtimestamp(value).strftime("%Y-%m-%d %H:%M") + + def _stamp(when: datetime | None) -> str: """One capture's date and time, or an empty string when it has neither.""" if when is None: @@ -738,6 +746,17 @@ class Browser: self.show_help = False self.reading = False # full-screen transcript self.read_top = 0 + # The register: everything ever looked up, on its own screen. Read + # the first time it is opened rather than at startup, because a + # browser pointed at a directory of recordings should not wait on + # two caches somebody may never ask to see. + self.showing_register = False + self.register: dict | None = None + self.reg_kind = "callsigns" + self.reg_index = 0 + self.reg_top = 0 + self.reg_query = "" + self.reg_searching = False self._undo: list[tuple[Path, Path]] = [] # the last move, to reverse self._undo_path: Path | None = None # where its .wav came from self._undo_label = "" @@ -1393,10 +1412,10 @@ class Browser: ("space", "stop"), ("/", "search"), ("t", "read"), ("S I N", "file"), ("d", "delete"), ("m", "mask"), ("s", f"sort by {self.sort}"), ("r", "reload"), - ("?", "keys"), ("q", "quit")] + ("c", "register"), ("?", "keys"), ("q", "quit")] # Everything given up here is still on the page ? opens, which is why # ? is among the last to go. - expendable = ["r", "s", "space", "m", "d", "S I N", "t", "/", + expendable = ["r", "s", "space", "m", "d", "S I N", "c", "t", "/", "⏎", "↑↓", "?"] width = self.console.size.width or 80 shown = items @@ -1458,8 +1477,10 @@ class Browser: names += f" or {FILING[-1][1]}/" for key, what in ( ("↑ ↓ / k j", "move through the recordings"), - ("PgUp PgDn", "a screenful at a time"), - ("Home End", "first and last"), + # Merged, because the page is exactly as tall as an + # eighty-by-twenty-four terminal and the register needed a + # row. These two were always the same thought. + ("PgUp PgDn", "a screenful at a time; Home End for the ends"), ("Enter", "play the highlighted recording"), ("space", "stop playing"), ("t", "read the whole transcript, full screen"), @@ -1475,6 +1496,12 @@ class Browser: ("m", "lock this frequency out of every later scan"), ("", ""), ("callsigns", "found in transcripts and looked up for you"), + # One line, not five. Eighty by twenty-four is still what a + # new terminal is, and a list of keys that has to be + # scrolled to reach "q" has failed at its one job -- so the + # register's own keys are named on the register's own screen + # rather than here. + ("c", "every callsign and aircraft ever looked up"), ("? h", "this list"), ("q", "quit")): t.add_row(key, what) @@ -1544,6 +1571,8 @@ class Browser: return Layout(Align.center(self._help(), vertical="middle")) if self.reading: return Layout(self._reader()) + if self.showing_register: + return self._register() layout = Layout() # One height, computed once: the panel and the region it sits in have # to agree, or the difference shows up as a gap in the middle of the @@ -1573,6 +1602,8 @@ class Browser: return True if self.reading: return self._handle_reader(key) + if self.showing_register: + return self._handle_register(key) rows = self._rows() if key in ("q", "escape") and not self.query: @@ -1613,6 +1644,8 @@ class Browser: self.query = "" elif key in ("?", "h"): self.show_help = not self.show_help + elif key == "c": + self._open_register() elif key == "t": cap = self.current # Either kind of content: the decoded panel says "press t to read @@ -1663,6 +1696,240 @@ class Browser: self._delete() return True + # -- the register ----------------------------------------------------- + def _open_register(self) -> None: + from . import register as reg + + if self.register is None: + self.register = reg.load_all() + self.showing_register = True + self.reg_index = self.reg_top = 0 + counts = ", ".join(f"{len(v)} {k}" + for k, v in self.register.items()) + self.message = counts or "nothing looked up yet" + + # The detail panel's share of the screen: two borders and eight rows of + # label-and-value in two columns, which holds the sixteen fields a + # licence has. Fixed rather than fitted, so the list above it does not + # jump about as the cursor moves between a full record and a bare one. + _REG_DETAIL = 11 + + def _reg_rows(self) -> int: + """How many list rows the screen has room for beside the detail.""" + height = self.console.size.height or 24 + # 3 for the heading panel, the detail panel, 1 for the footer, and + # 1 for the table's own column headings. + return max(3, height - 3 - self._REG_DETAIL - 1 - 1) + + def _reg_all(self) -> list: + from . import register as reg + + if self.register is None: + return [] + return reg.search(self.register.get(self.reg_kind) or [], + self.reg_query) + + @property + def reg_current(self): + found = self._reg_all() + if not found: + return None + return found[min(self.reg_index, len(found) - 1)] + + def _register(self) -> Group: + """The register: a list, and everything known about the current one.""" + from . import register as reg + + found = self._reg_all() + rows = self._reg_rows() + if found: + self.reg_index = max(0, min(self.reg_index, len(found) - 1)) + self.reg_top = max(0, min(self.reg_top, max(0, len(found) - rows))) + if self.reg_index < self.reg_top: + self.reg_top = self.reg_index + elif self.reg_index >= self.reg_top + rows: + self.reg_top = self.reg_index - rows + 1 + + other = "aircraft" if self.reg_kind == "callsigns" else "callsigns" + held = len(self.register.get(self.reg_kind) or []) if self.register \ + else 0 + mappable = sum(1 for e in found if e.mappable) + head = Text.from_markup( + f"[bold]{self.reg_kind}[/bold] — " + f"[cyan]{len(found)}[/cyan] of {held} shown, " + f"{mappable} with somewhere to point a map at" + + (f" [yellow]matching {escape(self.reg_query)!r}[/yellow]" + if self.reg_query else "")) + + table = Table(box=None, header_style="bold", pad_edge=False, + expand=True) + table.add_column(" ", width=1, style="cyan") + table.add_column("callsign" if self.reg_kind == "callsigns" + else "aircraft", width=12, no_wrap=True) + table.add_column("who" if self.reg_kind == "callsigns" else "what", + ratio=3, no_wrap=True, style="white") + table.add_column("where", ratio=2, no_wrap=True, style="grey62") + table.add_column("looked up", width=16, no_wrap=True, + style="bright_black") + table.add_column("map", width=3, justify="center") + for i in range(self.reg_top, min(len(found), self.reg_top + rows)): + entry = found[i] + here = i == self.reg_index + table.add_row( + "›" if here else "", + Text(entry.title, style="bold reverse" if here else "bold"), + entry.what or Text("—", style="grey37"), + entry.where or "", + _reg_when(entry.when), + Text("◉", style="green") if entry.mappable + else Text("·", style="grey37")) + if not found: + table.add_row("", "", Text("nothing here", style="yellow"), + "", "", "") + + # A split rather than a group, so the list takes whatever is left + # and the footer stays on the bottom line. Grouped, a short list + # left the footer floating in the middle of the screen with the + # tail of the previous frame under it. + layout = Layout() + layout.split_column( + Layout(Panel(head, border_style="cyan", padding=(0, 1), + title="[bold]the register[/bold]", + title_align="left", + subtitle=f"[bright_black]tab → {other}" + f"[/bright_black]", + subtitle_align="right"), size=3), + Layout(table, ratio=1), + Layout(self._reg_detail(self.reg_current), + size=self._REG_DETAIL), + Layout(Text.from_markup(self._reg_footer()), size=1), + ) + return layout + + def _reg_detail(self, entry) -> Panel: + """Everything known about one entry, in two columns of label and + value. + + Two columns rather than one long list because a licence has a dozen + fields and a screen is wider than it is tall, and because the thing + somebody is looking for is usually further down than the fold. + """ + if entry is None: + return Panel(Text("select something to see what is known " + "about it", style="grey62"), + border_style="grey37", padding=(0, 1)) + grid = Table.grid(padding=(0, 2), expand=True) + grid.add_column(justify="right", style="grey62", width=16) + grid.add_column(ratio=1, overflow="fold") + grid.add_column(justify="right", style="grey62", width=16) + grid.add_column(ratio=1, overflow="fold") + pairs = list(entry.rows) + half = (len(pairs) + 1) // 2 + left, right = pairs[:half], pairs[half:] + for i in range(half): + a_label, a_value = left[i] + b_label, b_value = right[i] if i < len(right) else ("", "") + grid.add_row(a_label, Text(a_value), b_label, Text(b_value)) + where = (f"[green]g[/green] opens this place in a browser" + if entry.mappable + else "[grey37]no position, so nowhere to open[/grey37]") + return Panel(grid, border_style="cyan", padding=(0, 1), + title=f"[bold]{escape(entry.title)}[/bold]", + title_align="left", subtitle=where, + subtitle_align="right") + + def _reg_footer(self) -> str: + if self.reg_searching: + return (f"[cyan]search[/cyan] {escape(self.reg_query)}[reverse] " + f"[/reverse] [bright_black]⏎ done esc clear" + f"[/bright_black]") + if self.message: + return f"[green]{escape(self.message)}[/green]" + return ("[bright_black]↑↓ move tab switch / search " + "g map c or esc back q quit[/bright_black]") + + def _handle_register(self, key: str) -> bool: + if self.reg_searching: + return self._handle_reg_search(key) + rows = self._reg_rows() + if key in ("c", "escape"): + self.showing_register = False + self.reg_query = "" + elif key == "q": + return False + elif key in ("up", "k"): + self.reg_index -= 1 + elif key in ("down", "j"): + self.reg_index += 1 + elif key in ("pgup", "left"): + self.reg_index -= rows + elif key in ("pgdn", "right"): + self.reg_index += rows + elif key == "home": + self.reg_index = 0 + elif key == "end": + self.reg_index = 1 << 20 # clamped when the panel is drawn + elif key == "tab": + self.reg_kind = ("aircraft" if self.reg_kind == "callsigns" + else "callsigns") + self.reg_index = self.reg_top = 0 + elif key == "/": + self.reg_searching = True + self.reg_query = "" + elif key == "g": + self._open_map() + elif key == "r": + from . import register as reg + + self.register = reg.load_all() + self.message = "read the caches again" + self.reg_index = max(0, self.reg_index) + return True + + def _handle_reg_search(self, key: str) -> bool: + if key == "enter": + self.reg_searching = False + elif key == "escape": + self.reg_searching = False + self.reg_query = "" + elif key == "backspace": + self.reg_query = self.reg_query[:-1] + elif len(key) == 1 and key.isprintable(): + self.reg_query += key + self.reg_index = self.reg_top = 0 + return True + + def _open_map(self) -> None: + """Show where this is, in whatever browser the desktop uses. + + Only the coordinates go into the URL. A map does not need to be + told whose licence it is looking at, and the one part of this that + leaves the machine should carry as little as it can. + """ + entry = self.reg_current + if entry is None: + return + url = entry.maps() + if not url: + self.message = "nothing is known about where this one is" + return + import contextlib + import io + import webbrowser + + try: + # Quietly: some browsers write to the terminal on startup, and + # this one is running a full-screen display that would be left + # with somebody else's text through the middle of it. + with contextlib.redirect_stdout(io.StringIO()), \ + contextlib.redirect_stderr(io.StringIO()): + opened = webbrowser.open(url) + except Exception as exc: + self.message = f"could not open a browser: {exc}" + return + self.message = (f"opened {entry.title} in a browser" if opened + else f"no browser to open — the place is {url}") + def _handle_reader(self, key: str) -> bool: rows = max(3, (self.console.size.height or 24) - 6) if key in ("t", "escape", "q", "space"): @@ -1774,6 +2041,8 @@ def build_parser() -> argparse.ArgumentParser: default=SORTS[0], metavar="ORDER", help="initial order: %s (default: %s, newest first)" % (", ".join(SORTS), SORTS[0])) + p.add_argument("--no-splash", action="store_true", + help="do not draw the title screen") p.add_argument("--filter", default="", metavar="TEXT", help="start with only recordings matching this") p.add_argument("--player", default=None, metavar="CMD", @@ -1870,6 +2139,18 @@ def _write_kml(console: Console, browser: "Browser", book: CallsignBook, def main(argv: list[str] | None = None) -> int: args = build_parser().parse_args(argv) console = Console() + + # Paused for a moment, because what follows takes the whole screen: the + # browser runs on the alternate screen, so without a beat here the + # banner is replaced in the same tenth of a second it was drawn in. It + # is still on the normal screen afterwards, and comes back on quitting. + from . import splash + + splash.show(console, splash.BROWSER_NAME, + ("Conceived by: The Dust Council", + "100% AI Coded by Claude Code.", + f"{__version__} · read back what a scan left behind"), + no_splash=getattr(args, "no_splash", False), pause=0.9) directory = (Path(args.directory).expanduser() if args.directory else default_directory()) if not directory.is_dir(): diff --git a/bandsaunter/callsign.py b/bandsaunter/callsign.py index cd6dd04..36bfce6 100644 --- a/bandsaunter/callsign.py +++ b/bandsaunter/callsign.py @@ -35,7 +35,7 @@ from pathlib import Path __all__ = ["Callsign", "CallsignBook", "find_callsigns", "describe_prefix", "grid_to_latlon", "PHONETIC", "LOOKUP_URL", "BACKUP_URL", - "is_service_call", "SHAPE", "SERVICE_SHAPE"] + "is_service_call", "licensed_call", "SHAPE", "SERVICE_SHAPE"] # The FCC's own licence data, served as JSON without an account or a key. # US callsigns only; everything else resolves to what the prefix alone says. @@ -130,6 +130,55 @@ SERVICE_SHAPE = re.compile( SERVICE_LICENCE = "GMRS or business licence" +def licensed_call(call: str) -> str: + """The callsign a register could know, out of one that was heard. + + Every mode dresses a callsign up in its own way and no licensing + authority has heard of any of it: + + W1AW-9 APRS: the ninth radio this licence runs + WIDE2-1* APRS: the star marks the hop a packet came through + ET3RFG/R FT8: a rover, operating away from the licensed address + W1AW/4 operating in another district + DL/G4ABC a guest in Germany -- G4ABC is the licence + <...> FT8: a callsign this receiver has not heard spelled out + + Returns the bare callsign, or "" where there is nothing to look up. + Shared rather than written twice because getting it wrong is silent: + asking a register about "W1AW-9" returns nothing at all, which looks + exactly like a station that is not licensed. + """ + call = (call or "").strip().upper().rstrip("*") + if not call or call.startswith("<"): + return "" + if call in ("CQ", "DE", "QRZ", "BEACON", "ALL", "MAIL", "TEST"): + return "" + if call.startswith("CQ "): + return "" + call = call.split("-", 1)[0] # the SSID is not licensed + if "/" in call: + # A guest call is prefix/home or home/suffix, and the licensed part + # is whichever piece is a callsign in its own right. Longest wins, + # because a suffix is one or two characters and a prefix is a + # country: neither is ever longer than the callsign itself. + pieces = [p for p in call.split("/") if p] + real = [p for p in pieces if _looks_licensed(p)] + if not real: + return "" + call = max(real, key=len) + return call if _looks_licensed(call) else "" + + +def _looks_licensed(call: str) -> bool: + """Whether something could be an amateur callsign at all. + + A letter or two, a digit, and a letter or three. Deliberately loose -- + the register is the authority on whether a callsign exists, and this + only has to keep obvious non-callsigns from being asked about. + """ + return bool(re.fullmatch(r"[A-Z0-9]{1,3}[0-9][A-Z]{1,4}", call or "")) + + def is_service_call(call: str) -> bool: """True for a GMRS, business or public-safety callsign, not a ham one. diff --git a/bandsaunter/cli.py b/bandsaunter/cli.py index 9db58f0..6f75015 100755 --- a/bandsaunter/cli.py +++ b/bandsaunter/cli.py @@ -18,6 +18,7 @@ from rich.table import Table from rich.text import Text from . import version_notice +from . import __version__ from .bandplan import CATEGORIES, PRESETS, fmt_hz, in_category, search from .config import (DEFAULT_CONFIG_DIR, DEFAULT_CONFIG_PATH, ScanConfig, is_first_run, list_profiles, load_config, load_default, @@ -25,6 +26,8 @@ from .config import (DEFAULT_CONFIG_DIR, DEFAULT_CONFIG_PATH, ScanConfig, from .device import RtlSdrError, list_devices, set_driver_messages from .librtlsdr import load_error from . import settings as st +from .ax25 import APRS_CHANNELS as _APRS_CHANNELS +from .ft8 import BANDS as _FT8_BANDS from .ranges import (RangeError, ScanRange, build_plan, parse_range_list) from .scanner import Scanner, ScannerCallbacks from .tui import TUIAbort, first_run_setup, run_tui, settings_menu @@ -58,6 +61,14 @@ examples: bandsaunter adsb --window live map of the aircraft bandsaunter flights --out sky.gif animate what they did bandsaunter flights --theme phosphor draw it as a vector display + bandsaunter weather read the 433 MHz weather sensors + bandsaunter weather --name 1A2B="back fence" name one while listening + bandsaunter readings --csv turn a weather log into a graph + bandsaunter sensors what is out there, and what it is called + bandsaunter aprs read the APRS channel on 144.39 MHz + bandsaunter aprs --region europe ...or 144.80 MHz, or wherever you are + bandsaunter aprs --window a live map of the stations heard + bandsaunter packets --kml turn an APRS log into a map bandsaunter scan -b 2m --simulate try it without hardware """) # The GNU form: the version, then who holds the copyright and what the @@ -65,6 +76,11 @@ examples: # are allowed to do with it and the program is the only thing in front # of them. p.add_argument("--version", action="version", version=version_notice()) + # Before the subcommand, because it is about the program rather than + # about any one thing it does. BANDSAUNTER_NO_SPLASH=1 does the same for + # anybody who wants it off for good. + p.add_argument("--no-splash", action="store_true", + help="do not draw the title screen") sub = p.add_subparsers(dest="command") # -- scan ------------------------------------------------------------ @@ -260,6 +276,25 @@ examples: ad.add_argument("--no-labels", dest="labels", action="store_false", default=None, help="draw the aircraft without labels beside them") + ad.add_argument("--pulse", dest="pulse", action="store_true", + default=None, help="swell each aircraft from bright to " + "dim and back") + ad.add_argument("--no-pulse", dest="pulse", action="store_false", + default=None, help="draw the aircraft at a steady " + "brightness") + ad.add_argument("--pulse-rate", type=float, default=None, + metavar="SECONDS", + help="how long one swell takes, in seconds of watching") + ad.add_argument("--echo", dest="echo", action="store_true", + default=None, help="rings travelling outward from each " + "aircraft") + ad.add_argument("--no-echo", dest="echo", action="store_false", + default=None, help="no rings") + ad.add_argument("--echo-every", type=float, default=None, + metavar="SECONDS", + help="how long between one ring and the next") + ad.add_argument("--echo-size", type=int, default=None, metavar="PIXELS", + help="how far a ring gets before it has faded away") ad.add_argument("--theme", default=None, metavar="NAME", choices=("night", "digital", "phosphor", "amber", "red", "blue", "green", "orange", "wargames", "norad", @@ -269,6 +304,231 @@ examples: "phosphor, amber and red") ad.set_defaults(log_frames=True, lookup=True) + # -- aprs ----------------------------------------------------------------- + ap = sub.add_parser("aprs", + help="read the APRS channel: positions, weather, " + "messages, telemetry") + ap.add_argument("--seconds", type=float, default=None, + help="stop after this long (default: until interrupted)") + ap.add_argument("--rate", type=float, default=None, + help="sample rate in Hz; 96 kS/s is the least that holds " + "the channel") + ap.add_argument("--gain", default=None, help="tuner gain in dB, or auto") + ap.add_argument("--device", type=int, default=None, help="which receiver") + ap.add_argument("--region", default=None, + choices=[key for key, _hz, _w in _APRS_CHANNELS], + help="which APRS channel to listen on") + ap.add_argument("--frequency", "--freq", dest="frequency", type=float, + default=None, metavar="HZ", + help="the exact frequency, if the region's channel is " + "not what you want") + ap.add_argument("--lookup", dest="lookup", action="store_true", + default=None, + help="look up who each callsign is licensed to, and " + "cache the answers (the default)") + ap.add_argument("--no-lookup", dest="lookup", action="store_false", + help="do not look callsigns up over the network") + ap.add_argument("--window", action="store_true", + help="open a window and show the stations on a map as " + "they are heard, instead of a table in the terminal") + ap.add_argument("--radius", type=float, default=None, metavar="KM", + help="how far around the aerial the window reaches") + ap.add_argument("--theme", default=None, metavar="NAME", + choices=("night", "digital", "phosphor", "amber", "red"), + help="how the window looks") + ap.add_argument("--map-brightness", type=int, default=None, + metavar="PERCENT", + help="how bright the ground under the stations is (10-100)") + ap.add_argument("--basemap", dest="basemap", action="store_true", + default=None, help="draw a real map under the stations") + ap.add_argument("--no-basemap", dest="basemap", action="store_false", + default=None, help="no map tiles") + ap.add_argument("--window-rings", dest="window_rings", + action="store_true", default=None, + help="faint range discs around the aerial") + ap.add_argument("--no-window-rings", dest="window_rings", + action="store_false", default=None, help="no range rings") + ap.add_argument("--box-opacity", type=int, default=None, + metavar="PERCENT", + help="how solid the card behind each information box is") + ap.add_argument("--trails", dest="trails", action="store_true", + default=None, help="draw the path behind what moved") + ap.add_argument("--no-trails", dest="trails", action="store_false", + default=None, help="no trails") + ap.add_argument("--tiles", dest="tile_url", default=None, metavar="URL", + help="where map tiles come from ({z}/{x}/{y}.png)") + ap.add_argument("--find-channel", nargs="?", type=float, const=20.0, + default=None, metavar="SECONDS", + help="listen on each region's channel in turn and say " + "which has traffic, instead of listening on one " + "(default: 20 seconds each)") + ap.add_argument("--simulate", dest="simulate", action="store_true", + default=None, + help="invent a channel full of stations, for a receiver " + "with no aerial") + ap.add_argument("--no-simulate", dest="simulate", action="store_false", + default=None, help="listen to real stations") + ap.add_argument("--log", default=None, metavar="FILE", + help="where to write the packet log " + "(default: aprs_