diff --git a/INSTALL.md b/INSTALL.md index baa9e40..eef7b3f 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -9,8 +9,10 @@ If you are on Debian, Ubuntu or Mint, [build the package](#a-debian-ubuntu-mint- [a virtual environment](#b-anywhere-else-a-virtual-environment). **You can try the whole program before buying or plugging in a receiver.** -`--simulate` runs the scanner against a synthetic band, and `bandsaunter adsb ---simulate` flies imaginary aircraft past an imaginary receiver. Neither needs +`--simulate` runs the scanner against a synthetic band, `bandsaunter adsb +--simulate` flies imaginary aircraft past an imaginary receiver, and +`bandsaunter weather --simulate` puts six weather sensors on a fence that does +not exist. None of the three needs hardware or a network. --- @@ -116,6 +118,8 @@ bandsaunter bands # the built-in US band plan; no hardware need bandsaunter scan -b 2m --simulate # a whole scan against a synthetic band bandsaunter adsb --simulate --seconds 30 bandsaunter flights # draws what the last command heard +bandsaunter weather --simulate --seconds 60 +bandsaunter readings --csv # turns what it heard into a spreadsheet ``` With a receiver plugged in: @@ -205,6 +209,13 @@ package. `python3-pyqt6` on Debian and Fedora, `python-pyqt6` on Arch, or `pip install PyQt6` in the environment bandsaunter runs from. Without it the window is the only thing missing, and the program says so rather than failing. +**The weather sensors need nothing at all** beyond what is already in the +table above — no network, no extra package, no key. They are a few milliwatts +on 433.92 MHz and the only thing that decides whether they are heard is the +aerial: a quarter-wave whip is 17 cm, which the stock telescopic aerial does +if it is collapsed to about that. `bandsaunter weather --simulate` runs the +whole thing without one. + **Aircraft and callsign lookups need no installation**, only a network. They ask public registers about a callsign or a 24-bit address and cache the answers for a month; `--no-lookup` turns them off, and what the address and diff --git a/README.md b/README.md index 9ae28df..0e759ef 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,11 @@ built-in US band plan — and it sweeps them, stops on anything above the noise floor, records it, and works out what kind of signal it was. CW/Morse is decoded to text. +Two things get sections of their own, because neither fits through a scan: +[the aircraft overhead](#aircraft) on 1090 MHz, drawn on a moving map, and +[the weather sensors](#weather-sensors-on-433-mhz) on 433 MHz, which you can +give names to as they arrive. + Two programs: `bandsaunter` scans, and [`saunterbrowse`](#browsing-what-you-recorded) reads back what it collected — transcripts, identifications and playback, in one screen. @@ -240,6 +245,8 @@ bandsaunter # the menus: set up and scan bandsaunter scan -b 2m -b marine-vhf # band-plan presets bandsaunter scan -r 144M-148M -r 420M-450M # your own ranges bandsaunter scan -b 2m --simulate # try it without hardware +bandsaunter adsb --window # aircraft overhead, on a map +bandsaunter weather # the weather sensors on 433 MHz ``` ## Two ways to drive it @@ -253,11 +260,18 @@ table of settings, so neither can offer something the other cannot. 2 Band plan 107 US presets 3 Settings record no limit, hang 6s, squelch +12 dB, keep voice, cw 4 Saved settings and profiles + 5 Aircraft (ADS-B) listen on 1090 MHz, draw where they went + 6 Weather sensors listen on 433 MHz, name what is out there h Help s Start scanning q Quit ``` +Items 5 and 6 are sections of their own, with their own options and their own +menus, because neither fits through a scan: ADS-B is a megabit a second and a +scan channel is 12.5 kHz wide, and a weather sensor message is a burst of a +carrier switched on and off that a scan would record as clicks. + Settings are grouped, show their current value against the built-in default, and carry their own help: @@ -1784,30 +1798,299 @@ form of every one. copy between machines. API keys are deliberately **not** kept there — see [Knowing the leg for certain](#knowing-the-leg-for-certain). -## Meters and weather sensors +## Weather sensors on 433 MHz -Two things on the ISM bands are worth naming rather than reporting as hex. +A consumer weather station is two things. The display on the kitchen wall is +one of them; the other is a plastic box on a fence post that says what it can +see every sixteen seconds, in the clear, on 433.92 MHz, to anyone who happens +to be listening. `bandsaunter weather` reads the box. + +``` +bandsaunter weather listen, and name things as they arrive +bandsaunter weather --simulate a garden of sensors that are not there +bandsaunter readings --csv turn a log into something plottable +bandsaunter sensors what is out there, and what it is called +``` + +It is a section of its own, like the aircraft one, and for the same reason: +it does not fit through the scanner. A sensor message is a burst of a carrier +switched on and off, a fifth of a second long, and the scan path is a squelch +and a recorder — it would record the bursts as clicks in a WAV file and decode +nothing. + +``` +╭──────────────────────────────────────────────────────────────────────────────────────╮ +│ 433.92 MHz 6 sensors 148 messages 41:02 weather_2026-09-07_11_31_04.jsonl │ +│ n to name (2 unnamed) control-C to stop │ +╰──────────────────────────────────────────────────────────────────────────────────────╯ + # name id model readings batt msgs ago + 1 back fence 1A2B Tower 592TXR temperature 8.4 C humidity 88% ok 41 0:07 + 2 mast 0777 5-in-1 06014RM wind 14.2 km/h wind from 248° WSW ok 36 0:11 + rain 41.4 mm temperature 8.1 C + humidity 90% + 3 greenhouse 0C41 Tower 592TXR temperature 15.9 C humidity 61% ok 39 0:04 + 4 unnamed 0311 Lightning 6045M temperature 8.6 C humidity 87% ok 24 0:19 + strikes 4 storm 19 km + 5 shed 5C 609TXC temperature 3.1 C humidity 94% low 21 0:12 + 6 unnamed 93 606TX temperature -18.2 C ok 19 0:22 +``` + +### Naming things while they arrive + +A sensor broadcasts an identity, and that identity is a number that came out +of a hat in a factory — or a different number out of the same hat the next +time the batteries were changed. It is enough to tell one sensor from another +and it is no use at all for telling which is which. `1A2B` is not a place. + +So **press `n` while listening**. The display comes down, the sensors are +listed with numbers, you pick one and type a name, and it goes back up. The +receiver keeps running throughout: a slow typist loses a few seconds of +weather and nothing else. + +That is the moment it is possible to do. The sensor is on the screen saying +3.1 degrees, and the person watching is the one who knows that the cold one is +the shed. An hour later it is a list of hexadecimal again. + +Names can also be given from the command line, before or after anything has +been heard: + +``` +bandsaunter weather --name 1A2B="back fence" --name 5C=shed +bandsaunter sensors --name 0311="lightning detector" --note 0311="south gable" +bandsaunter sensors --forget 93 +``` + +A name given before the sensor has ever been received waits under its +identity alone — nothing yet knows which model it is, because that is in the +message and there has not been one — and moves across the moment the first +message arrives. So naming the garden from a list and then listening works, +and the display says `shed` from the first reception rather than listing the +shed and the sensor on it as two separate things. + +They live in `sensors.yaml` beside the settings, one entry per sensor, meant +to be opened and edited by hand — it is a list of things in a garden, and +typing them is often quicker than tagging them one at a time off the air. +Names are written the moment they are given rather than when the program +exits, because it exits on control-C; and they are written to a neighbouring +file which is then renamed over the old one, so a machine losing power halfway +through leaves either the old names or the new ones and never half of each. + +Nothing is ever dropped for being stale. A sensor whose battery ran out two +winters ago keeps its name, because the alternative is that putting a battery +back in loses it. + +### What it reads + +Five families, each with its own framing and its own check: + +| model | bytes | what it says | +|---|---|---| +| Tower 592TXR / 06002RM | 7 | temperature, humidity | +| 5-in-1 06014RM / VN1TXC | 8 | wind speed, wind direction, rainfall / wind speed, temperature, humidity | +| Lightning 6045M | 9 | temperature, humidity, strike count, how far off the storm is | +| 609TXC | 5 | temperature, humidity | +| 606TX | 4 | temperature | + +Battery state comes from all of them. The 5-in-1 has more to say than fits in +one message and alternates two of them, so its last message is always either +the wind and the rain or the temperature and the humidity, never both — the +display keeps the newest value of each quantity rather than the newest +message, so the line is the whole sensor rather than half of it flickering. + +**What is not read.** The Atlas, the 986 and 515 fridge thermometers, the +00275rm room monitor and the 899 standalone rain gauge. They are on the same +band and are not decoded. A message from one of them whose framing happens to +match is reported as an unknown message type with its identity and nothing +else, rather than guessed at — because knowing that something is out there +transmitting, with an identity that stays the same, is worth a line on its +own. `--no-unknown` leaves them off. + +These formats are implemented from their published descriptions and are +checked against frames built from the same descriptions. That proves the +framing, the parity, the checksums and the arithmetic; it is not the same as +having held every one of these sensors. + +### Why nothing false gets through + +433 MHz is a crowded band. Doorbells, car keys, tyre-pressure sensors, garage +doors and the neighbours' weather station are all on it, and a decoder that +looks at every bit offset of every burst will find a message in noise if it is +allowed to. Four things stop it. + +**The checks the message carries.** The three newer models have an eight-bit +sum plus odd parity in the top bit of every payload byte — twelve to fourteen +bits of check on a message of seven to nine bytes. + +**Arriving twice.** The two older models carry one byte of check between them, +which is one false message in two hundred and fifty-six, and that is not a +rate a display can be trusted at. So those two are only believed when the same +message arrives twice. It costs nothing: these sensors send everything three +times in a row, for exactly this reason. + +**A plausibility range.** A checksum can be satisfied by a message the +hardware could not have sent. Nothing outside −40 to 70 °C, 0 to 100% or a +wind the anemometer cannot physically report is accepted. + +**Where the message sits.** This is the one that is easy to get wrong. A +seven-byte message read out of the front of a real eight-byte one is made of +that message's own payload bytes, whose parity is *already correct* — so the +parity bits contribute nothing at all and what is left is one byte of sum. +Corroboration does not help either, because the copies of a message are +identical, so a coincidence in one copy is a coincidence in all three. What +gives that window away every time is that it ends a whole byte before the +burst does. A real message begins after the sync and runs to the end. + +### Off the air + +Three things happen between the aerial and a bit, in this order. + +The receiver is tuned a little to one side of 433.92 MHz, because every +RTL-SDR puts a spike of its own at whatever it is tuned to, and a spike +sitting on top of a signal that works by being switched on and off is the one +thing that stops it being off. The sensors are shifted back to the middle in +software, which puts the spike out at the edge instead. `--offset 0` tunes +straight at them, which is worth trying once to see what the spike was costing. + +Then a running average of the complex samples, long enough that its first null +lands on the spike. It is a crude filter and a deliberately crude one: what it +has to reject is one tone at a frequency this end chose. + +Only then is the magnitude taken. Filtering before detection rather than after +is what keeps the neighbours — a doorbell, a tyre sensor, a car key — from +adding themselves to the envelope of the sensor. + +Slicing the envelope into bits never measures anything against a clock. The +newer sensors vary the length of the pulse and keep the gaps even; the two +older ones keep the pulse even and vary the gap. Both readings of the same +pulses are tried and the checksums say which it was — rather than deciding +from the timings, which is guessing, and wrong on a weak burst where the edges +have moved. A transmitter running ten per cent fast is read correctly and +never noticed, which matters: these transmitters are unlocked and drift tens +of kilohertz and a few per cent of rate with the temperature. An outdoor +sensor in January is not the one that was on the fence in July. + +### Afterwards + +``` +sensors heard +name id model ch msgs every battery last heard +back fence 1A2B Tower 592TXR A 148 16 s ok 23:14:07 +mast 0777 5-in-1 06014RM A 131 18 s ok 23:14:02 +greenhouse 0C41 Tower 592TXR B 140 17 s ok 23:14:05 +shed 5C 609TXC 76 30 s low 23:13:58 + +what they said +sensor reading first last lowest highest +back fence temperature 8.4 C 6.1 C 5.9 C 9.2 C + humidity 88% 94% 86% 94% +mast wind 14.2 km/h 3.1 km/h 0.0 km/h 38.6 km/h + wind from 248° WSW 202° SSW + rain 41.4 mm 43.9 mm 41.4 mm 43.9 mm +``` + +The two tables are split on purpose. The first is about reception — who, how +often, how well — and is the one to look at when something is missing; `every` +is the honest measure of an aerial, because these transmit on a fixed cycle, +so thirty seconds from a sensor that sends every sixteen means half of them +are being missed. The second is about the weather, and is the one to look at +when nothing is. + +There is no average. These arrive every sixteen seconds when the sensor is in +range and not at all when it is not, and rain and cold both shorten the range +of a 433 MHz transmitter — so the mean of what was received is the mean of a +sample whose gaps are themselves the weather, and it would read like a number +when it is not one. A bearing gets no lowest or highest either: north is 0 and +also 360. + +`--csv`, or `bandsaunter readings --csv` afterwards, writes a column per +quantity and a row per reading, with the sensor's name in the second column +and the unit in the heading rather than beside every number — a column of +"21.5 C" is text, and a column of 21.5 is a temperature. + +The log keeps the raw bytes of every message underneath whatever was made of +them, because the message is the evidence and the rest of the line is an +opinion about it. A later version of this program that reads a model this one +cannot will be able to go back through old logs and read them properly. + +Readings are converted once, on the way in, to degrees Celsius, kilometres an +hour, millimetres and kilometres — different models report in different units, +and the tower sends Celsius while the 5-in-1 and the lightning detector send +Fahrenheit. The log holds the converted value, so `--units imperial` changes +only what is shown and can be changed afterwards on an old log. + +### Every weather option + +| option | flag | default | what it does | +|---|---|---|---| +| Receiver | `--device` | 0 | which receiver, when more than one is plugged in | +| Gain | `--gain` | auto | tuner gain in dB, or automatic | +| Sample rate | `--rate` | 1.024 MS/s | how fast to sample; 250 kS/s is the minimum | +| Sensors on | `--frequency`, `--freq` | 433.92 MHz | where the sensors transmit | +| Tuning offset | `--offset` | 250 kHz | how far to one side of them to tune | +| Invent a garden | `--simulate` / `--no-simulate` | no | six sensors that are not there | +| Listen for | `--seconds` | until stopped | how long before stopping | +| Write a log | `--log` / `--no-log` | yes | one line of JSON per message | +| Print every message | `--messages` / `--no-messages` | no | a stream of lines instead of a table | +| Keep on screen for | `--hold` | 1800 s | how long a sensor stays after its last message | +| Show readings in | `--units` | metric | metric or imperial, for the display and the export | +| Only named sensors | `--only-named` / `--all-sensors` | no | ignore anything without a name | +| Show unreadable models | `--unknown` / `--no-unknown` | yes | list sensors whose model cannot be read | +| Report at the end | `--report` / `--no-report` | yes | print what each sensor said | +| Also write a spreadsheet | `--csv` / `--no-csv` | no | CSV beside the log | + +`--name ID=NAME` is not a setting: it names a sensor and is repeatable. + +Saved in `weather.yaml` beside the other settings, from the menu's **s** or by +hand. `bandsaunter readings` also takes `--sensor NAME` to narrow a log to one +sensor, and `--no-report` to write the spreadsheet and say nothing. + +### Without a sensor + +`--simulate` puts six sensors on a fence that does not exist — one of every +model — transmitting real messages with real checksums, keyed on and off as a +real one does, through the real filter, the real slicer and the real decoders. +Nothing touches the receiver. The naming, the log, the report and the export +can all be tried before an aerial exists. + +The weather moves at roughly a real afternoon's pace, which is slower than it +sounds like it should be: about a degree an hour of drift, wind gusting around +a mean, rain falling a tenth of a millimetre at a time. A garden that swung +six degrees a minute would exercise everything downstream perfectly well and +would look, to anyone running `--simulate` to see what the program does, like +a program that cannot read a thermometer. + +### If nothing is heard + +These are a few milliwatts. A quarter-wave whip for 433.92 MHz is 17 cm of +wire, and the stock telescopic aerial set to about that length works well. +Indoors, behind a wall, with the dongle in the back of a machine, is usually +the problem. Try `--gain 40` if the automatic gain control is not finding +them, and `--messages` to watch individual receptions arrive while moving the +aerial about. + +## Meters on 900 MHz + +A scan of 902–928 MHz that turns up a burst gets it named rather than reported +as hex. ``` 915.0 MHz decoded: electricity meter 12345678 reading 987654 -433.92 MHz decoded: AcuRite sensor 1234 temperature 21.5 C humidity 48% channel A +433.92 MHz decoded: Tower 592TXR 1A2B temperature 21.5 C humidity 48% channel A ``` **Utility meters.** The Itron ERT modules fitted to electricity, gas and water meters across North America broadcast their reading every thirty seconds or so on 902–928 MHz, in the clear, so a van can drive past and read a street. The message says which meter, what kind, what the register reads, and whether the -tamper switches have been tripped. +tamper switches have been tripped. It is not guessed at: the message carries a +16-bit BCH check and nothing is reported that has not satisfied it. -**AcuRite sensors.** The 433.92 MHz outdoor sensors sold with every consumer -weather station send temperature, humidity, battery state and a channel letter -every sixteen seconds. - -Neither is guessed at: a meter message carries a 16-bit BCH check and a sensor -message a checksum and four parity bits, and nothing is reported that has not -satisfied them. Both are implemented from their published descriptions and -checked against frames built from the same descriptions — which proves the -framing and the arithmetic, and is not the same as having held a meter. +**Weather sensors.** A scan of 433 MHz that catches one of these names it the +same way, using the same decoders as +[the weather section](#weather-sensors-on-433-mhz) — which is where they +belong, because a scan catches one burst and the weather is a night of them. +The two thinly-checked models are not reported from a scan at all: they need +the same message twice before they are believed, and a scan hears one. ## Hex into words diff --git a/bandsaunter/__init__.py b/bandsaunter/__init__.py index 6751165..d916c88 100755 --- a/bandsaunter/__init__.py +++ b/bandsaunter/__init__.py @@ -8,8 +8,8 @@ and transcribing speech. # Versions are the release date and a revision within that day, so # 2026-08-21_02 is the second build made on the 21st. The revision is padded # to two digits so versions sort as text. -VERSION_DATE = "2026-09-06" -VERSION_REVISION = 4 +VERSION_DATE = "2026-09-07" +VERSION_REVISION = 1 __version__ = f"{VERSION_DATE}_{VERSION_REVISION:02d}" diff --git a/bandsaunter/acurite.py b/bandsaunter/acurite.py new file mode 100644 index 0000000..ce33a5c --- /dev/null +++ b/bandsaunter/acurite.py @@ -0,0 +1,1313 @@ +"""AcuRite weather sensors: the messages they send, and how to get them off +the air. + +A consumer weather station is two things. The display on the kitchen wall is +one of them; the other is a plastic box on a fence post that says what it can +see every sixteen seconds or so, in the clear, on 433.92 MHz, to anyone who +happens to be listening. This reads the box. + +**What is here.** Five families, each with its own framing and its own check: + +=========================== ===== ================================== +model bytes what it says +=========================== ===== ================================== +Tower 592TXR / 06002RM 7 temperature, humidity +5-in-1 06014RM / VN1TXC 8 wind, direction, rain / wind, temperature, humidity +Lightning 6045M 9 temperature, humidity, strikes, how far off +609TXC 5 temperature, humidity +606TX 4 temperature +=========================== ===== ================================== + +The first three share a framing -- two bytes of identity, a byte saying what +kind of message this is, the payload, and a checksum which is the sum of +everything before it -- and the payload bytes each carry odd parity in their +top bit. That is between twelve and fourteen bits of check on a message of +seven to nine bytes, which is enough to accept a message on a band where +doorbells, car keys, tyre sensors and someone's garage all transmit. + +The last two are older and cheaper and carry one byte of check between them: +a sum on the 609 and a CRC-8 on the 606. Eight bits is one false message in +two hundred and fifty-six tries, and this looks at every bit offset of every +burst, so eight bits on its own is not enough. Those two are therefore only +believed when the *same* message arrives twice in one burst -- which costs +nothing, because these sensors send every message three times in a row for +exactly this reason. + +**What is not here.** The Atlas, the 986 and 515 fridge thermometers, the +00275rm room monitor and the 899 standalone rain gauge. They are on the same +band and are not read. A message from one of them whose framing happens to +match is reported as an unknown message type with its identity and nothing +else, rather than guessed at. + +That last part is not a nicety. The Atlas is nine bytes like the lightning +detector and lays its payload out differently, and nothing but the message +type tells them apart -- so every decoder here insists on a type it knows, +and a nine-byte message that is not a 6045M is reported as a sensor of an +unknown model rather than as a lightning detector reading a temperature off +the wrong bits. Wrong weather under somebody's sensor name is a worse +answer than no weather at all. + +**Where the numbers come from.** These formats are implemented from their +published descriptions and are checked here against frames built from the +same descriptions. That proves the framing, the parity, the checksums and +the arithmetic; it is not the same as having held every one of these +sensors. The reason it is nonetheless safe to run is that nothing is +reported which has not satisfied its own check. + +**Units.** Different models report in different units -- the tower sends +tenths of a degree Celsius, the 5-in-1 and the lightning detector send tenths +of a degree Fahrenheit, and the rain gauge sends hundredths of an inch. All +of it is converted here, once, to Celsius, kilometres an hour, millimetres +and kilometres, so that a log holds one kind of number and the person reading +it can be shown whichever they prefer. The raw counts survive alongside the +converted values where they mean something on their own, which for the rain +gauge they do: it is a tipping bucket and the count only ever goes up. +""" + +from __future__ import annotations + +import math +from dataclasses import dataclass, field + +import numpy as np + +__all__ = ["Reading", "Measure", "format_measure", + "decode", "decode_burst", "candidates", "confirmed", + "readings_from", "MODELS", "MODEL_NAMES", "CHANNELS", "ACURITE_HZ", + "tower_frame", "five_in_one_wind_rain", "five_in_one_weather", + "lightning_frame", "frame_609", "frame_606", + "bursts", "Burst", "bits_pwm", "bits_ppm", "pulse_train", + "modulate", "baseband", "WIND_POINTS", "compass", "parity8", + "crc8", "VirtualSensor", "SimulatedSensors", "default_sensors"] + + +# Where every one of these sensors transmits. Nominally 433.92 MHz; the +# cheap transmitters in them drift by tens of kilohertz with temperature, +# which is why what follows works on the envelope of a band rather than on a +# carrier at an exact frequency. +ACURITE_HZ = 433_920_000.0 + + +# --------------------------------------------------------------------------- +# The small arithmetic every one of these depends on +# --------------------------------------------------------------------------- + +def parity8(value: int) -> int: + """Odd-parity bit of one byte: 1 when an odd number of bits are set.""" + value ^= value >> 4 + value ^= value >> 2 + value ^= value >> 1 + return value & 1 + + +def crc8(data, poly: int = 0x07, init: int = 0x00) -> int: + """The plain byte-at-a-time CRC-8 the 606TX carries.""" + crc = init + for byte in data: + crc ^= byte + for _ in range(8): + crc = ((crc << 1) ^ poly) & 0xFF if crc & 0x80 else (crc << 1) & 0xFF + return crc + + +# The channel switch on the back of a tower sensor has three positions, and +# they are not encoded in the order anyone would guess: A is 3, B is 2 and C +# is 0. Code 1 is not a switch position at all; the published descriptions +# call it E and so does this, rather than pretending it is a D. +CHANNELS = ("C", "E", "B", "A") + +# The sixteen wind directions, in the order the sensor numbers them, which is +# also not the order anyone would guess. Index it with the low nibble. +WIND_POINTS = (315.0, 247.5, 292.5, 270.0, 337.5, 225.0, 0.0, 202.5, + 67.5, 135.0, 90.0, 112.5, 45.0, 157.5, 22.5, 180.0) + +_COMPASS = ("N", "NNE", "NE", "ENE", "E", "ESE", "SE", "SSE", + "S", "SSW", "SW", "WSW", "W", "WNW", "NW", "NNW") + + +def compass(degrees: float) -> str: + """The sixteen-point name for a bearing, for people rather than plots.""" + return _COMPASS[int((float(degrees) % 360.0) / 22.5 + 0.5) % 16] + + +# One tip of the bucket, in millimetres. The gauge counts hundredths of an +# inch, which is 0.254 mm, and the count is cumulative for the life of the +# battery. +RAIN_PER_COUNT_MM = 0.254 + +# What a sensor can physically report. A checksum can be satisfied by a +# message the hardware could not have sent, and on a band this crowded that +# happens often enough to be worth catching. +TEMPERATURE_RANGE = (-40.0, 70.0) # Celsius +HUMIDITY_RANGE = (0, 100) # per cent +WIND_RANGE = (0.0, 260.0) # km/h; the sensor tops out below this + +# The message type a 6045M sends. Insisted on rather than assumed from the +# length; see :func:`decode_lightning` for why. +LIGHTNING_MESSAGE = 0x2F + +# Where in a burst a message is allowed to sit. A real one begins after the +# sync and runs to the end, so a handful of bits of sync in front of it and +# almost nothing behind. See :func:`candidates`, where these do the work. +LEAD_BITS = 8 # room for twice the four sync pulses these sensors send +TAIL_BITS = 4 # room for a couple of stray edges at the end +# A message this cannot read is held to the tighter figures: it has no type +# field that means anything here and no range that can be checked, so where +# it sits is the only evidence there is that it is a message at all. +UNREAD_LEAD_BITS = 4 +UNREAD_TAIL_BITS = 1 + + +def _fahrenheit(f: float) -> float: + return (f - 32.0) * 5.0 / 9.0 + + +# --------------------------------------------------------------------------- +# What one message came to +# --------------------------------------------------------------------------- + +@dataclass(frozen=True) +class Measure: + """One quantity a sensor reported, with the unit it is held in.""" + + name: str + value: float + unit: str = "" + # What the sensor actually sent, where that is a different number from + # the one above: the rain counter, the strike distance in miles. Kept + # because it is the evidence and the converted value is a restatement. + raw: float | None = None + + @property + def text(self) -> str: + return format_measure(self) + + +def format_measure(measure: Measure, imperial: bool = False) -> str: + """One measurement as a person would write it, in either system.""" + name, value, unit = measure.name, measure.value, measure.unit + if unit == "C": + return f"{value * 9 / 5 + 32:.1f} F" if imperial else f"{value:.1f} C" + if unit == "%": + return f"{value:.0f}%" + if unit == "km/h": + return f"{value / 1.609344:.1f} mph" if imperial else f"{value:.1f} km/h" + if unit == "mm": + return f"{value / 25.4:.2f} in" if imperial else f"{value:.1f} mm" + if unit == "km": + return f"{value / 1.609344:.0f} mi" if imperial else f"{value:.0f} km" + if unit == "deg": + return f"{value:.0f}° {compass(value)}" + if unit == "lux": + return f"{value:.0f} lux" + if unit: + return f"{value:g} {unit}" + # A bare count -- strikes, bucket tips -- reads better without a ".0". + return f"{value:g}" if value != int(value) else f"{int(value)}" + + +@dataclass +class Reading: + """One message that satisfied its own checks, and what it said. + + The bits are kept because they are the evidence; everything else in here + is an opinion about them, and the two should not be confused. + """ + + model: str = "" # what to call it to a person + family: str = "" # what to file it under: tower, 5n1, ... + sensor: str = "" # the identity in the message, as hex + channel: str = "" # A, B or C, where the model has a switch + battery_low: bool | None = None + message: int = 0 # the message-type field, as sent + measures: tuple = () + bits: str = "" + checks: tuple = () + at: float = 0.0 # when it arrived, as a clock time + copies: int = 1 # how many times it arrived, identically + offset: int = 0 # where in the burst its first bit was + + @property + def key(self) -> str: + """What a friendly name is attached to. + + Family and identity, not channel: the channel switch is on the + outside of the box and someone will move it, and moving it should not + lose the name they gave the box. + """ + return f"{self.family}/{self.sensor}" + + def value(self, name: str): + """One measurement by name, or None.""" + for measure in self.measures: + if measure.name == name: + return measure.value + return None + + def describe(self, imperial: bool = False) -> str: + head = f"{self.model} {self.sensor}" + if self.channel: + head += f" ch {self.channel}" + rest = " ".join(f"{m.name} {format_measure(m, imperial)}" + for m in self.measures) + if self.battery_low: + rest += " battery low" + if not self.measures: + rest = rest or f"message type {self.message:#04x}, not understood" + return f"{head} {rest}".strip() + + +# Every family, in the order they are tried, with how many copies of a +# message must arrive in one burst before it is believed. See the module +# docstring: it is a function of how much check the message carries. +MODELS = ( + ("tower", "Tower 592TXR", 7, 1), + ("5n1", "5-in-1 06014RM", 8, 1), + ("6045", "Lightning 6045M", 9, 1), + ("609", "609TXC", 5, 2), + ("606", "606TX", 4, 2), +) + +MODEL_NAMES = {family: name for family, name, _bytes, _r in MODELS} +MESSAGE_BYTES = {family: n for family, _name, n, _r in MODELS} +REPEATS_NEEDED = {family: repeats for family, _name, _bytes, repeats in MODELS} + + +# --------------------------------------------------------------------------- +# The three that share a framing: tower, 5-in-1, lightning detector +# --------------------------------------------------------------------------- + +def _txr_framed(data: list[int]) -> bool: + """The checks the whole TXR family carries: a sum, and odd parity. + + The sum covers every byte before it. The parity is in the top bit of + each payload byte -- everything but the two identity bytes and the + checksum itself -- and is odd, so a byte of all zeroes fails it and a + run of silence read as zeroes cannot become a message. + """ + if (sum(data[:-1]) & 0xFF) != data[-1]: + return False + if not any(data): + return False + return all(parity8(byte) == 1 for byte in data[2:-1]) + + +def _txr_identity(data: list[int], id_bits: int) -> tuple[str, str]: + """The channel letter and the sensor identity, as the family encodes it.""" + channel = CHANNELS[(data[0] >> 6) & 0x03] + mask = (1 << (id_bits - 8)) - 1 + return channel, f"{((data[0] & mask) << 8) | data[1]:04X}" + + +def decode_tower(data: list[int]) -> Reading | None: + """The 592TXR / 06002RM outdoor sensor: temperature and humidity. + + Fourteen bits of identity with the channel above them, a status byte + whose bit 6 is set while the battery is good, humidity in seven bits, and + temperature in eleven -- tenths of a degree Celsius offset by a hundred, + so that the coldest thing it can report is still a positive number. + """ + if len(data) != 7 or not _txr_framed(data): + return None + if (data[2] & 0x3F) != 0x04: + return _unknown("tower", data, 14) + + channel, sensor = _txr_identity(data, 14) + humidity = data[3] & 0x7F + raw = ((data[4] & 0x0F) << 7) | (data[5] & 0x7F) + celsius = raw / 10.0 - 100.0 + if not _sane_temperature(celsius) or not _sane_humidity(humidity): + return None + return Reading( + model="Tower 592TXR", family="tower", sensor=sensor, channel=channel, + battery_low=not data[2] & 0x40, message=data[2] & 0x3F, + measures=(Measure("temperature", round(celsius, 1), "C"), + Measure("humidity", float(humidity), "%")), + checks=("checksum-8", "parity")) + + +def decode_5n1(data: list[int]) -> Reading | None: + """The 5-in-1, which says half of what it knows at a time. + + It alternates two messages, because there is more to say than fits in one + of them: a wind message with the direction and the rain counter, and a + weather message with the temperature and the humidity. Both carry the + wind speed, which is the one number worth having on every transmission. + """ + if len(data) != 8 or not _txr_framed(data): + return None + kind = data[2] & 0x3F + channel, sensor = _txr_identity(data, 12) + common = dict(model="5-in-1 06014RM", family="5n1", sensor=sensor, + channel=channel, battery_low=not data[2] & 0x40, + message=kind, checks=("checksum-8", "parity")) + + # The same eight bits in both messages: five from one byte and three + # from the next. Zero means stopped; anything else is a count the + # published description turns into kilometres an hour with a slope and + # an offset, and the offset is why a turning cup never reads zero. + raw_wind = ((data[3] & 0x1F) << 3) | ((data[4] & 0x70) >> 4) + wind = 0.0 if raw_wind == 0 else raw_wind * 0.8278 + 1.0 + if not WIND_RANGE[0] <= wind <= WIND_RANGE[1]: + return None + + if kind == 0x31: # wind, where from, and rain + degrees = WIND_POINTS[data[4] & 0x0F] + counter = ((data[5] & 0x7F) << 7) | (data[6] & 0x7F) + return Reading( + measures=(Measure("wind", round(wind, 1), "km/h"), + Measure("wind from", degrees, "deg"), + Measure("rain", round(counter * RAIN_PER_COUNT_MM, 2), + "mm", raw=float(counter))), + **common) + if kind == 0x38: # wind, temperature and humidity + raw = ((data[4] & 0x0F) << 7) | (data[5] & 0x7F) + celsius = _fahrenheit((raw - 400) / 10.0) + humidity = data[6] & 0x7F + if not _sane_temperature(celsius) or not _sane_humidity(humidity): + return None + return Reading( + measures=(Measure("temperature", round(celsius, 1), "C"), + Measure("humidity", float(humidity), "%"), + Measure("wind", round(wind, 1), "km/h")), + **common) + return _unknown("5n1", data, 12) + + +def decode_lightning(data: list[int]) -> Reading | None: + """The 6045M, which is a tower sensor that also counts lightning. + + The strike counter is cumulative and wraps at 127, and the distance is + the storm's, in miles, as the detector estimates it -- zero to thirty-one, + where thirty-one means "further away than this can tell". There is also + a bit which is set when the detector believes it is being interfered with, + and it is worth showing, because a strike count that climbs while that + bit is set is not lightning. + + The message type is insisted on, and that is the whole reason this can be + run at all. The Atlas is a nine-byte sensor too and lays its payload out + differently, and nothing but the type field tells them apart: decoded as + this, an Atlas would give a temperature read off the wrong bits, which + would pass the checksum, pass the parity, and land inside a plausible + range often enough to be believed. Wrong weather under somebody's sensor + name is a worse answer than none, so anything else nine bytes long comes + back as a sensor of a model this cannot read. + """ + if len(data) != 9 or not _txr_framed(data): + return None + kind = data[2] & 0x3F + if kind != LIGHTNING_MESSAGE: + return _unknown("6045", data, 12) + channel, sensor = _txr_identity(data, 12) + humidity = data[3] & 0x7F + raw = ((data[4] & 0x1F) << 7) | (data[5] & 0x7F) + celsius = _fahrenheit(raw / 10.0 - 100.0) + if not _sane_temperature(celsius) or not _sane_humidity(humidity): + return _unknown("6045", data, 12) + + strikes = data[6] & 0x7F + miles = data[7] & 0x1F + measures = [Measure("temperature", round(celsius, 1), "C"), + Measure("humidity", float(humidity), "%"), + Measure("strikes", float(strikes), "")] + if miles < 0x1F: + measures.append(Measure("storm", round(miles * 1.609344, 1), "km", + raw=float(miles))) + flags = ["checksum-8", "parity"] + if data[7] & 0x20: + flags.append("interference") + return Reading( + model="Lightning 6045M", family="6045", sensor=sensor, channel=channel, + battery_low=not data[2] & 0x40, message=kind, + measures=tuple(measures), checks=tuple(flags)) + + +def _unknown(family: str, data: list[int], id_bits: int) -> Reading: + """A message that framed correctly and is not one this can read. + + Reported rather than dropped, because knowing that there is a sensor of + some kind on the fence, with an identity that stays the same, is worth + something on its own -- it can be given a name and watched -- and because + silently discarding a well-formed message is how a decoder comes to look + like a receiver problem. + """ + channel, sensor = _txr_identity(data, id_bits) + return Reading(model=f"AcuRite {len(data)}-byte sensor", + family=family, sensor=sensor, channel=channel, + battery_low=not data[2] & 0x40, message=data[2] & 0x3F, + measures=(), checks=("checksum-8", "parity")) + + +# --------------------------------------------------------------------------- +# The two older ones +# --------------------------------------------------------------------------- + +def _signed12(high: int, low: int) -> int: + """Twelve bits of two's-complement temperature, in tenths of a degree.""" + raw = ((high & 0x0F) << 8) | low + return raw - 0x1000 if raw & 0x800 else raw + + +def decode_609(data: list[int]) -> Reading | None: + """The 609TXC: one byte of identity, temperature, humidity, a sum. + + The identity is only eight bits and is redrawn at random every time the + battery is changed, so two of these in one garden will collide about once + in every two hundred and fifty battery changes. Nothing can be done + about that from here; it is worth knowing when a sensor appears to have + changed its mind about the weather. + """ + if len(data) != 5 or (sum(data[:4]) & 0xFF) != data[4] or not any(data): + return None + celsius = _signed12(data[1], data[2]) / 10.0 + humidity = data[3] + if not _sane_temperature(celsius) or not _sane_humidity(humidity): + return None + return Reading( + model="609TXC", family="609", sensor=f"{data[0]:02X}", + battery_low=not data[1] & 0x80, + measures=(Measure("temperature", round(celsius, 1), "C"), + Measure("humidity", float(humidity), "%")), + checks=("checksum-8", "twice")) + + +def decode_606(data: list[int]) -> Reading | None: + """The 606TX: identity, temperature, a CRC-8, and nothing else at all.""" + if len(data) != 4 or crc8(data[:3]) != data[3] or not any(data): + return None + celsius = _signed12(data[1], data[2]) / 10.0 + if not _sane_temperature(celsius): + return None + return Reading( + model="606TX", family="606", sensor=f"{data[0]:02X}", + battery_low=not data[1] & 0x80, + measures=(Measure("temperature", round(celsius, 1), "C"),), + checks=("crc-8", "twice")) + + +def _sane_temperature(celsius: float) -> bool: + return TEMPERATURE_RANGE[0] <= celsius <= TEMPERATURE_RANGE[1] + + +def _sane_humidity(humidity: float) -> bool: + return HUMIDITY_RANGE[0] <= humidity <= HUMIDITY_RANGE[1] + + +_DECODERS = ((7, decode_tower), (8, decode_5n1), (9, decode_lightning), + (5, decode_609), (4, decode_606)) + + +# --------------------------------------------------------------------------- +# From bits to readings +# --------------------------------------------------------------------------- + +def _bytes_at(bits: str, at: int, count: int) -> list[int]: + return [int(bits[at + i * 8:at + (i + 1) * 8], 2) for i in range(count)] + + +def candidates(bits: str) -> list[Reading]: + """Every message that can be read out of a run of bits, at any offset. + + Every offset, because what arrives here has a sync pattern of some length + in front of it and the message does not begin on a byte boundary of the + recovered bits. Every length, because a burst does not say which model + sent it. The checks are what make that affordable: a message which + satisfies a sum, a parity set and a plausibility range at a bit offset it + does not belong at is rare enough to be worth the search. + """ + found: list = [] + for count, decoder in _DECODERS: + # A message has to sit where a message sits: a few bits of sync in + # front of it, and the end of the burst immediately behind it. This + # is the check that stops a short model being read out of a long one. + # + # It is needed because neither of the other two defences reaches the + # case. Corroboration does not: the copies of a message are + # identical, so a coincidence in one copy is a coincidence in all + # three, and asking for it three times over asks for nothing. Parity + # does not either, and this is the part that is easy to get wrong -- + # a seven-byte window taken from the front of a real eight-byte + # message is made of that message's own payload bytes, whose parity + # is already correct, so the four parity bits contribute nothing at + # all and what is left is one byte of sum. One in two hundred and + # fifty-six is not a rate at which a display can be trusted. + # + # Where it sits, though, gives that window away every time: it ends + # a whole byte before the burst does. + need = count * 8 + for at in range(0, len(bits) - need + 1): + if at > LEAD_BITS or len(bits) - at - need > TAIL_BITS: + continue + try: + data = _bytes_at(bits, at, count) + except ValueError: + break + reading = decoder(data) + if reading is None: + continue + if not reading.measures and (at > UNREAD_LEAD_BITS + or len(bits) - at - need + > UNREAD_TAIL_BITS): + continue + reading.bits = bits[at:at + need] + reading.offset = at + found.append((at, need, reading)) + return _one_per_window(found, len(bits)) + + +def _one_per_window(found: list, length: int) -> list[Reading]: + """Of two messages sharing bits, keep one: they cannot both be real. + + A message forty bits long, found in a burst forty-one bits long, can be + read at two offsets, and one byte of checksum is satisfied by the wrong + one of them about once in every two hundred and fifty-six bursts. Both + survive corroboration, because both are in all three copies. What tells + them apart is that a real message ends where the burst does, and the + other one leaves a bit hanging off the end. + + Longer messages and better-understood ones are preferred first, so that + a burst which really does hold an eight-byte message is never given up + for a five-byte window inside it. + """ + found.sort(key=lambda item: (len(item[2].measures), item[1], + -(length - item[0] - item[1]), -item[0]), + reverse=True) + kept: list[Reading] = [] + taken: list[tuple[int, int]] = [] + for at, need, reading in found: + if any(at < end and start < at + need for start, end in taken): + continue + taken.append((at, at + need)) + kept.append(reading) + return kept + + +def decode(bits: str, confirm: bool = True) -> Reading | None: + """The one message a run of bits holds, or None. + + Where a model's checks are thin -- the two older ones carry a single + byte between them -- the same message has to turn up more than once + before it is believed, and one run of bits is one copy. So by default + this reads the three well-checked models and not the two thin ones; the + thin ones are reached through :func:`readings_from`, which sees a whole + block and so can see the copies. + + ``confirm=False`` reads whatever framed correctly, for a caller with its + own reason to believe a single message: a stored log, or a test holding + the encoder that made it. + """ + return _best(candidates(bits), confirm) + + +def _best(found: list[Reading], confirm: bool = True) -> Reading | None: + """The most-corroborated message in a pile of candidates. + + Ordered by how many copies arrived, then by how much check the model + carries, then by how long the message is -- a longer message that framed + correctly is better evidence than a shorter one that did, and the + longest ones are also the ones with the most to say. + """ + if not found: + return None + seen: dict[tuple, list[Reading]] = {} + for reading in found: + seen.setdefault((reading.family, reading.bits), []).append(reading) + good = [(copies, group[0]) for (family, _bits), group in seen.items() + if (copies := len(group)) >= (REPEATS_NEEDED[family] if confirm + else 1)] + if not good: + return None + good.sort(key=lambda pair: (pair[0], len(pair[1].bits), + len(pair[1].measures)), reverse=True) + return good[0][1] + + +def decode_burst(burst: "Burst") -> Reading | None: + """One burst of pulses, read as both codings, decoded as whatever it is. + + These sensors do not agree on how a bit is drawn. The newer ones vary + the length of the pulse and keep the gap between pulses roughly even; + the two older ones keep the pulse even and vary the gap. Rather than + decide which from the timings -- which is guessing, and wrong on a weak + burst where the edges have moved -- both readings of the same pulses are + tried, and the checksums say which one it was. + """ + return _best(candidates(bits_pwm(burst)) + candidates(bits_ppm(burst))) + + + + + +# --------------------------------------------------------------------------- +# From the air to bits: an on-off-keyed envelope, sliced +# --------------------------------------------------------------------------- + +@dataclass +class Burst: + """One run of pulses with no long silence in it. + + Marks and spaces alternate and are in microseconds, starting with a mark; + there is always one more mark than there are spaces, because the silence + that ended the burst is not part of it. + """ + + at: float = 0.0 # seconds from the start of the block + marks: tuple = () + spaces: tuple = () + level: float = 0.0 # how far above the noise floor, as a ratio + + @property + def pulses(self) -> int: + return len(self.marks) + + @property + def length_us(self) -> float: + return float(sum(self.marks) + sum(self.spaces)) + + +def baseband(iq: np.ndarray, sample_rate: float, offset: float = 0.0, + want_rate: float = 250_000.0) -> tuple[np.ndarray, float]: + """The envelope of the sensor's channel, at a rate worth slicing. + + Three things happen here, in this order and for this reason. + + The receiver is tuned a little to one side of 433.92 MHz, because every + RTL-SDR puts a spike of its own at whatever it is tuned to and a spike + sitting on top of an on-off-keyed signal is the one thing that stops it + being on and off. So the first step is to shift the sensor back to the + middle, which puts the spike out at the edge instead. + + The second is a plain running average of the complex samples, long enough + that its first null lands on the spike. It is a crude filter and a + deliberately crude one: it is four instructions wide, it runs on a whole + second of samples without noticing, and what it has to reject is one + tone at a frequency this end chose. + + Only then is the magnitude taken. Filtering before detection rather than + after is what keeps the neighbours -- a doorbell, a tyre sensor, a car + key -- from adding themselves to the envelope of the sensor. + """ + if iq.size == 0: + return np.zeros(0, dtype=np.float32), float(sample_rate) + signal = np.asarray(iq) + if offset: + turn = np.exp(-2j * math.pi * offset + * np.arange(signal.size, dtype=np.float64) / sample_rate) + signal = signal * turn.astype(np.complex64) + decimate = max(1, int(sample_rate // max(1.0, want_rate))) + if decimate > 1: + usable = (signal.size // decimate) * decimate + if usable == 0: + return np.zeros(0, dtype=np.float32), float(sample_rate) + signal = signal[:usable].reshape(-1, decimate).mean(axis=1) + return np.abs(signal).astype(np.float32), float(sample_rate) / decimate + + +def bursts(envelope: np.ndarray, rate: float, gap_us: float = 3_000.0, + min_pulses: int = 8, min_run_us: float = 70.0, + min_level: float = 1.8) -> list[Burst]: + """Split an envelope into the runs of pulses worth trying to read. + + The threshold is set between the noise floor and the peak rather than at + a fixed level, because a sensor on the fence and a sensor three gardens + away differ by forty decibels and both should be read. The floor is the + median of the whole block, which is honest here in a way it usually is + not: a block is a second long, a message is a fiftieth of that, and so + the median of a block genuinely is the sound of nothing happening. + """ + if envelope.size < 4: + return [] + floor = float(np.median(envelope)) + peak = float(np.percentile(envelope, 99.95)) + if peak <= floor * min_level or peak <= 0.0: + return [] # nothing above the noise worth slicing + + # Smoothed over a fraction of the shortest pulse these sensors send, so + # that a sample of noise on the wrong side of the threshold does not + # become a pulse, and the shortest real pulse still survives it. + span = max(1, int(round(rate * 30e-6))) + if span > 1: + pad = np.concatenate(([0.0], np.cumsum(envelope, dtype=np.float64))) + smooth = ((pad[span:] - pad[:-span]) / span).astype(np.float32) + else: + smooth = envelope + on = smooth > (floor + 0.5 * (peak - floor)) + if not on.any(): + return [] + + per_us = rate / 1e6 + lengths, values, starts = _runs(on) + lengths, values, starts = _merge_short(lengths, values, starts, + min_run_us * per_us) + + out: list[Burst] = [] + marks: list[float] = [] + spaces: list[float] = [] + began = 0 + for length, high, start in zip(lengths, values, starts): + micro = length / per_us + if high: + if not marks: + began = start + marks.append(micro) + elif marks: + if micro > gap_us: + out.append(_burst_of(marks, spaces, began, rate, floor, peak)) + marks, spaces = [], [] + else: + spaces.append(micro) + if marks: + out.append(_burst_of(marks, spaces, began, rate, floor, peak)) + return [b for b in out if b.pulses >= min_pulses] + + +def _burst_of(marks, spaces, began, rate, floor, peak) -> Burst: + # One more mark than spaces, always: the silence that ended the burst + # belongs to what came after it. + # + # Everything is cast to an ordinary float on the way out. These numbers + # came from numpy and end up as a timestamp in a log and in a file of + # names, and neither JSON nor YAML will write a numpy float -- which is + # the sort of thing that only shows up when a sensor is finally named at + # two in the morning. + marks = [float(m) for m in marks][:len(spaces) + 1] + return Burst(at=float(began) / float(rate), marks=tuple(marks), + spaces=tuple(float(s) for s in spaces), + level=float(peak / floor) if floor > 0 else float("inf")) + + +def _runs(on: np.ndarray): + """Every run of equal values: how long, which value, where it started.""" + change = np.flatnonzero(np.diff(on.view(np.int8))) + edges = np.concatenate(([0], change + 1, [on.size])) + return np.diff(edges), on[edges[:-1]], edges[:-1] + + +def _merge_short(lengths, values, starts, floor_samples: float): + """Fold runs too short to be a pulse into whatever is around them. + + A single sample the wrong side of the threshold in the middle of a mark + would otherwise split one four-hundred-microsecond pulse into three, and + everything after that is arithmetic on nonsense. Smoothing removes most + of it; this removes the rest. + """ + if lengths.size == 0: + return lengths, values, starts + keep = lengths >= max(1.0, floor_samples) + if keep.all(): + return lengths, values, starts + if not keep.any(): + return lengths[:0], values[:0], starts[:0] + out_len, out_val, out_start = [], [], [] + for length, value, start in zip(lengths[keep], values[keep], starts[keep]): + if out_val and out_val[-1] == value: + # Two runs of the same value now touching: the short run between + # them was noise, and its samples belong to the pulse it split. + out_len[-1] = start + length - out_start[-1] + else: + out_len.append(int(length)) + out_val.append(bool(value)) + out_start.append(int(start)) + return (np.array(out_len), np.array(out_val, dtype=bool), + np.array(out_start)) + + +def bits_pwm(burst: Burst) -> str: + """Read a burst as pulse width: a long mark is a one, a short one a zero. + + Nothing here is measured against a clock. The bit is decided by + comparing each mark with the gap that follows it, which is a comparison + between two numbers that arrived a few hundred microseconds apart and so + cannot have drifted with respect to each other. A transmitter running + ten per cent fast is read correctly and never noticed. + + The last pulse of a burst is the exception, and has to be handled or the + final bit of the checksum is lost and with it the message: the gap after + it is the silence before the next copy rather than part of the bit. What + stands in for that gap is the bit period -- a mark and its gap together, + which in this coding is the same length whichever bit it is -- taken as + the median over the burst. A pulse longer than half a bit period is a + one, which is what the coding means. + """ + if not burst.marks: + return "" + spaces = list(burst.spaces) + if len(burst.marks) > len(spaces): + if not spaces: + return "" + period = float(np.median([mark + space for mark, space + in zip(burst.marks, spaces)])) + spaces.append(period / 2.0) + return "".join("1" if mark > space else "0" + for mark, space in zip(burst.marks, spaces)) + + +def bits_ppm(burst: Burst) -> str: + """Read a burst as pulse position: a long gap is a one, a short one zero. + + Here there is nothing to compare a gap against except the other gaps, so + the threshold is halfway between the shortest and the longest of them. + A burst whose gaps are all much the same length is not this coding, and + comes back as an empty string rather than as a row of zeroes that + something downstream might believe. + """ + if len(burst.spaces) < 2: + return "" + low, high = min(burst.spaces), max(burst.spaces) + if high < low * 1.5: + return "" + middle = (low + high) / 2.0 + return "".join("1" if space > middle else "0" for space in burst.spaces) + + +def readings_from(iq: np.ndarray, sample_rate: float, offset: float = 0.0, + when: float = 0.0) -> list[Reading]: + """Every distinct sensor message in one block of samples, in order. + + ``when`` is the clock time the block began, so that each reading carries + the real moment it arrived rather than its offset in a buffer. A log of + weather that started again from zero would be unusable, and a sensor + reports every sixteen seconds, so the difference matters. + + The three copies of a message are looked for across the whole block + rather than inside one burst, because they are not one burst: these + sensors leave ten milliseconds of silence between the copies, which is + three times what ends a burst. Corroboration therefore has to happen + here, where the whole second is in view, and a message that arrived three + times comes back as one reading which knows it arrived three times. + """ + envelope, rate = baseband(iq, sample_rate, offset) + found = [] + for burst in bursts(envelope, rate): + for reading in candidates(bits_pwm(burst)) + candidates(bits_ppm(burst)): + reading.at = when + burst.at + found.append(reading) + return confirmed(found) + + +def confirmed(found: list[Reading]) -> list[Reading]: + """One reading per distinct message, once enough copies have arrived. + + Identical bits are one message heard more than once; different bits are + different messages, even from the same sensor, which is how the 5-in-1's + two halves both survive a block that holds them both. + """ + seen: dict[tuple, list[Reading]] = {} + for reading in found: + seen.setdefault((reading.family, reading.bits), []).append(reading) + out = [] + for (family, _bits), group in seen.items(): + if len(group) < REPEATS_NEEDED[family]: + continue + first = min(group, key=lambda r: r.at) + first.copies = len(group) + out.append(first) + return sorted(out, key=lambda r: (r.at, r.family, r.sensor)) + + +# --------------------------------------------------------------------------- +# Building the same messages, so the decoder can be held to them +# --------------------------------------------------------------------------- +# +# Every one of these lives beside the decoder that reads it, so that the two +# cannot drift apart, and so a test can put a temperature in and take the +# same one out. They are also what the simulator transmits, which means +# `--simulate` runs the real slicer and the real decoder rather than a +# shortcut past them. + +def _txr_bits(data: list[int]) -> str: + """Finish a TXR-family message: parity on the payload, then the sum.""" + data = list(data) + for i in range(2, len(data)): + data[i] &= 0x7F + if parity8(data[i]) != 1: + data[i] |= 0x80 + data.append(sum(data) & 0xFF) + return "".join(format(byte, "08b") for byte in data) + + +def _status(message: int, battery_low: bool) -> int: + return (message & 0x3F) | (0x00 if battery_low else 0x40) + + +def _wind_raw(kph: float) -> int: + return 0 if kph <= 0 else max(1, min(255, int(round((kph - 1.0) / 0.8278)))) + + +def tower_frame(sensor: int, celsius: float, humidity: int, + channel: str = "A", battery_low: bool = False) -> str: + """One 592TXR message: temperature, humidity, checksum and parity.""" + raw = int(round((celsius + 100.0) * 10.0)) + return _txr_bits([((CHANNELS.index(channel) & 3) << 6) | ((sensor >> 8) & 0x3F), + sensor & 0xFF, + _status(0x04, battery_low), + humidity & 0x7F, + (raw >> 7) & 0x0F, + raw & 0x7F]) + + +def five_in_one_wind_rain(sensor: int, kph: float, degrees: float, + rain_counter: int, channel: str = "A", + battery_low: bool = False) -> str: + """The 5-in-1's wind message: speed, where from, and the rain counter.""" + wind = _wind_raw(kph) + point = min(range(16), key=lambda i: abs(WIND_POINTS[i] - degrees % 360)) + counter = rain_counter & 0x3FFF + return _txr_bits([((CHANNELS.index(channel) & 3) << 6) | ((sensor >> 8) & 0x0F), + sensor & 0xFF, + _status(0x31, battery_low), + (wind >> 3) & 0x1F, + ((wind & 0x07) << 4) | point, + (counter >> 7) & 0x7F, + counter & 0x7F]) + + +def five_in_one_weather(sensor: int, kph: float, celsius: float, + humidity: int, channel: str = "A", + battery_low: bool = False) -> str: + """The 5-in-1's other message: speed, temperature and humidity.""" + wind = _wind_raw(kph) + raw = int(round((celsius * 9 / 5 + 32) * 10.0)) + 400 + return _txr_bits([((CHANNELS.index(channel) & 3) << 6) | ((sensor >> 8) & 0x0F), + sensor & 0xFF, + _status(0x38, battery_low), + (wind >> 3) & 0x1F, + ((wind & 0x07) << 4) | ((raw >> 7) & 0x0F), + raw & 0x7F, + humidity & 0x7F]) + + +def lightning_frame(sensor: int, celsius: float, humidity: int, + strikes: int = 0, miles: int = 31, + channel: str = "A", battery_low: bool = False, + interference: bool = False) -> str: + """One 6045M message: the weather, the strike count and how far off.""" + raw = int(round(((celsius * 9 / 5 + 32) + 100.0) * 10.0)) + return _txr_bits([((CHANNELS.index(channel) & 3) << 6) | ((sensor >> 8) & 0x0F), + sensor & 0xFF, + _status(LIGHTNING_MESSAGE, battery_low), + humidity & 0x7F, + (raw >> 7) & 0x1F, + raw & 0x7F, + strikes & 0x7F, + (0x20 if interference else 0x00) | (miles & 0x1F)]) + + +def _twelve(celsius: float) -> int: + raw = int(round(celsius * 10.0)) + return raw + 0x1000 if raw < 0 else raw + + +def frame_609(sensor: int, celsius: float, humidity: int, + battery_low: bool = False) -> str: + """One 609TXC message: identity, temperature, humidity, a sum.""" + raw = _twelve(celsius) + data = [sensor & 0xFF, + (0x00 if battery_low else 0x80) | ((raw >> 8) & 0x0F), + raw & 0xFF, + humidity & 0xFF] + data.append(sum(data) & 0xFF) + return "".join(format(byte, "08b") for byte in data) + + +def frame_606(sensor: int, celsius: float, battery_low: bool = False) -> str: + """One 606TX message: identity, temperature, a CRC-8, and that is all.""" + raw = _twelve(celsius) + data = [sensor & 0xFF, + (0x00 if battery_low else 0x80) | ((raw >> 8) & 0x0F), + raw & 0xFF] + data.append(crc8(data)) + return "".join(format(byte, "08b") for byte in data) + + +# --------------------------------------------------------------------------- +# Turning bits back into something a receiver would have heard +# --------------------------------------------------------------------------- + +# The timings, in microseconds. The newer sensors vary the pulse; the two +# older ones keep the pulse and vary the gap. Neither is read against these +# numbers -- the slicer compares a pulse with its own gap, or a gap with the +# other gaps -- so they are here to transmit with, not to decode by. +PWM_TIMING = {"sync_mark": 600.0, "sync_space": 600.0, "syncs": 4, + "one": (400.0, 220.0), "zero": (220.0, 400.0)} +PPM_TIMING = {"sync_mark": 500.0, "sync_space": 1000.0, "syncs": 1, + "one": (500.0, 2000.0), "zero": (500.0, 1000.0)} +TIMINGS = {"pwm": PWM_TIMING, "ppm": PPM_TIMING} + + +def pulse_train(bits: str, coding: str = "pwm") -> tuple[list, list]: + """One message as the marks and spaces a transmitter would key out.""" + timing = TIMINGS[coding] + marks, spaces = [], [] + for _ in range(timing["syncs"]): + marks.append(timing["sync_mark"]) + spaces.append(timing["sync_space"]) + for bit in bits: + mark, space = timing["one" if bit == "1" else "zero"] + marks.append(mark) + spaces.append(space) + if coding == "ppm": + # In pulse-position coding the bit *is* the gap, so the last gap has + # to be closed by a pulse or there is no gap there to read. That + # trailing pulse carries no bit of its own. + marks.append(timing["sync_mark"]) + return marks, spaces + + +def modulate(messages, sample_rate: float = 1_024_000.0, + coding: str = "pwm", repeats: int = 3, offset: float = 0.0, + amplitude: float = 1.0, noise: float = 0.0, seed: int = 0, + lead_us: float = 4_000.0, gap_us: float = 8_000.0) -> np.ndarray: + """What a receiver would have heard, for one or more messages. + + Each message is sent ``repeats`` times with a short silence between the + copies and a longer one after them, which is what the real sensors do and + is why the two thinly-checked models can be believed at all. + """ + per_us = sample_rate / 1e6 + parts = [np.zeros(int(lead_us * per_us), dtype=np.float32)] + for bits in ([messages] if isinstance(messages, str) else messages): + marks, spaces = pulse_train(bits, coding) + for _ in range(max(1, repeats)): + for mark, space in zip(marks, spaces + [0.0] * len(marks)): + parts.append(np.full(int(round(mark * per_us)), amplitude, + dtype=np.float32)) + if space: + parts.append(np.zeros(int(round(space * per_us)), + dtype=np.float32)) + parts.append(np.zeros(int(gap_us * per_us), dtype=np.float32)) + parts.append(np.zeros(int(gap_us * per_us * 2), dtype=np.float32)) + envelope = np.concatenate(parts) + return _to_air(envelope, sample_rate, offset, noise, seed) + + +def _to_air(envelope: np.ndarray, sample_rate: float, offset: float, + noise: float, seed: int) -> np.ndarray: + """An envelope, put where on the band the receiver expects to find it.""" + signal = envelope.astype(np.complex64) + if offset: + turn = np.exp(2j * math.pi * offset + * np.arange(signal.size, dtype=np.float64) / sample_rate) + signal = signal * turn.astype(np.complex64) + if noise: + rng = np.random.default_rng(seed) + signal = signal + noise * ( + rng.standard_normal(signal.size) + + 1j * rng.standard_normal(signal.size)).astype(np.complex64) + return signal.astype(np.complex64) + + +# --------------------------------------------------------------------------- +# Sensors that are not there +# --------------------------------------------------------------------------- + +@dataclass +class VirtualSensor: + """A sensor on a fence post that does not exist. + + It has an identity, a model, a channel switch and an opinion about the + weather, and it broadcasts all of that exactly as a real one does -- + through the real encoders, the real modulator, the real slicer and the + real decoder. Nothing downstream can tell the difference, which is the + point: everything downstream is what is being exercised. + """ + + family: str = "tower" + sensor: int = 0x1234 + channel: str = "A" + every: float = 16.0 # seconds between transmissions + phase: float = 0.0 # so they do not all speak at once + strength: float = 1.0 + celsius: float = 18.0 + humidity: float = 55.0 + kph: float = 8.0 + rain_counter: int = 0 + strikes: int = 0 + storm_miles: int = 31 + battery_low: bool = False + label: str = "" # what a person would call it + clock: float = 0.0 + turn: int = 0 + + @property + def coding(self) -> str: + return "ppm" if self.family in ("609", "606") else "pwm" + + def messages(self) -> list[str]: + """What it says this time round. + + The 5-in-1 alternates, because it has more to say than fits in one + message; everything else says the same thing every time. + """ + self.turn += 1 + if self.family == "tower": + return [tower_frame(self.sensor, self.celsius, int(self.humidity), + self.channel, self.battery_low)] + if self.family == "5n1": + if self.turn % 2: + return [five_in_one_wind_rain( + self.sensor, self.kph, (self.turn * 23.0) % 360.0, + self.rain_counter, self.channel, self.battery_low)] + return [five_in_one_weather(self.sensor, self.kph, self.celsius, + int(self.humidity), self.channel, + self.battery_low)] + if self.family == "6045": + return [lightning_frame(self.sensor, self.celsius, + int(self.humidity), self.strikes, + self.storm_miles, self.channel, + self.battery_low)] + if self.family == "609": + return [frame_609(self.sensor, self.celsius, int(self.humidity), + self.battery_low)] + return [frame_606(self.sensor, self.celsius, self.battery_low)] + + def advance(self, seconds: float, rng) -> None: + """Let the weather move on, slowly and within reason. + + Slowly matters. A garden that swings six degrees in a minute would + exercise everything downstream perfectly well and would look, to + anyone running ``--simulate`` to see what the program does, like a + program that cannot read a thermometer. So the rates here are + roughly a real afternoon's: about a degree an hour of drift, wind + that gusts around a mean, and rain that falls in tenths of a + millimetre at a time. Everything stays inside what the hardware + could report, so the decoder's own plausibility check is never the + thing that rejects it. + """ + self.clock += seconds + turn = math.sin(self.clock / 1800.0) # half an hour a swing + self.celsius = min(45.0, max(-30.0, self.celsius + + seconds * (0.0004 * turn + + rng.normal(0.0, 0.0008)))) + self.humidity = min(99.0, max(5.0, self.humidity + + seconds * (-0.0016 * turn + + rng.normal(0.0, 0.004)))) + gust = 1.0 + 0.5 * math.sin(self.clock / 41.0) + self.kph = min(90.0, max(0.0, self.kph + + seconds * 0.08 * (gust * 11.0 - self.kph))) + if self.family == "5n1" and rng.random() < 0.01 * seconds: + self.rain_counter += 1 + if self.family == "6045": + if rng.random() < 0.006 * seconds: + self.strikes = (self.strikes + 1) & 0x7F + self.storm_miles = int(rng.integers(1, 25)) + elif rng.random() < 0.002 * seconds: + self.storm_miles = 31 + + +def default_sensors() -> list[VirtualSensor]: + """One of each, in a garden that does not exist. + + The transmission periods are deliberately awkward numbers. Two sensors + on exactly sixteen seconds would transmit over the top of each other + every time they lined up and never come unstuck again, which would be + both unrealistic and unfair: real ones are timed by a cheap oscillator in + the cold and drift past each other within a few minutes. Here they are + simply given periods with no common factor, which has the same effect and + keeps the garden reproducible from a seed. + """ + return [ + VirtualSensor(family="tower", sensor=0x1A2B, channel="A", every=16.1, + phase=1.0, celsius=17.4, humidity=62, + label="back fence"), + VirtualSensor(family="tower", sensor=0x0C41, channel="B", every=16.7, + phase=6.5, celsius=20.9, humidity=44, strength=0.55, + label="greenhouse"), + VirtualSensor(family="5n1", sensor=0x0777, channel="A", every=18.3, + phase=3.0, celsius=16.8, humidity=71, kph=11.0, + rain_counter=1284, label="mast"), + VirtualSensor(family="6045", sensor=0x0311, channel="C", every=24.9, + phase=9.0, celsius=19.1, humidity=58, strikes=3, + storm_miles=12, label="lightning"), + VirtualSensor(family="609", sensor=0x5C, every=30.2, phase=12.0, + celsius=4.2, humidity=80, battery_low=True, + label="shed"), + VirtualSensor(family="606", sensor=0x93, every=32.6, phase=15.0, + celsius=-3.5, strength=0.7, label="freezer"), + ] + + +class SimulatedSensors: + """A receiver-shaped source of weather that is not happening. + + It answers ``read_samples`` the way the dongle does and hands back the + same on-off-keyed bursts a garden full of sensors would, so the whole + section -- the slicer, every decoder, the naming, the log, the report and + the export -- can be run through without an aerial, a sensor or a garden. + """ + + def __init__(self, sensors=None, sample_rate: float = 1_024_000.0, + offset: float = 250_000.0, noise: float = 0.02, + seed: int = 0, realtime: bool = False): + self.sensors = list(sensors) if sensors is not None \ + else default_sensors() + self.sample_rate = float(sample_rate) + self.offset = float(offset) + self.noise = noise + self.rng = np.random.default_rng(seed) + self.frequency = ACURITE_HZ + self.clock = 0.0 + # A block is a second of samples but takes longer than a second to + # slice, so sensors advanced by the length of the block fall behind + # the clock their readings are stamped with. Listening for real, the + # weather moves by the time that actually passed; in a test, by the + # block, so the same seed gives the same garden twice. + self.realtime = realtime + self._last = None + + # -- the shape of a device ------------------------------------------- + def open(self): + return self + + def close(self) -> None: + return None + + def tune(self, hz: float, settle: bool = True) -> int: + self.frequency = hz + return int(hz) + + def read_samples(self, count: int, flush: bool = False) -> np.ndarray: + """One block of garden: whoever is due to speak, speaks.""" + import time as _time + + seconds = count / self.sample_rate + if self.realtime: + now = _time.monotonic() + if self._last is not None: + seconds = max(0.0, now - self._last) + self._last = now + block = np.zeros(count, dtype=np.complex64) + began, ended = self.clock, self.clock + seconds + for sensor in self.sensors: + for at in _due(sensor, began, ended): + burst = modulate(sensor.messages(), self.sample_rate, + coding=sensor.coding, offset=self.offset, + amplitude=sensor.strength, lead_us=0.0) + start = int((at - began) * self.sample_rate) + room = min(burst.size, max(0, count - start)) + if room > 0: + block[start:start + room] += burst[:room] + sensor.advance(seconds, self.rng) + self.clock = ended + if self.noise: + block = block + self.noise * ( + self.rng.standard_normal(count) + + 1j * self.rng.standard_normal(count)).astype(np.complex64) + return block.astype(np.complex64) + + def read_seconds(self, seconds: float, flush: bool = False) -> np.ndarray: + return self.read_samples(int(self.sample_rate * seconds)) + + +def _due(sensor: VirtualSensor, began: float, ended: float) -> list[float]: + """Every moment in a block at which one sensor is due to transmit.""" + if sensor.every <= 0 or ended <= began: + return [] + first = math.ceil((began - sensor.phase) / sensor.every) + out = [] + at = sensor.phase + first * sensor.every + while at < ended: + if at >= began: + out.append(at) + at += sensor.every + return out diff --git a/bandsaunter/aircraft.py b/bandsaunter/aircraft.py index 72de9e9..11d7c32 100644 --- a/bandsaunter/aircraft.py +++ b/bandsaunter/aircraft.py @@ -28,7 +28,7 @@ from .adsb import ADSB_HZ, SAMPLE_RATE from .flightlog import read_position from .settings import Setting, format_value -__all__ = ["AircraftOptions", "OPTIONS", "listen", "watch", "draw", +__all__ = ["AircraftOptions", "OPTIONS", "defaults", "listen", "watch", "draw", "logs_in", "open_device", "open_log", "pump", "finish", "windowed", "format_option", @@ -519,6 +519,15 @@ OPTION_GROUPS = ("Receiver", "Listening", "Aircraft", "Animation", "The map", "Labels", "Effects") +def defaults() -> AircraftOptions: + """A fresh set, for showing what has been changed from it. + + Named the same as the weather section's, so that the menus can show + either without knowing which they are showing. + """ + return AircraftOptions() + + def in_group(group: str) -> list[Setting]: return [o for o in OPTIONS if o.group == group] diff --git a/bandsaunter/cli.py b/bandsaunter/cli.py index 4749a35..0150878 100755 --- a/bandsaunter/cli.py +++ b/bandsaunter/cli.py @@ -58,6 +58,10 @@ examples: bandsaunter adsb --window live map of the aircraft bandsaunter flights --out sky.gif animate what they did bandsaunter flights --theme phosphor draw it as a vector display + bandsaunter weather read the 433 MHz weather sensors + bandsaunter weather --name 1A2B="back fence" name one while listening + bandsaunter readings --csv turn a weather log into a graph + bandsaunter sensors what is out there, and what it is called bandsaunter scan -b 2m --simulate try it without hardware """) # The GNU form: the version, then who holds the copyright and what the @@ -288,6 +292,92 @@ examples: "phosphor, amber and red") ad.set_defaults(log_frames=True, lookup=True) + # -- weather ------------------------------------------------------------- + we = sub.add_parser("weather", + help="read the AcuRite weather sensors on 433 MHz") + we.add_argument("--seconds", type=float, default=None, + help="stop after this long (default: until interrupted)") + we.add_argument("--rate", type=float, default=None, + help="sample rate in Hz; a quarter of a megasample is the " + "minimum") + we.add_argument("--gain", default=None, help="tuner gain in dB, or auto") + we.add_argument("--device", type=int, default=None, help="which receiver") + we.add_argument("--frequency", "--freq", dest="frequency", type=float, + default=None, metavar="HZ", + help="where the sensors are (default: 433.92 MHz)") + we.add_argument("--offset", type=float, default=None, metavar="HZ", + help="how far to one side of them to tune, to keep the " + "receiver's own spike off the signal") + we.add_argument("--simulate", dest="simulate", action="store_true", + default=None, + help="invent a garden of sensors, for a receiver with no " + "aerial") + we.add_argument("--no-simulate", dest="simulate", action="store_false", + default=None, help="listen to real sensors") + we.add_argument("--log", default=None, metavar="FILE", + help="where to write the message log " + "(default: weather_