Read the parity the way the sensors write it
Their sensors are on the air, five of them, and every message was arriving
intact. The bytes they sent, recovered from their own capture:
A7 1B 44 A6 09 4B 00 channel B id 271B 22.7 C 38%
F9 35 44 2B 09 C9 6F channel A id 3935 22.5 C 43%
21 E5 84 1E 0A 9F 51 channel C id 21E5 31.1 C 30% battery low
F9 7D 44 28 09 50 3B channel A id 397D 23.2 C 40%
0D D6 44 A9 09 CF A8 channel C id 0DD6 23.1 C 41%
Every checksum correct, every message type 0x04, every reading plausible. The
framing was right, the bit offset was right, the byte order was right, the
pulses had been recovered perfectly for days. One bit of convention was
wrong: the parity in the top bit of each payload byte is even, and this
required it to be odd. Twenty payload bytes across five independent messages,
every one of them even, which is not something twenty bytes do by chance.
That is the whole fault. Everything else changed in this and the two commits
before it was real and worth doing, and none of it was why nothing decoded.
Three things follow.
The five messages are now a test, checked byte for byte against the weather
they carry. They are worth more than everything else in that file put
together: every other test there puts a reading in through an encoder written
from the same description as the decoder, so the two agree by construction and
agree about anything they are both wrong about -- which is exactly what
happened. An encoder tested against its own decoder cannot find a fault in
the description they share, and no amount of it would ever have found this.
The emptiness check earns its place now. Odd parity rejects a byte of all
zeroes; even parity accepts one, so a run of silence read as zeroes satisfies
both the parity and a sum of zero, and the only thing standing between that
and a display full of sensors is the test that some byte is non-zero. It was
there for tidiness and is now load-bearing; the comment says so.
And the readings of a burst are tried in order and the search stops at the
first that yields anything, rather than pooling them. Half a dozen readings
at two byte orders is sixteen times the chances for a coincidence to satisfy a
twelve-bit check, and sensors that were not there began appearing in the
invented garden the moment the alternatives went in -- caught by the test that
asks whether everything heard is something that exists. Stopping early costs
nothing: a burst that reads correctly the ordinary way never reaches the
alternatives, and one that does not reaches them exactly as before.
Full suite 2351 passed, checked against three more deliberately broken builds.
Sixty seconds of receiver noise yields nothing and eight hundred seconds of
the invented garden yields no sensor that is not there. Built as
2026-09-07_05.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PsWPTweCT6pwxKngvVxcg
This commit is contained in:
The Dust Council
parent
dfea5cb2f2
commit
872eadac37
6 changed files with 193 additions and 36 deletions
26
README.md
26
README.md
|
|
@ -1906,9 +1906,13 @@ transmitting, with an identity that stays the same, is worth a line on its
|
|||
own. `--no-unknown` leaves them off.
|
||||
|
||||
These formats are implemented from their published descriptions and are
|
||||
checked against frames built from the same descriptions. That proves the
|
||||
framing, the parity, the checksums and the arithmetic; it is not the same as
|
||||
having held every one of these sensors.
|
||||
checked against frames built from the same descriptions — which proves the
|
||||
framing, the parity, the checksums and the arithmetic, and proves nothing
|
||||
about anything the description and this both get wrong. The tower sensor is
|
||||
also checked against five messages recovered from real hardware, byte for
|
||||
byte, and those are worth more than all the rest put together: they are what
|
||||
caught the parity, which no amount of testing an encoder against its own
|
||||
decoder ever could.
|
||||
|
||||
### Why nothing false gets through
|
||||
|
||||
|
|
@ -1918,8 +1922,10 @@ looks at every bit offset of every burst will find a message in noise if it is
|
|||
allowed to. Four things stop it.
|
||||
|
||||
**The checks the message carries.** The three newer models have an eight-bit
|
||||
sum plus odd parity in the top bit of every payload byte — twelve to fourteen
|
||||
bits of check on a message of seven to nine bytes.
|
||||
sum plus **even** parity in the top bit of every payload byte — twelve to
|
||||
fourteen bits of check on a message of seven to nine bytes. Even, not odd:
|
||||
that one bit of convention was wrong here for four days and the cost was that
|
||||
nothing decoded at all, every other check passing and saying so.
|
||||
|
||||
**Arriving twice.** The two older models carry one byte of check between them,
|
||||
which is one false message in two hundred and fifty-six, and that is not a
|
||||
|
|
@ -1931,6 +1937,16 @@ times in a row, for exactly this reason.
|
|||
hardware could not have sent. Nothing outside −40 to 70 °C, 0 to 100% or a
|
||||
wind the anemometer cannot physically report is accepted.
|
||||
|
||||
**How many readings are tried.** The readings of a burst are tried in order of
|
||||
likelihood and the search stops at the first that yields anything, rather than
|
||||
pooling all of them. Half a dozen readings at two byte orders is sixteen times
|
||||
as many chances for a coincidence to satisfy a checksum, and twelve bits of
|
||||
check is a rate that is comfortable once and uncomfortable sixteen times over
|
||||
— sensors that were not there began appearing the moment the alternatives went
|
||||
in. Stopping early costs nothing: a burst that reads correctly the ordinary
|
||||
way never reaches the alternatives, and one that does not reaches them exactly
|
||||
as before.
|
||||
|
||||
**Where the message sits.** This is the one that is easy to get wrong. A
|
||||
seven-byte message read out of the front of a real eight-byte one is made of
|
||||
that message's own payload bytes, whose parity is *already correct* — so the
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ and transcribing speech.
|
|||
# 2026-08-21_02 is the second build made on the 21st. The revision is padded
|
||||
# to two digits so versions sort as text.
|
||||
VERSION_DATE = "2026-09-07"
|
||||
VERSION_REVISION = 4
|
||||
VERSION_REVISION = 5
|
||||
|
||||
__version__ = f"{VERSION_DATE}_{VERSION_REVISION:02d}"
|
||||
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@ Lightning 6045M 9 temperature, humidity, strikes, how far off
|
|||
|
||||
The first three share a framing -- two bytes of identity, a byte saying what
|
||||
kind of message this is, the payload, and a checksum which is the sum of
|
||||
everything before it -- and the payload bytes each carry odd parity in their
|
||||
everything before it -- and the payload bytes each carry even parity in their
|
||||
top bit. That is between twelve and fourteen bits of check on a message of
|
||||
seven to nine bytes, which is enough to accept a message on a band where
|
||||
doorbells, car keys, tyre sensors and someone's garage all transmit.
|
||||
|
|
@ -285,18 +285,28 @@ REPEATS_NEEDED = {family: repeats for family, _name, _bytes, repeats in MODELS}
|
|||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _txr_framed(data: list[int]) -> bool:
|
||||
"""The checks the whole TXR family carries: a sum, and odd parity.
|
||||
"""The checks the whole TXR family carries: a sum, and even parity.
|
||||
|
||||
The sum covers every byte before it. The parity is in the top bit of
|
||||
each payload byte -- everything but the two identity bytes and the
|
||||
checksum itself -- and is odd, so a byte of all zeroes fails it and a
|
||||
run of silence read as zeroes cannot become a message.
|
||||
checksum itself -- and is *even*: the bit is set so that the number of
|
||||
bits in the byte comes out even.
|
||||
|
||||
Even rather than odd, which is worth saying plainly because this had it
|
||||
the other way round and nothing whatever was decoded for it. Five
|
||||
messages off five real sensors settled it: twenty payload bytes, every
|
||||
one of them even, which is not something twenty bytes do by chance.
|
||||
|
||||
One consequence has to be handled here rather than assumed away. Odd
|
||||
parity rejects a byte of all zeroes and even parity accepts it, so a run
|
||||
of silence read as zeroes satisfies both this and a sum of zero -- which
|
||||
is why the emptiness test below is a check and not a nicety.
|
||||
"""
|
||||
if (sum(data[:-1]) & 0xFF) != data[-1]:
|
||||
return False
|
||||
if not any(data):
|
||||
return False
|
||||
return all(parity8(byte) == 1 for byte in data[2:-1])
|
||||
return all(parity8(byte) == 0 for byte in data[2:-1])
|
||||
|
||||
|
||||
def _txr_identity(data: list[int], id_bits: int) -> tuple[str, str]:
|
||||
|
|
@ -313,6 +323,9 @@ def decode_tower(data: list[int]) -> Reading | None:
|
|||
whose bit 6 is set while the battery is good, humidity in seven bits, and
|
||||
temperature in eleven -- tenths of a degree Celsius offset by a hundred,
|
||||
so that the coldest thing it can report is still a positive number.
|
||||
|
||||
This one has been held against messages off real sensors rather than only
|
||||
against messages built here; see the tests.
|
||||
"""
|
||||
if len(data) != 7 or not _txr_framed(data):
|
||||
return None
|
||||
|
|
@ -672,10 +685,34 @@ def decode_burst(burst: "Burst") -> Reading | None:
|
|||
burst where the edges have moved -- both readings of the same pulses are
|
||||
tried, and the checksums say which one it was.
|
||||
"""
|
||||
found = []
|
||||
return _best(_from_burst(burst))
|
||||
|
||||
|
||||
def _from_burst(burst: "Burst") -> list[Reading]:
|
||||
"""The messages in one burst, from the first reading of it that yields any.
|
||||
|
||||
The readings are tried in order of likelihood and the search stops at the
|
||||
first one that produces anything, rather than pooling all of them. That
|
||||
matters more than it looks.
|
||||
|
||||
Half a dozen readings of a burst, each tried at both byte orders, is
|
||||
sixteen times as many chances for a coincidence to satisfy a checksum as
|
||||
one reading was -- and the checks here are twelve to fourteen bits, which
|
||||
is a rate that is comfortable once and uncomfortable sixteen times over.
|
||||
Sensors that are not there began appearing in the invented garden the
|
||||
moment the alternatives went in.
|
||||
|
||||
Stopping early costs nothing. A burst that reads correctly under the
|
||||
ordinary reading never reaches the alternatives; a burst that does not
|
||||
reach them exactly as it did before. The fallbacks are there for the
|
||||
sensor whose bits are drawn some other way, and that sensor is not in
|
||||
competition with anything.
|
||||
"""
|
||||
for bits in slicings(burst):
|
||||
found += candidates(bits)
|
||||
return _best(found)
|
||||
found = candidates(bits)
|
||||
if found:
|
||||
return found
|
||||
return []
|
||||
|
||||
|
||||
|
||||
|
|
@ -1272,9 +1309,8 @@ def readings_from(iq: np.ndarray, sample_rate: float, offset: float = 0.0,
|
|||
# two thinly-checked models and a display full of sensors that are
|
||||
# not there.
|
||||
here: dict = {}
|
||||
for bits in slicings(burst):
|
||||
for reading in candidates(bits):
|
||||
here.setdefault((reading.family, reading.bits), reading)
|
||||
for reading in _from_burst(burst):
|
||||
here.setdefault((reading.family, reading.bits), reading)
|
||||
for reading in here.values():
|
||||
reading.at = when + burst.at
|
||||
found.append(reading)
|
||||
|
|
@ -1365,7 +1401,7 @@ def _checked(count: int, at: int, data: list[int],
|
|||
"""Every check that framing would have to pass, run one at a time."""
|
||||
if count in (7, 8, 9):
|
||||
checks = (("sum", (sum(data[:-1]) & 0xFF) == data[-1]),
|
||||
("parity", all(parity8(byte) == 1 for byte in data[2:-1])),
|
||||
("parity", all(parity8(byte) == 0 for byte in data[2:-1])),
|
||||
("type", (data[2] & 0x3F) in _KNOWN_TYPES))
|
||||
elif count == 5:
|
||||
checks = (("sum", (sum(data[:4]) & 0xFF) == data[4]),)
|
||||
|
|
@ -1474,7 +1510,7 @@ def _txr_bits(data: list[int]) -> str:
|
|||
data = list(data)
|
||||
for i in range(2, len(data)):
|
||||
data[i] &= 0x7F
|
||||
if parity8(data[i]) != 1:
|
||||
if parity8(data[i]) != 0: # even, as the sensors send it
|
||||
data[i] |= 0x80
|
||||
data.append(sum(data) & 0xFF)
|
||||
return "".join(format(byte, "08b") for byte in data)
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
.\" Generated by packaging/make-man.py -- do not edit by hand.
|
||||
.TH BANDSAUNTER 1 "2026-09-07" "bandsaunter 2026-09-07_04" "User Commands"
|
||||
.TH BANDSAUNTER 1 "2026-09-07" "bandsaunter 2026-09-07_05" "User Commands"
|
||||
.SH NAME
|
||||
bandsaunter \- scan, record and identify radio signals with an RTL-SDR
|
||||
.SH SYNOPSIS
|
||||
|
|
@ -2206,9 +2206,11 @@ to match is reported as an unknown message type with its identity and nothing
|
|||
else, rather than guessed at.
|
||||
.PP
|
||||
These formats are implemented from their published descriptions and are
|
||||
checked against frames built from the same descriptions. That proves the
|
||||
framing, the parity, the checksums and the arithmetic; it is not the same as
|
||||
having held every one of these sensors.
|
||||
checked against frames built from the same descriptions, which proves the
|
||||
framing, the parity, the checksums and the arithmetic and proves nothing about
|
||||
anything a description and an implementation of it both get wrong. The tower
|
||||
sensor is additionally checked against messages recovered from real hardware,
|
||||
byte for byte.
|
||||
.SS Naming a sensor
|
||||
A sensor broadcasts an identity, and that identity is a number that came out
|
||||
of a hat in a factory \[em] or a different number out of the same hat the next
|
||||
|
|
@ -2245,8 +2247,10 @@ being stale.
|
|||
garage doors \[em] and a decoder that looks at every bit offset of every burst
|
||||
will find a message in noise if it is allowed to. Four things stop it.
|
||||
.PP
|
||||
The three newer models carry an eight-bit sum plus odd parity in the top bit
|
||||
of every payload byte, which is twelve to fourteen bits of check.
|
||||
The three newer models carry an eight-bit sum plus even parity in the top bit
|
||||
of every payload byte, which is twelve to fourteen bits of check. Even and not
|
||||
odd: that one bit of convention was wrong here and the cost was that nothing
|
||||
decoded at all, every other check in the format passing and saying so.
|
||||
.PP
|
||||
The two older models carry one byte of check between them, which is one false
|
||||
message in two hundred and fifty-six, so those two are only believed when the
|
||||
|
|
|
|||
|
|
@ -1318,9 +1318,11 @@ to match is reported as an unknown message type with its identity and nothing
|
|||
else, rather than guessed at.
|
||||
.PP
|
||||
These formats are implemented from their published descriptions and are
|
||||
checked against frames built from the same descriptions. That proves the
|
||||
framing, the parity, the checksums and the arithmetic; it is not the same as
|
||||
having held every one of these sensors.
|
||||
checked against frames built from the same descriptions, which proves the
|
||||
framing, the parity, the checksums and the arithmetic and proves nothing about
|
||||
anything a description and an implementation of it both get wrong. The tower
|
||||
sensor is additionally checked against messages recovered from real hardware,
|
||||
byte for byte.
|
||||
.SS Naming a sensor
|
||||
A sensor broadcasts an identity, and that identity is a number that came out
|
||||
of a hat in a factory \[em] or a different number out of the same hat the next
|
||||
|
|
@ -1357,8 +1359,10 @@ being stale.
|
|||
garage doors \[em] and a decoder that looks at every bit offset of every burst
|
||||
will find a message in noise if it is allowed to. Four things stop it.
|
||||
.PP
|
||||
The three newer models carry an eight-bit sum plus odd parity in the top bit
|
||||
of every payload byte, which is twelve to fourteen bits of check.
|
||||
The three newer models carry an eight-bit sum plus even parity in the top bit
|
||||
of every payload byte, which is twelve to fourteen bits of check. Even and not
|
||||
odd: that one bit of convention was wrong here and the cost was that nothing
|
||||
decoded at all, every other check in the format passing and saying so.
|
||||
.PP
|
||||
The two older models carry one byte of check between them, which is one false
|
||||
message in two hundred and fifty-six, so those two are only believed when the
|
||||
|
|
|
|||
|
|
@ -31,6 +31,101 @@ def heard(bits, coding="pwm", rate=RATE, offset=OFFSET, noise=0.05,
|
|||
return a.readings_from(iq, rate, offset=offset)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Messages off real sensors
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# Five 592TXR messages recovered from the air by somebody who owns the
|
||||
# sensors, checked here byte for byte. They are worth more than everything
|
||||
# below them put together: every other test in this file puts a reading in
|
||||
# through an encoder written from the same description as the decoder, so the
|
||||
# two agree by construction and agree about anything they are both wrong
|
||||
# about. These do not come from here, and the one thing they caught is the
|
||||
# thing that construction could never catch.
|
||||
#
|
||||
# What they caught was the parity. This read the top bit of every payload
|
||||
# byte as odd parity; these five say it is even, twenty payload bytes out of
|
||||
# twenty, which is not something twenty bytes do by accident. Nothing was
|
||||
# decoded at all until they arrived.
|
||||
|
||||
REAL_MESSAGES = [
|
||||
# (bytes, channel, id, temperature, humidity, battery low)
|
||||
("A7 1B 44 A6 09 4B 00", "B", "271B", 22.7, 38, False),
|
||||
("F9 35 44 2B 09 C9 6F", "A", "3935", 22.5, 43, False),
|
||||
("21 E5 84 1E 0A 9F 51", "C", "21E5", 31.1, 30, True),
|
||||
("F9 7D 44 28 09 50 3B", "A", "397D", 23.2, 40, False),
|
||||
("0D D6 44 A9 09 CF A8", "C", "0DD6", 23.1, 41, False),
|
||||
]
|
||||
|
||||
|
||||
def as_bits(text):
|
||||
return "".join(format(int(byte, 16), "08b") for byte in text.split())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("text,channel,sensor,celsius,humidity,flat",
|
||||
REAL_MESSAGES)
|
||||
def test_a_message_off_a_real_sensor_reads_as_the_weather_it_was(
|
||||
text, channel, sensor, celsius, humidity, flat):
|
||||
got = a.decode(as_bits(text), confirm=False)
|
||||
assert got is not None, "a real message this program could not read"
|
||||
assert got.family == "tower" and got.sensor == sensor
|
||||
assert got.channel == channel
|
||||
assert got.value("temperature") == pytest.approx(celsius, abs=0.05)
|
||||
assert got.value("humidity") == humidity
|
||||
assert got.battery_low is flat
|
||||
|
||||
|
||||
@pytest.mark.parametrize("text,_c,_s,_t,_h,_b", REAL_MESSAGES)
|
||||
def test_the_payload_bytes_of_a_real_message_carry_even_parity(text, _c, _s,
|
||||
_t, _h, _b):
|
||||
"""Stated on its own, because it is the whole of what went wrong.
|
||||
|
||||
Held the other way round, this decoded nothing whatever -- not badly, not
|
||||
occasionally, but nothing at all -- while every other check in the format
|
||||
passed and said so.
|
||||
"""
|
||||
data = [int(byte, 16) for byte in text.split()]
|
||||
assert all(a.parity8(byte) == 0 for byte in data[2:-1])
|
||||
assert (sum(data[:-1]) & 0xFF) == data[-1]
|
||||
|
||||
|
||||
def test_a_real_message_survives_the_whole_path_from_the_air():
|
||||
"""Keyed at the timings their receiver actually measured.
|
||||
|
||||
Four sync pulses at 612 microseconds, data at 216 and 403 with gaps that
|
||||
complete the bit period, and the terminating pulse a real transmitter
|
||||
sends after the last bit -- which nothing here had ever seen, because
|
||||
nothing here had ever sent one.
|
||||
"""
|
||||
text = REAL_MESSAGES[0][0]
|
||||
bits = as_bits(text)
|
||||
pairs = ([(612.0, 601.0)] * 4
|
||||
+ [(403.0, 209.0) if bit == "1" else (216.0, 395.0)
|
||||
for bit in bits]
|
||||
+ [(216.0, 0.0)]) # the terminator, carrying no bit
|
||||
got = a.readings_from(keyed_at(bits, pairs, reset_us=9_000.0), 250_000.0)
|
||||
assert [r.sensor for r in got] == ["271B"]
|
||||
assert got[0].value("temperature") == pytest.approx(22.7, abs=0.05)
|
||||
|
||||
|
||||
def test_a_burst_off_a_real_sensor_is_sixty_one_pulses_and_not_sixty():
|
||||
"""Fifty-six bits of message, four of sync, and one to close the last bit.
|
||||
|
||||
Worth pinning because everything here was written against sixty, the
|
||||
simulator not sending the terminating pulse that real hardware does, and
|
||||
"it happens to work anyway" is not a thing to leave unrecorded.
|
||||
"""
|
||||
bits = as_bits(REAL_MESSAGES[0][0])
|
||||
pairs = ([(612.0, 601.0)] * 4
|
||||
+ [(403.0, 209.0) if bit == "1" else (216.0, 395.0)
|
||||
for bit in bits]
|
||||
+ [(216.0, 0.0)])
|
||||
iq = keyed_at(bits, pairs, reset_us=9_000.0)
|
||||
found = a.bursts(*a.baseband(iq, 250_000.0, 0.0))
|
||||
assert found and all(b.pulses == 61 for b in found)
|
||||
assert len(bits) == 56
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The tower sensor: what most people have
|
||||
# ---------------------------------------------------------------------------
|
||||
|
|
@ -734,7 +829,9 @@ def test_the_pulse_lengths_of_a_burst_are_reported_as_the_protocol_shape():
|
|||
noise=0.02), RATE, OFFSET))[0]
|
||||
marks = a.timings(burst.marks)
|
||||
assert len(marks) == 3 # short, long, sync
|
||||
assert [n for _v, n in marks] == [28, 28, 4]
|
||||
short, long, sync = (n for _v, n in marks)
|
||||
assert sync == 4 # the sync pulses
|
||||
assert short + long == 56 # one per bit of the message
|
||||
assert [round(v / 10) * 10 for v, _n in marks] == [220, 400, 600]
|
||||
assert sum(n for _v, n in marks) == burst.pulses
|
||||
|
||||
|
|
@ -964,15 +1061,15 @@ def test_a_message_is_read_from_whichever_end_of_a_byte_it_arrives(order):
|
|||
def test_reading_a_byte_backwards_keeps_its_parity_and_breaks_its_sum():
|
||||
"""The signature that names this fault, which is why it is worth having.
|
||||
|
||||
Reversing the bits of a byte does not change how many are set, so odd
|
||||
parity survives it; a checksum does not. A message read from the wrong
|
||||
end therefore shows every parity holding and the sum failing, on every
|
||||
copy -- which is a fingerprint rather than a guess.
|
||||
Reversing the bits of a byte does not change how many are set, so parity
|
||||
survives it; a checksum does not. A message read from the wrong end
|
||||
therefore shows every parity holding and the sum failing, on every copy
|
||||
-- which is a fingerprint rather than a guess.
|
||||
"""
|
||||
frame = a.tower_frame(0x1A2B, 21.5, 48, "A")
|
||||
backwards = reversed_bytes(frame)
|
||||
plain = [int(backwards[i:i + 8], 2) for i in range(0, len(backwards), 8)]
|
||||
assert all(a.parity8(byte) == 1 for byte in plain[2:-1])
|
||||
assert all(a.parity8(byte) == 0 for byte in plain[2:-1])
|
||||
assert (sum(plain[:-1]) & 0xFF) != plain[-1]
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue