Read the pictures, the aircraft, the meters and the sensors
Hexadecimal is a true answer to "what did that say" and not a useful one. This is the work of turning the rest of what a receiver hears into something a person can read, and most of it is pictures. PICTURES Three of the things on the air are images rather than sounds, and all three arrive as the audio a scan already records: SSTV 14.230 and 144.5 MHz Martin M1/M2, Scottie S1/S2/DX, Robot 36/72 APT 137-138 MHz the NOAA weather satellites HF fax 2-20 MHz, sideband the marine weather charts Each is written from its published specification, and the generators used to test them are written from the same specification without reference to the decoders -- so a picture that comes back matching the one that went in is evidence about the format. Every SSTV mode reproduces its published line time exactly, which is worth failing a test over: a line a few milliseconds long walks the picture off the screen inside ten lines. Against synthetic transmissions at 30 dB SNR, SSTV is 96-98% of pixels exact, APT correlates at 0.97 and fax at 0.998; all three still read at 6-12 dB. None of the three is guessed at, and that is what makes it safe to try them on every recording. SSTV needs its VIS header, APT needs both line syncs at the right distance from each other, fax needs the phasing signal. No false pictures in 295 attempts over noise, tones, speech and swept whistles. Two things had to be got right beyond the arithmetic. A band-pass does not switch between two tones, it slides between them, so every edge is measured at the midpoint of the slide rather than at the first sample past a threshold -- the earlier version was reading the coarse search stride back as the edge and shifting Martin M1 sideways by a whole colour bar. And a picture now keeps its capture whatever the content check made of it: a satellite is a steady tone with a wobble on it and SSTV is a whistle, so both were being discarded as "no signal content" having already been recognised. PNG is written here rather than pulled in from Pillow. A scanner that cannot start because an imaging library is missing is worse than one that cannot draw. saunterbrowse marks a picture in the list, gives its path in full -- wrapped rather than cut off, because half a path opens nothing -- and moves or deletes the PNGs with the recording. o prints the picture's path, not the audio's. GRIB is not a modulation and is not pretended to be one. It is the format weather models are published in and it travels by satellite link and by e-mail; where a decoded byte stream begins with its magic number it is named, and that is all. AIRCRAFT `bandsaunter adsb` parks the receiver on 1090 MHz and reads Mode S extended squitter: address, callsign, altitude, position, speed. A command of its own because a megabit a second will not go through a channel twelve and a half kilohertz wide. Every frame carries a 24-bit checksum so there is no threshold anywhere in it -- with one trap, which is that a frame of all zeros satisfies that checksum and silence is exactly that. Positions round trip exactly through compact position reporting, and a pair straddling a longitude-zone boundary is refused rather than resolved against two grids. METERS AND SENSORS Itron ERT utility meters on 900 MHz and AcuRite weather sensors on 433 MHz are named rather than reported as hex, and neither is believed without its own checksum -- BCH(255,239) for the meter, a checksum and four parity bits for the sensor. Both are implemented from published descriptions and checked against frames built from the same descriptions, which proves the framing and the arithmetic and is not the same as having held a meter. HEX INTO WORDS Everything else that decodes to bits now gets its fields named where the shape is standard, its text read out where there is text, and its bytes laid out in groups with the printable characters beside them. The text search is where the care went, because printability is not evidence. Forty framings of each packet, and seven-bit values printable three in four, meant a bar set on printability called 64% of random payloads text. Real text is nearly all one case where random letters are half and half, two fifths vowels where random is a fifth, and mostly alphanumeric where random draws punctuation one time in four. Together: under 0.5%, measured in the suite. CALLSIGNS The licensed address is recorded in full -- the street, not merely the town -- and goes into the KML with everything else. US amateur records are public by law and carry it; holding it and not saying so is worse than either showing it or not asking, and --no-lookup asks for none of it. Also here: Morse is decoded again from the whole recording where the capture was made in cw mode. The first pass works from the classifier's buffer, which holds a few seconds -- enough to say "this is Morse", not enough to catch a callsign whole between two word gaps, so a beacon repeating every eight seconds through an eight-second window was never identified. And classify._psk_order took the logarithm of zero on a silent block. 1318 tests, up from 1161. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016PsWPTweCT6pwxKngvVxcg
This commit is contained in:
The Dust Council
parent
b4718aa425
commit
3d7f76118e
32 changed files with 4426 additions and 39 deletions
149
README.md
149
README.md
|
|
@ -819,6 +819,154 @@ text is still shown; it is the *identification* that is held to the stricter
|
|||
standard. Over 1805 truncated captures of four different messages, that turns
|
||||
107 invented callsigns into none, while still recovering 550 correct ones.
|
||||
|
||||
## Pictures
|
||||
|
||||
Three of the things a receiver can hear are images rather than sounds. All
|
||||
three are analogue, all three encode brightness as a frequency, and all three
|
||||
arrive as the audio the scanner already records — so they are looked for in
|
||||
every recording and written out as PNG beside it.
|
||||
|
||||
| | Where | What it is |
|
||||
|---|---|---|
|
||||
| **SSTV** | 14.230 MHz, 144.5 MHz | amateur slow-scan television, in colour — Martin M1/M2, Scottie S1/S2/DX, Robot 36/72 |
|
||||
| **APT** | 137–138 MHz | the NOAA weather satellites, one continuous picture per fifteen-minute pass |
|
||||
| **HF fax** | 2–20 MHz, single sideband | the marine weather charts, 60–240 lines a minute |
|
||||
|
||||
```
|
||||
147.06 MHz 118.0s SNR 24.1 dB SSTV (Martin M1)
|
||||
picture: SSTV Martin M1 320x256 → ~/bandsaunter/0014.230000MHz--2026-08-29_14_02_11-usb.png
|
||||
```
|
||||
|
||||
**None of the three is guessed at**, which is what makes it safe to try them
|
||||
on every recording. SSTV announces itself with a VIS header that says which
|
||||
mode follows. APT carries two different sync patterns exactly 1040 words
|
||||
apart. Fax opens with twenty seconds of phasing — black lines with a pulse at
|
||||
the start of each — that nothing else on the air sends. A decoder without one
|
||||
of those draws static beautifully, and a directory of beautifully rendered
|
||||
static is worse than an empty one. Measured over noise, tones, speech and
|
||||
frequency-swept whistles: **no false pictures in 295 attempts**.
|
||||
|
||||
Accuracy, against transmissions built from the published specifications:
|
||||
|
||||
| | 30 dB SNR | 12 dB | 6 dB |
|
||||
|---|---|---|---|
|
||||
| SSTV (all seven modes) | 96–98% of pixels exact | 91–98% | — (header lost below 9 dB) |
|
||||
| APT | 0.97 correlation | 0.94 | 0.88 |
|
||||
| HF fax | 0.998 correlation | 0.99 | 0.97 |
|
||||
|
||||
Each mode's line timing is checked against its published line time — 446.446 ms
|
||||
for Martin M1, 428.220 for Scottie S1, and so on — because a line a few
|
||||
milliseconds long walks the picture off the bottom of the screen within ten
|
||||
lines, and that is a thing worth failing a test over rather than noticing in a
|
||||
PNG.
|
||||
|
||||
**A picture keeps its capture whatever the content check made of it.** A
|
||||
satellite is a steady tone with a wobble on it and an SSTV transmission is a
|
||||
whistle: neither is speech, neither has symbol structure, and both were being
|
||||
discarded as "no signal content" *having already been recognised*.
|
||||
|
||||
Pictures take minutes rather than seconds, so `--record` has to be long enough
|
||||
or what arrives is the top of one. A partial picture is kept and labelled
|
||||
partial rather than thrown away — most SSTV captures are partial, and half a
|
||||
picture is still a picture.
|
||||
|
||||
> **GRIB** is sometimes asked about in the same breath and is not a modulation:
|
||||
> it is the binary format weather models are published in, and it travels by
|
||||
> satellite data link and by e-mail rather than as something a receiver
|
||||
> demodulates. Where a decoded byte stream begins with its magic number it is
|
||||
> named; nothing here fetches or renders one.
|
||||
|
||||
## Aircraft
|
||||
|
||||
```bash
|
||||
bandsaunter adsb # listen on 1090 MHz until interrupted
|
||||
bandsaunter adsb --frames # print every frame as it arrives
|
||||
bandsaunter adsb --kml planes.kml # and write what was heard as a map
|
||||
```
|
||||
|
||||
Every airliner overhead broadcasts its address, callsign, altitude, position
|
||||
and speed twice a second, unencrypted, to nobody in particular.
|
||||
|
||||
```
|
||||
ICAO callsign altitude position speed frames
|
||||
4CA1FA RYR1234 35000 ft 51.5000, -0.1200 308 kt 126° 47
|
||||
A0B1C2 UAL99 12000 ft 40.7000, -74.0000 180 kt 274° 31
|
||||
```
|
||||
|
||||
A command of its own because **ADS-B does not fit through the scanner**: the
|
||||
signalling is a megabit a second, which needs at least two megasamples a second
|
||||
of raw receiver output, and the scan path decimates everything to a channel
|
||||
12.5 kHz wide long before a decoder sees it.
|
||||
|
||||
Every frame carries a 24-bit checksum, so there is no threshold and nothing to
|
||||
disbelieve — a frame passes or it is dropped. The one trap is that a frame of
|
||||
all zeros satisfies that checksum, and silence between transmissions is exactly
|
||||
that, so silence would otherwise decode as an endless stream of aircraft
|
||||
`000000`.
|
||||
|
||||
A position takes **two** frames. The encoding sends a fraction of a zone rather
|
||||
than a coordinate, so one frame alone is ambiguous by hundreds of miles; an
|
||||
aircraft is placed once an even and an odd frame have both arrived, about a
|
||||
second apart. A pair that straddles a longitude-zone boundary is refused rather
|
||||
than resolved against two different grids.
|
||||
|
||||
An aerial cut for 1090 MHz is the difference between hearing the airport and
|
||||
hearing the county; the whip supplied with a dongle is a quarter of the length
|
||||
it wants.
|
||||
|
||||
## Meters and weather sensors
|
||||
|
||||
Two things on the ISM bands are worth naming rather than reporting as hex.
|
||||
|
||||
```
|
||||
915.0 MHz decoded: electricity meter 12345678 reading 987654
|
||||
433.92 MHz decoded: AcuRite sensor 1234 temperature 21.5 C humidity 48% channel A
|
||||
```
|
||||
|
||||
**Utility meters.** The Itron ERT modules fitted to electricity, gas and water
|
||||
meters across North America broadcast their reading every thirty seconds or so
|
||||
on 902–928 MHz, in the clear, so a van can drive past and read a street. The
|
||||
message says which meter, what kind, what the register reads, and whether the
|
||||
tamper switches have been tripped.
|
||||
|
||||
**AcuRite sensors.** The 433.92 MHz outdoor sensors sold with every consumer
|
||||
weather station send temperature, humidity, battery state and a channel letter
|
||||
every sixteen seconds.
|
||||
|
||||
Neither is guessed at: a meter message carries a 16-bit BCH check and a sensor
|
||||
message a checksum and four parity bits, and nothing is reported that has not
|
||||
satisfied them. Both are implemented from their published descriptions and
|
||||
checked against frames built from the same descriptions — which proves the
|
||||
framing and the arithmetic, and is not the same as having held a meter.
|
||||
|
||||
## Hex into words
|
||||
|
||||
Everything else that decodes to bits gets read rather than dumped:
|
||||
|
||||
```
|
||||
EV1527 / PT2262: address 0x8B2F1 button B
|
||||
text (8-bit ASCII): "STATION OPEN"
|
||||
0000 53 54 41 54 49 4F 4E 20 |STATION |
|
||||
0008 4F 50 45 4E 0D 0A 00 91 |OPEN....|
|
||||
```
|
||||
|
||||
A decoder that stops at a bit string has done half the job, and `4A 3F 1B 22`
|
||||
is a true answer to "what did the doorbell say" and not a useful one. Where the
|
||||
packet is a shape somebody standardised its fields are named; where there is
|
||||
text in it the text is read out; and underneath either, always, the bytes in
|
||||
groups with their printable characters beside them.
|
||||
|
||||
The text search is the part that needs care, and **printability is not
|
||||
evidence**. Every framing is tried at every bit offset in both bit orders —
|
||||
about forty readings of each packet — and seven-bit values are printable three
|
||||
times in four, so a bar set on printability alone called **64% of random
|
||||
payloads text**. What separates a message from a coincidence is that real text
|
||||
is nearly all one case where random letters are half and half, is about two
|
||||
fifths vowels where random letters over 52 are a fifth, and is mostly letters
|
||||
and digits where random draws punctuation one time in four. Together with a
|
||||
length bar those take random payloads to **under 0.5%**, which is measured in
|
||||
the test suite and fails there if it stops being true.
|
||||
|
||||
### Single sideband
|
||||
|
||||
SSB needs no `--mode usb`. Nothing else does either, but SSB is the mode where
|
||||
|
|
@ -1192,6 +1340,7 @@ or symbol rate where there is one, and the bands the frequency falls in.
|
|||
| `s` | sort by time, frequency or length |
|
||||
| `r` | re-read the directory, picking up what a running scan has written |
|
||||
| `o` | print the file's path and quit |
|
||||
| — | a picture is marked in the list, with the path of its PNG |
|
||||
| `S` `I` `N` | file it into `saved/`, `investigate/` or `noise/` |
|
||||
| `u` | put the last one filed back |
|
||||
| `d` | delete it and its sidecars, for good — asks first |
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ and transcribing speech.
|
|||
# 2026-08-21_02 is the second build made on the 21st. The revision is padded
|
||||
# to two digits so versions sort as text.
|
||||
VERSION_DATE = "2026-08-29"
|
||||
VERSION_REVISION = 2
|
||||
VERSION_REVISION = 3
|
||||
|
||||
__version__ = f"{VERSION_DATE}_{VERSION_REVISION:02d}"
|
||||
|
||||
|
|
|
|||
444
bandsaunter/adsb.py
Normal file
444
bandsaunter/adsb.py
Normal file
|
|
@ -0,0 +1,444 @@
|
|||
"""ADS-B: aircraft on 1090 MHz saying where they are.
|
||||
|
||||
Every airliner overhead broadcasts its identity, position, altitude and speed
|
||||
twice a second, unencrypted, to nobody in particular. The format is Mode S
|
||||
extended squitter and it is the easiest useful thing a receiver can decode,
|
||||
because every frame carries a 24-bit checksum that either comes out right or
|
||||
does not -- there is no judgement anywhere in this module about whether a
|
||||
decode is believable.
|
||||
|
||||
The signalling is pulse-position modulation at a megabit a second. Each bit
|
||||
is one microsecond wide and split in half: energy in the first half is a one,
|
||||
energy in the second half is a zero. A frame opens with a preamble of four
|
||||
pulses at 0, 1, 3.5 and 4.5 microseconds, which is what is searched for.
|
||||
|
||||
That rate is why this does not ride on the ordinary scan path. A megabit a
|
||||
second needs at least two megasamples a second of raw receiver output, and
|
||||
the scanner's channels are twelve and a half kilohertz wide; ``bandsaunter
|
||||
adsb`` parks the receiver on 1090 MHz at full rate instead.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
import numpy as np
|
||||
|
||||
__all__ = ["decode_adsb", "Frame", "Aircraft", "AircraftRegistry", "crc24",
|
||||
"ADSB_HZ", "SAMPLE_RATE", "PREAMBLE_US"]
|
||||
|
||||
ADSB_HZ = 1_090_000_000.0
|
||||
|
||||
# The lowest rate this can work at: one megabit a second, sampled twice a bit.
|
||||
SAMPLE_RATE = 2_000_000
|
||||
|
||||
PREAMBLE_US = (0.0, 1.0, 3.5, 4.5)
|
||||
SHORT_BITS = 56
|
||||
LONG_BITS = 112
|
||||
|
||||
# The characters a callsign can be built from, six bits each. The hashes are
|
||||
# the code points the standard leaves unassigned.
|
||||
CALLSIGN_CHARS = ("#ABCDEFGHIJKLMNOPQRSTUVWXYZ#####_###############"
|
||||
"0123456789######")
|
||||
|
||||
TYPE_NAMES = {
|
||||
(1, 4): "identification",
|
||||
(5, 8): "surface position",
|
||||
(9, 18): "airborne position",
|
||||
(19, 19): "velocity",
|
||||
(20, 22): "airborne position (GNSS height)",
|
||||
}
|
||||
|
||||
|
||||
def crc24(data: bytes) -> int:
|
||||
"""The Mode S parity, polynomial 0xFFF409.
|
||||
|
||||
Run over the whole frame including its three parity bytes, a good frame
|
||||
gives zero. That is the entire error check in this module and it is
|
||||
enough: twenty-four bits of it means a frame passes by chance one time in
|
||||
sixteen million.
|
||||
"""
|
||||
poly = 0xFFF409
|
||||
crc = 0
|
||||
for byte in data:
|
||||
crc ^= byte << 16
|
||||
for _ in range(8):
|
||||
crc = (((crc << 1) ^ poly) & 0xFFFFFF if crc & 0x800000
|
||||
else (crc << 1) & 0xFFFFFF)
|
||||
return crc
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# One frame
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class Frame:
|
||||
"""One Mode S frame that passed its checksum."""
|
||||
|
||||
bits: str = ""
|
||||
data: bytes = b""
|
||||
df: int = 0 # downlink format
|
||||
icao: str = "" # the aircraft's permanent 24-bit address
|
||||
type_code: int = 0
|
||||
at_sample: int = 0
|
||||
callsign: str = ""
|
||||
altitude_ft: int = 0
|
||||
latitude: float = 0.0
|
||||
longitude: float = 0.0
|
||||
cpr_odd: bool = False
|
||||
cpr_lat: int = 0
|
||||
cpr_lon: int = 0
|
||||
ground_speed_kt: float = 0.0
|
||||
track_deg: float = 0.0
|
||||
vertical_rate_fpm: int = 0
|
||||
|
||||
@property
|
||||
def what(self) -> str:
|
||||
for (low, high), name in TYPE_NAMES.items():
|
||||
if low <= self.type_code <= high:
|
||||
return name
|
||||
return f"type {self.type_code}"
|
||||
|
||||
def describe(self) -> str:
|
||||
bits = [self.icao]
|
||||
if self.callsign:
|
||||
bits.append(self.callsign)
|
||||
if self.altitude_ft:
|
||||
bits.append(f"{self.altitude_ft} ft")
|
||||
if self.latitude or self.longitude:
|
||||
bits.append(f"{self.latitude:.4f},{self.longitude:.4f}")
|
||||
if self.ground_speed_kt:
|
||||
bits.append(f"{self.ground_speed_kt:.0f} kt")
|
||||
bits.append(f"{self.track_deg:.0f}°")
|
||||
if self.vertical_rate_fpm:
|
||||
bits.append(f"{self.vertical_rate_fpm:+d} fpm")
|
||||
if len(bits) == 1:
|
||||
bits.append(self.what)
|
||||
return " ".join(bits)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Finding frames in the samples
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _magnitude(iq: np.ndarray) -> np.ndarray:
|
||||
if np.iscomplexobj(iq):
|
||||
return np.abs(iq).astype(np.float32)
|
||||
return np.abs(np.asarray(iq, dtype=np.float32))
|
||||
|
||||
|
||||
def _preamble_score(mag: np.ndarray, per_us: float) -> np.ndarray:
|
||||
"""How much each sample looks like the start of a preamble.
|
||||
|
||||
The four pulses minus the four gaps that have to be quiet between them.
|
||||
A correlation with the pattern alone finds a steady carrier just as
|
||||
happily; requiring the gaps is what makes it a preamble.
|
||||
"""
|
||||
pulses = [int(round(us * per_us)) for us in PREAMBLE_US]
|
||||
quiet = [int(round(us * per_us)) for us in (2.0, 2.5, 3.0, 6.0, 6.5, 7.0)]
|
||||
width = max(1, int(round(0.5 * per_us)))
|
||||
need = int(round(8.0 * per_us))
|
||||
if mag.size < need + width:
|
||||
return np.zeros(0, dtype=np.float32)
|
||||
|
||||
def at(offsets):
|
||||
total = np.zeros(mag.size - need - width, dtype=np.float32)
|
||||
for offset in offsets:
|
||||
total += mag[offset:offset + total.size]
|
||||
return total / len(offsets)
|
||||
|
||||
return at(pulses) - at(quiet)
|
||||
|
||||
|
||||
def _bits_at(mag: np.ndarray, start: int, per_us: float, count: int) -> str:
|
||||
"""Read ``count`` pulse-position bits: loud first half is a one."""
|
||||
out = []
|
||||
half = 0.5 * per_us
|
||||
base = start + 8.0 * per_us
|
||||
for i in range(count):
|
||||
first = base + i * per_us
|
||||
a0, a1 = int(round(first)), int(round(first + half))
|
||||
b0, b1 = a1, int(round(first + per_us))
|
||||
if b1 > mag.size:
|
||||
break
|
||||
early = float(mag[a0:a1].sum())
|
||||
late = float(mag[b0:b1].sum())
|
||||
out.append("1" if early > late else "0")
|
||||
return "".join(out)
|
||||
|
||||
|
||||
# The formats whose parity is the checksum itself. Everything else has the
|
||||
# aircraft's address exclusive-ored into it, so a decoder without a list of
|
||||
# the aircraft it expects to hear cannot check them at all.
|
||||
CHECKABLE_FORMATS = frozenset({11, 17, 18})
|
||||
|
||||
|
||||
def _plausible(data: bytes) -> bool:
|
||||
"""Whether a candidate is a real frame rather than a run of silence.
|
||||
|
||||
The checksum does the work, with one exception that matters: a frame of
|
||||
all zeros passes it, because zero divided by anything leaves nothing.
|
||||
Silence between transmissions is exactly that, so it would otherwise
|
||||
decode as an endless stream of aircraft 000000.
|
||||
"""
|
||||
if not any(data):
|
||||
return False
|
||||
if (data[0] >> 3) not in CHECKABLE_FORMATS:
|
||||
return False
|
||||
return crc24(data) == 0
|
||||
|
||||
|
||||
def _bytes_of(bits: str) -> bytes:
|
||||
whole = len(bits) - len(bits) % 8
|
||||
return bytes(int(bits[i:i + 8], 2) for i in range(0, whole, 8))
|
||||
|
||||
|
||||
def decode_frames(iq: np.ndarray, sample_rate: float) -> list[Frame]:
|
||||
"""Every Mode S frame in a block of raw receiver output.
|
||||
|
||||
Only frames whose checksum comes out right are returned, so there is no
|
||||
threshold to tune and nothing to disbelieve.
|
||||
"""
|
||||
if sample_rate < SAMPLE_RATE * 0.99:
|
||||
return []
|
||||
mag = _magnitude(iq)
|
||||
per_us = sample_rate / 1e6
|
||||
score = _preamble_score(mag, per_us)
|
||||
if score.size == 0:
|
||||
return []
|
||||
# A preamble stands well above the noise around it. This is only a
|
||||
# shortlist -- the checksum decides -- so it is set low enough to let
|
||||
# weak frames through and high enough not to try every sample.
|
||||
floor = float(np.median(mag)) * 2.0
|
||||
candidates = np.flatnonzero(score > max(floor, float(np.std(score))))
|
||||
frames: list[Frame] = []
|
||||
seen: set[int] = set()
|
||||
for start in candidates:
|
||||
if any(abs(start - s) < per_us for s in seen):
|
||||
continue
|
||||
for count in (LONG_BITS, SHORT_BITS):
|
||||
bits = _bits_at(mag, int(start), per_us, count)
|
||||
if len(bits) < count:
|
||||
continue
|
||||
data = _bytes_of(bits)
|
||||
if len(data) * 8 != count or not _plausible(data):
|
||||
continue
|
||||
frame = _read(bits, data)
|
||||
frame.at_sample = int(start)
|
||||
frames.append(frame)
|
||||
seen.add(int(start))
|
||||
break
|
||||
return frames
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# What a frame says
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _read(bits: str, data: bytes) -> Frame:
|
||||
frame = Frame(bits=bits, data=data, df=data[0] >> 3)
|
||||
frame.icao = f"{int.from_bytes(data[1:4], 'big'):06X}"
|
||||
if frame.df not in (17, 18) or len(data) < 11:
|
||||
return frame
|
||||
me = bits[32:88]
|
||||
frame.type_code = int(me[:5], 2)
|
||||
if 1 <= frame.type_code <= 4:
|
||||
frame.callsign = _callsign(me)
|
||||
elif 9 <= frame.type_code <= 18 or 20 <= frame.type_code <= 22:
|
||||
frame.altitude_ft = _altitude(me)
|
||||
frame.cpr_odd = me[21] == "1"
|
||||
frame.cpr_lat = int(me[22:39], 2)
|
||||
frame.cpr_lon = int(me[39:56], 2)
|
||||
elif frame.type_code == 19:
|
||||
_velocity(frame, me)
|
||||
return frame
|
||||
|
||||
|
||||
def _callsign(me: str) -> str:
|
||||
chars = [CALLSIGN_CHARS[int(me[8 + 6 * i:14 + 6 * i], 2)] for i in range(8)]
|
||||
return "".join(chars).replace("#", "").strip("_ ").strip()
|
||||
|
||||
|
||||
def _altitude(me: str) -> int:
|
||||
"""The 12-bit altitude field, in feet.
|
||||
|
||||
The Q bit says which of two encodings is in use: 25-foot steps, which is
|
||||
everything in normal service, or the older 100-foot Gillham code, which
|
||||
is not decoded here and comes back as zero rather than as a wrong number.
|
||||
"""
|
||||
field = me[8:20]
|
||||
if field == "0" * 12:
|
||||
return 0
|
||||
q_bit = field[7]
|
||||
if q_bit != "1":
|
||||
return 0
|
||||
value = int(field[:7] + field[8:], 2)
|
||||
return value * 25 - 1000
|
||||
|
||||
|
||||
def _velocity(frame: Frame, me: str) -> None:
|
||||
subtype = int(me[5:8], 2)
|
||||
if subtype not in (1, 2):
|
||||
return # airspeed rather than ground speed
|
||||
sign_ew = -1 if me[13] == "1" else 1
|
||||
ew = int(me[14:24], 2) - 1
|
||||
sign_ns = -1 if me[24] == "1" else 1
|
||||
ns = int(me[25:35], 2) - 1
|
||||
if ew < 0 or ns < 0:
|
||||
return
|
||||
scale = 4.0 if subtype == 2 else 1.0 # supersonic
|
||||
vx, vy = sign_ew * ew * scale, sign_ns * ns * scale
|
||||
frame.ground_speed_kt = math.hypot(vx, vy)
|
||||
frame.track_deg = math.degrees(math.atan2(vx, vy)) % 360.0
|
||||
rate = int(me[37:46], 2)
|
||||
if rate:
|
||||
frame.vertical_rate_fpm = (rate - 1) * 64 * (-1 if me[36] == "1" else 1)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Position, which takes two frames
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _nl(lat: float) -> int:
|
||||
"""How many longitude zones there are at this latitude."""
|
||||
if abs(lat) >= 87.0:
|
||||
return 1
|
||||
if lat == 0:
|
||||
return 59
|
||||
inner = 1 - (1 - math.cos(math.pi / (2 * 15))) / \
|
||||
math.cos(math.radians(abs(lat))) ** 2
|
||||
inner = max(-1.0, min(1.0, inner))
|
||||
return int(math.floor(2 * math.pi / math.acos(inner)))
|
||||
|
||||
|
||||
def global_position(even: Frame, odd: Frame,
|
||||
even_first: bool = True) -> tuple[float, float] | None:
|
||||
"""Where an aircraft is, from one even and one odd position frame.
|
||||
|
||||
Compact position reporting sends a latitude and longitude as fractions of
|
||||
a zone, with the zones laid out differently in the two frame types. One
|
||||
frame alone is ambiguous by hundreds of miles; the pair is not.
|
||||
"""
|
||||
lat_even, lon_even = even.cpr_lat / 131072.0, even.cpr_lon / 131072.0
|
||||
lat_odd, lon_odd = odd.cpr_lat / 131072.0, odd.cpr_lon / 131072.0
|
||||
j = math.floor(59 * lat_even - 60 * lat_odd + 0.5)
|
||||
rlat_even = (360.0 / 60) * ((j % 60) + lat_even)
|
||||
rlat_odd = (360.0 / 59) * ((j % 59) + lat_odd)
|
||||
if rlat_even >= 270:
|
||||
rlat_even -= 360
|
||||
if rlat_odd >= 270:
|
||||
rlat_odd -= 360
|
||||
if _nl(rlat_even) != _nl(rlat_odd):
|
||||
return None # the two straddle a zone boundary
|
||||
nl = _nl(rlat_even)
|
||||
if even_first:
|
||||
lat = rlat_even
|
||||
ni = max(nl, 1)
|
||||
m = math.floor(lon_even * (nl - 1) - lon_odd * nl + 0.5)
|
||||
lon = (360.0 / ni) * ((m % ni) + lon_even)
|
||||
else:
|
||||
lat = rlat_odd
|
||||
ni = max(nl - 1, 1)
|
||||
m = math.floor(lon_even * (nl - 1) - lon_odd * nl + 0.5)
|
||||
lon = (360.0 / ni) * ((m % ni) + lon_odd)
|
||||
if lon >= 180:
|
||||
lon -= 360
|
||||
return lat, lon
|
||||
|
||||
|
||||
@dataclass
|
||||
class Aircraft:
|
||||
"""What has been heard from one aircraft."""
|
||||
|
||||
icao: str
|
||||
callsign: str = ""
|
||||
altitude_ft: int = 0
|
||||
latitude: float = 0.0
|
||||
longitude: float = 0.0
|
||||
ground_speed_kt: float = 0.0
|
||||
track_deg: float = 0.0
|
||||
vertical_rate_fpm: int = 0
|
||||
messages: int = 0
|
||||
first_seen: float = 0.0
|
||||
last_seen: float = 0.0
|
||||
_even: Frame | None = field(default=None, repr=False)
|
||||
_odd: Frame | None = field(default=None, repr=False)
|
||||
|
||||
@property
|
||||
def located(self) -> bool:
|
||||
return bool(self.latitude or self.longitude)
|
||||
|
||||
def describe(self) -> str:
|
||||
bits = [self.icao]
|
||||
if self.callsign:
|
||||
bits.append(self.callsign)
|
||||
if self.located:
|
||||
bits.append(f"{self.latitude:.4f},{self.longitude:.4f}")
|
||||
if self.altitude_ft:
|
||||
bits.append(f"{self.altitude_ft} ft")
|
||||
if self.ground_speed_kt:
|
||||
bits.append(f"{self.ground_speed_kt:.0f} kt {self.track_deg:.0f}°")
|
||||
return " ".join(bits)
|
||||
|
||||
|
||||
class AircraftRegistry:
|
||||
"""Everything heard, gathered by aircraft rather than by frame.
|
||||
|
||||
Position needs an even frame and an odd one, which arrive half a second
|
||||
apart, so something has to remember the first while the second is on its
|
||||
way. This is that, and it is also what turns four hundred frames into
|
||||
the dozen aircraft they came from.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.aircraft: dict[str, Aircraft] = {}
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self.aircraft)
|
||||
|
||||
def add(self, frame: Frame, when: float = 0.0) -> Aircraft:
|
||||
seen = self.aircraft.get(frame.icao)
|
||||
if seen is None:
|
||||
seen = Aircraft(icao=frame.icao, first_seen=when)
|
||||
self.aircraft[frame.icao] = seen
|
||||
seen.messages += 1
|
||||
seen.last_seen = when
|
||||
if frame.callsign:
|
||||
seen.callsign = frame.callsign
|
||||
if frame.altitude_ft:
|
||||
seen.altitude_ft = frame.altitude_ft
|
||||
if frame.ground_speed_kt:
|
||||
seen.ground_speed_kt = frame.ground_speed_kt
|
||||
seen.track_deg = frame.track_deg
|
||||
if frame.vertical_rate_fpm:
|
||||
seen.vertical_rate_fpm = frame.vertical_rate_fpm
|
||||
if frame.cpr_lat or frame.cpr_lon:
|
||||
if frame.cpr_odd:
|
||||
seen._odd = frame
|
||||
else:
|
||||
seen._even = frame
|
||||
if seen._even is not None and seen._odd is not None:
|
||||
# Whichever of the pair arrived later is the one the position
|
||||
# is reported at.
|
||||
even_first = seen._even.at_sample > seen._odd.at_sample
|
||||
found = global_position(seen._even, seen._odd, even_first)
|
||||
if found is not None:
|
||||
seen.latitude, seen.longitude = found
|
||||
return seen
|
||||
|
||||
def described(self) -> list[str]:
|
||||
return [craft.describe() for craft in
|
||||
sorted(self.aircraft.values(), key=lambda a: a.icao)]
|
||||
|
||||
|
||||
def decode_adsb(iq: np.ndarray, sample_rate: float,
|
||||
registry: AircraftRegistry | None = None
|
||||
) -> tuple[list[Frame], AircraftRegistry]:
|
||||
"""Decode every frame in a block, and fold them into aircraft."""
|
||||
registry = registry if registry is not None else AircraftRegistry()
|
||||
frames = decode_frames(iq, sample_rate)
|
||||
for frame in frames:
|
||||
registry.add(frame, when=frame.at_sample / sample_rate)
|
||||
return frames, registry
|
||||
210
bandsaunter/apt.py
Normal file
210
bandsaunter/apt.py
Normal file
|
|
@ -0,0 +1,210 @@
|
|||
"""APT: the pictures the NOAA weather satellites have been sending since 1978.
|
||||
|
||||
A polar orbiter crosses the sky in about fifteen minutes and spends all of it
|
||||
sending one continuous picture on 137 MHz. The format is stubbornly simple.
|
||||
A 2400 Hz tone is amplitude-modulated with the brightness of the image, two
|
||||
lines every second, 2080 words to a line -- and the line is sent twice over,
|
||||
once for each of the two sensors the satellite is carrying. Where one of
|
||||
them is visible light and the other infrared, you get the same coastline
|
||||
twice, once as daylight and once as temperature.
|
||||
|
||||
The line is laid out to fixed word counts, which is what makes it decodable
|
||||
without knowing anything about the satellite:
|
||||
|
||||
sync A 39 a 1040 Hz square wave, seven cycles
|
||||
space A 47 a marker for the minute
|
||||
channel A 909 the picture
|
||||
telemetry A 45 a staircase of known greys, for calibration
|
||||
sync B 39 a 832 Hz square wave, seven cycles
|
||||
space B 47
|
||||
channel B 909 the other sensor
|
||||
telemetry B 45
|
||||
|
||||
The whole of the decoding is finding where sync A starts and then counting.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
|
||||
from .images import ImageDecode, resample_to, tone_amplitude
|
||||
|
||||
__all__ = ["decode_apt", "WORD_RATE", "WORDS_PER_LINE", "SUBCARRIER_HZ",
|
||||
"CHANNEL_A", "CHANNEL_B", "apt_frequency"]
|
||||
|
||||
SUBCARRIER_HZ = 2400.0
|
||||
WORD_RATE = 4160.0 # words per second
|
||||
WORDS_PER_LINE = 2080
|
||||
LINES_PER_SECOND = 2.0
|
||||
|
||||
# Where each part of a line begins and how long it is, in words.
|
||||
SYNC_A = (0, 39)
|
||||
SPACE_A = (39, 47)
|
||||
CHANNEL_A = (86, 909)
|
||||
TELEMETRY_A = (995, 45)
|
||||
SYNC_B = (1040, 39)
|
||||
SPACE_B = (1079, 47)
|
||||
CHANNEL_B = (1126, 909)
|
||||
TELEMETRY_B = (2035, 45)
|
||||
|
||||
# The band the satellites use. Nothing else is APT, and running the decoder
|
||||
# over every capture on every band would find sync in static often enough to
|
||||
# matter.
|
||||
APT_BAND = (136.0e6, 138.5e6)
|
||||
|
||||
# How far the sync correlation has to stand above the rest of the line before
|
||||
# the picture is believed, measured in median absolute deviations of the
|
||||
# other 2079 phases. Static scores about five, because the largest of two
|
||||
# thousand roughly-normal numbers is a few deviations up whatever produced
|
||||
# them; a real pass scores sixty at 6 dB and a hundred and fifty on a good
|
||||
# one. Twenty sits in the gap.
|
||||
MIN_SYNC_PEAK = 20.0
|
||||
|
||||
# A picture modulates the subcarrier; a steady tone does not. Without this,
|
||||
# an unmodulated carrier scores five hundred -- not because it looks like
|
||||
# sync but because a flat envelope has no spread for the peak to be measured
|
||||
# against, and dividing by nothing produces a large number rather than an
|
||||
# error.
|
||||
MIN_ENVELOPE_SPREAD = 0.05
|
||||
|
||||
MIN_LINES = 12
|
||||
|
||||
|
||||
def apt_frequency(hz: float) -> bool:
|
||||
"""Whether a frequency is in the band the APT satellites transmit on."""
|
||||
return APT_BAND[0] <= hz <= APT_BAND[1]
|
||||
|
||||
|
||||
def _sync_template(cycle: tuple[float, ...]) -> np.ndarray:
|
||||
"""One sync pattern, in words: seven cycles, padded to 39.
|
||||
|
||||
At 4160 words a second, sync A's 1040 Hz is four words a cycle -- two
|
||||
high, two low -- and sync B's 832 Hz is five. Written out rather than
|
||||
synthesised because that is what they are.
|
||||
"""
|
||||
pattern = [0.0, 0.0, 0.0, 0.0] + list(cycle) * 7
|
||||
pattern += [0.0] * max(0, 39 - len(pattern))
|
||||
return np.array(pattern[:39], dtype=np.float64)
|
||||
|
||||
|
||||
SYNC_A_CYCLE = (1.0, 1.0, 0.0, 0.0)
|
||||
SYNC_B_CYCLE = (1.0, 1.0, 1.0, 0.0, 0.0)
|
||||
|
||||
|
||||
def _fold(words: np.ndarray, template: np.ndarray) -> np.ndarray | None:
|
||||
"""Correlate with one sync pattern and fold at the line length.
|
||||
|
||||
Every line's sync lands on the same phase, so summing over lines pulls it
|
||||
out of the noise however weak any single line is.
|
||||
"""
|
||||
template = template - template.mean()
|
||||
corr = np.correlate(words - words.mean(), template, mode="valid")
|
||||
lines = corr.size // WORDS_PER_LINE
|
||||
if lines < 2:
|
||||
return None
|
||||
return corr[:lines * WORDS_PER_LINE].reshape(
|
||||
lines, WORDS_PER_LINE).sum(axis=0)
|
||||
|
||||
|
||||
def _standout(score: np.ndarray, at: int) -> float:
|
||||
spread = float(np.median(np.abs(score - np.median(score))))
|
||||
if spread <= 1e-9:
|
||||
return 0.0
|
||||
return float(score[at] - np.median(score)) / spread
|
||||
|
||||
|
||||
def _find_phase(words: np.ndarray) -> tuple[int, float]:
|
||||
"""Where sync A falls, and how strongly.
|
||||
|
||||
Both syncs are required, and required in the right places. A line
|
||||
carries sync A at its head and sync B exactly 1040 words later -- a
|
||||
different pattern at a fixed distance -- and static that manages one of
|
||||
those by chance does not manage the other one 1040 words along.
|
||||
"""
|
||||
score_a = _fold(words, _sync_template(SYNC_A_CYCLE))
|
||||
if score_a is None:
|
||||
return 0, 0.0
|
||||
phase = int(np.argmax(score_a))
|
||||
peak = _standout(score_a, phase)
|
||||
|
||||
score_b = _fold(words, _sync_template(SYNC_B_CYCLE))
|
||||
if score_b is None:
|
||||
return phase, 0.0
|
||||
at_b = (phase + SYNC_B[0]) % WORDS_PER_LINE
|
||||
# The weaker of the two, so that both have to be there.
|
||||
return phase, min(peak, _standout(score_b, at_b))
|
||||
|
||||
|
||||
def _stretch(image: np.ndarray) -> np.ndarray:
|
||||
"""Map the picture onto 0-255 by its own distribution.
|
||||
|
||||
The satellite's own telemetry wedge is the calibrated way to do this, but
|
||||
it is only worth reading when the whole wedge has arrived; percentiles of
|
||||
the picture work on a pass that started halfway through, which most of
|
||||
them do.
|
||||
"""
|
||||
body = image[np.isfinite(image)]
|
||||
if body.size == 0:
|
||||
return np.zeros_like(image, dtype=np.uint8)
|
||||
low = float(np.percentile(body, 1.0))
|
||||
high = float(np.percentile(body, 99.0))
|
||||
if high <= low:
|
||||
return np.zeros_like(image, dtype=np.uint8)
|
||||
scaled = (image - low) / (high - low) * 255.0
|
||||
return np.clip(scaled, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def decode_apt(audio: np.ndarray, rate: float,
|
||||
frequency: float = 0.0) -> ImageDecode | None:
|
||||
"""Decode an APT picture from the demodulated audio of a satellite pass.
|
||||
|
||||
``frequency`` is the dial reading, used only to refuse work: APT is a
|
||||
137 MHz thing, and a decoder let loose on every capture would eventually
|
||||
find sync in static. Pass zero to decode regardless, which is what the
|
||||
analyse command does when a person has asked for it directly.
|
||||
"""
|
||||
audio = np.asarray(audio, dtype=np.float64).ravel()
|
||||
if frequency and not apt_frequency(frequency):
|
||||
return None
|
||||
seconds = audio.size / rate
|
||||
if seconds < MIN_LINES / LINES_PER_SECOND:
|
||||
return None
|
||||
|
||||
envelope = tone_amplitude(audio, rate, SUBCARRIER_HZ, width=1400.0)
|
||||
if envelope.size == 0:
|
||||
return None
|
||||
words = resample_to(envelope, int(round(seconds * WORD_RATE)))
|
||||
if words.size < 2 * WORDS_PER_LINE:
|
||||
return None
|
||||
|
||||
spread = float(np.std(words)) / max(1e-9, float(np.mean(np.abs(words))))
|
||||
if spread < MIN_ENVELOPE_SPREAD:
|
||||
return ImageDecode(kind="APT",
|
||||
note="a steady 2400 Hz tone with no picture on it")
|
||||
|
||||
phase, peak = _find_phase(words)
|
||||
if peak < MIN_SYNC_PEAK:
|
||||
return ImageDecode(kind="APT", note="no APT line sync in it")
|
||||
|
||||
lines = (words.size - phase) // WORDS_PER_LINE
|
||||
if lines < MIN_LINES:
|
||||
return ImageDecode(kind="APT",
|
||||
note=f"APT sync, but only {lines} line(s) arrived")
|
||||
frame = words[phase:phase + lines * WORDS_PER_LINE]
|
||||
frame = frame.reshape(lines, WORDS_PER_LINE)
|
||||
picture = _stretch(frame)
|
||||
|
||||
out = ImageDecode(
|
||||
ok=True, kind="APT", mode="NOAA APT", width=WORDS_PER_LINE,
|
||||
height=lines, pixels=picture, complete=True,
|
||||
seconds=lines / LINES_PER_SECOND,
|
||||
confidence=float(min(0.95, 0.5 + peak / 40.0)))
|
||||
# The two sensors, cut out and kept separately, because that is what
|
||||
# anybody actually looks at: one is the daylight picture and the other is
|
||||
# the infrared, and side by side with the sync bars between them they are
|
||||
# harder to read than either alone.
|
||||
out.channels = {
|
||||
"A": frame[:, CHANNEL_A[0]:CHANNEL_A[0] + CHANNEL_A[1]],
|
||||
"B": frame[:, CHANNEL_B[0]:CHANNEL_B[0] + CHANNEL_B[1]],
|
||||
}
|
||||
return out
|
||||
|
|
@ -68,6 +68,7 @@ CATEGORY_STYLE = {
|
|||
"trunk": "bright_black",
|
||||
"carrier": "magenta",
|
||||
"noise": "bright_black",
|
||||
"image": "bold magenta",
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -187,6 +188,71 @@ class Capture:
|
|||
bits.append(str(checks[0]))
|
||||
return " ".join(bits)
|
||||
|
||||
@property
|
||||
def image_path(self) -> str:
|
||||
"""The picture this recording turned out to be, if it was one.
|
||||
|
||||
The sidecar is asked first and the directory second: a recording
|
||||
copied somewhere else keeps its sidecar but not the absolute path in
|
||||
it, and the picture is beside the .wav either way.
|
||||
"""
|
||||
saved = str(self.meta.get("image_path", "")).strip()
|
||||
if saved and Path(saved).is_file():
|
||||
return saved
|
||||
beside = self.path.with_suffix(".png")
|
||||
if beside.is_file():
|
||||
return str(beside)
|
||||
return saved
|
||||
|
||||
@property
|
||||
def image_kind(self) -> str:
|
||||
return str(self.meta.get("image_kind", "")).strip()
|
||||
|
||||
@property
|
||||
def image_mode(self) -> str:
|
||||
return str(self.meta.get("image_mode", "")).strip()
|
||||
|
||||
@property
|
||||
def image_size(self) -> str:
|
||||
width = int(self.meta.get("image_width") or 0)
|
||||
height = int(self.meta.get("image_height") or 0)
|
||||
return f"{width}x{height}" if width and height else ""
|
||||
|
||||
@property
|
||||
def image_headline(self) -> str:
|
||||
"""What kind of picture it is, in one line."""
|
||||
if not (self.image_kind or self.image_path):
|
||||
return ""
|
||||
bits = [self.image_kind or "image"]
|
||||
if self.image_mode:
|
||||
bits.append(self.image_mode)
|
||||
if self.image_size:
|
||||
bits.append(self.image_size)
|
||||
if self.meta.get("image_complete") is False:
|
||||
bits.append("partial")
|
||||
return " ".join(bits)
|
||||
|
||||
@property
|
||||
def images(self) -> list[str]:
|
||||
"""Every picture file belonging to this recording.
|
||||
|
||||
More than one for a weather satellite: the pass is written whole, and
|
||||
each of the satellite's two sensors is written again on its own,
|
||||
because that is what anybody actually looks at.
|
||||
"""
|
||||
found: list[str] = []
|
||||
main = self.image_path
|
||||
if main:
|
||||
found.append(main)
|
||||
for path in (self.meta.get("image_paths") or []):
|
||||
if str(path) not in found and Path(str(path)).is_file():
|
||||
found.append(str(path))
|
||||
stem = self.path.stem
|
||||
for extra in sorted(self.path.parent.glob(glob.escape(stem) + "_*.png")):
|
||||
if str(extra) not in found:
|
||||
found.append(str(extra))
|
||||
return found
|
||||
|
||||
@property
|
||||
def morse(self) -> str:
|
||||
"""What a CW capture was keying, if anything."""
|
||||
|
|
@ -304,12 +370,20 @@ class Capture:
|
|||
if p.is_file())
|
||||
except OSError:
|
||||
pass
|
||||
# The two that do not follow the pattern, because a transcript is
|
||||
# The ones that do not follow the pattern, because a transcript is
|
||||
# named after the recording rather than sharing its extension.
|
||||
for tail in ("_transcription.txt", "_data.txt"):
|
||||
side = self.path.with_name(stem + tail)
|
||||
if side.is_file():
|
||||
found.add(side)
|
||||
# A weather satellite pass is written three times over -- the whole
|
||||
# frame and one file per sensor -- and all three belong to it.
|
||||
try:
|
||||
found.update(p for p in
|
||||
self.path.parent.glob(glob.escape(stem) + "_*.png")
|
||||
if p.is_file())
|
||||
except OSError:
|
||||
pass
|
||||
return sorted(found)
|
||||
|
||||
|
||||
|
|
@ -654,6 +728,10 @@ class Browser:
|
|||
return True
|
||||
if q in cap.morse.lower():
|
||||
return True
|
||||
# And by what kind of picture it is: "/sstv" and "/apt" are how
|
||||
# anybody would look for the ones worth keeping.
|
||||
if cap.image_headline and q in cap.image_headline.lower():
|
||||
return True
|
||||
return q in cap.transcript.lower()
|
||||
|
||||
@property
|
||||
|
|
@ -905,6 +983,9 @@ class Browser:
|
|||
# ceiling when there are any, and never fewer than enough to show them.
|
||||
ceiling = max(6, min(20 if calls else 16, screen // (2 if calls else 3)))
|
||||
cap = self.current
|
||||
if cap is not None and cap.image_headline:
|
||||
body = len(self._image_lines(cap))
|
||||
else:
|
||||
body = (len(self._transcript_lines()) or len(self._morse_lines())
|
||||
or (len(cap.decoded) + 2
|
||||
if cap is not None and cap.decoded else 0))
|
||||
|
|
@ -955,6 +1036,8 @@ class Browser:
|
|||
height = self._transcript_height()
|
||||
if cap is None:
|
||||
return Panel("", border_style="bright_black", height=height)
|
||||
if cap.image_headline:
|
||||
return self._image_panel(cap, height)
|
||||
morse = self._morse_lines()
|
||||
if morse:
|
||||
calls = self._callsign_lines()
|
||||
|
|
@ -1025,6 +1108,44 @@ class Browser:
|
|||
return Panel(block, title="transcript", title_align="left",
|
||||
border_style="green", padding=(1, 3), height=height)
|
||||
|
||||
def _image_lines(self, cap: Capture, pad: int = 3) -> list[str]:
|
||||
"""What the picture is and where it was written.
|
||||
|
||||
The path in full and on a line of its own, because it is the thing
|
||||
somebody came here for: the browser cannot draw a PNG in a terminal,
|
||||
so the most useful thing it can do is say exactly what to open.
|
||||
"""
|
||||
width = max(20, (self.console.size.width or 80) - 2 - 2 * pad)
|
||||
lines = [cap.image_headline[:width], ""]
|
||||
files = cap.images
|
||||
if not files:
|
||||
return lines + ["the picture file is not beside the recording"]
|
||||
for path in files:
|
||||
# Wrapped rather than cut off. Half a path is not something
|
||||
# anybody can open, and the path is the entire reason this panel
|
||||
# exists: the browser cannot draw a PNG in a terminal, so saying
|
||||
# exactly what to open is the most useful thing it can do.
|
||||
text = Text(path)
|
||||
lines += [line.plain.rstrip()
|
||||
for line in text.wrap(self.console, width)]
|
||||
return lines
|
||||
|
||||
def _image_panel(self, cap: Capture, height: int) -> Panel:
|
||||
body = Text()
|
||||
lines = self._image_lines(cap)
|
||||
body.append(lines[0] + "\n", style="bold magenta")
|
||||
for line in lines[1:]:
|
||||
body.append(line + "\n", style="bold white" if line else "")
|
||||
calls = self._callsign_lines()
|
||||
if calls:
|
||||
body.append("\n")
|
||||
body.append(calls[0], style="not bold bold cyan")
|
||||
for line in calls[1:]:
|
||||
body.append("\n")
|
||||
body.append(line, style="not bold white")
|
||||
return Panel(body, title="picture", title_align="left",
|
||||
border_style="magenta", padding=(1, 3), height=height)
|
||||
|
||||
@staticmethod
|
||||
def _why_no_transcript(cap: Capture) -> str:
|
||||
"""Say which of the several reasons applies, rather than just 'none'.
|
||||
|
|
@ -1072,6 +1193,8 @@ class Browser:
|
|||
if cap.meta.get("morse_wpm"):
|
||||
line.append(f" {float(cap.meta['morse_wpm']):.0f} WPM",
|
||||
style="yellow")
|
||||
if cap.image_kind:
|
||||
line.append(f" {len(cap.images)} file(s)", style="magenta")
|
||||
checks = cap.meta.get("data_checks") or []
|
||||
if checks:
|
||||
# A checksum that came out right is the strongest thing anyone
|
||||
|
|
@ -1124,7 +1247,8 @@ class Browser:
|
|||
base = "on grey19 " if here else ""
|
||||
cat = CATEGORY_STYLE.get(cap.category, "white")
|
||||
when = cap.when
|
||||
summary = (cap.transcript.replace("\n", " ")
|
||||
summary = (cap.image_headline
|
||||
or cap.transcript.replace("\n", " ")
|
||||
or cap.morse
|
||||
or (cap.decoded[0] if cap.decoded else "")
|
||||
or cap.classification)
|
||||
|
|
@ -1279,6 +1403,9 @@ class Browser:
|
|||
room = max(3, screen - 4 - (len(calls) + 1 if calls else 0))
|
||||
lines = self._transcript_lines(pad=4)
|
||||
title_word = "transcript"
|
||||
if not lines and cap is not None and cap.image_headline:
|
||||
lines = self._image_lines(cap, pad=4)
|
||||
title_word = "picture"
|
||||
if not lines and cap is not None and cap.morse:
|
||||
lines = self._morse_lines(pad=4)
|
||||
title_word = "Morse"
|
||||
|
|
@ -1400,7 +1527,7 @@ class Browser:
|
|||
# it all" when a long paging capture overflows it, and the key has
|
||||
# to mean what the panel says it means.
|
||||
if cap is not None and (cap.transcript or cap.morse
|
||||
or cap.decoded):
|
||||
or cap.decoded or cap.image_headline):
|
||||
self.reading = True
|
||||
self.read_top = 0
|
||||
else:
|
||||
|
|
@ -1408,7 +1535,10 @@ class Browser:
|
|||
elif key == "o":
|
||||
cap = self.current
|
||||
if cap is not None:
|
||||
self.message = str(cap.path)
|
||||
# The picture where there is one: on a capture that turned
|
||||
# out to be an image, the PNG is what anybody wants to pipe
|
||||
# into something else, not the audio it arrived as.
|
||||
self.message = cap.image_path or str(cap.path)
|
||||
return False
|
||||
elif key in _FILING_KEYS:
|
||||
self._file_into(_FILING_KEYS[key])
|
||||
|
|
@ -1572,6 +1702,8 @@ def build_parser() -> argparse.ArgumentParser:
|
|||
|
||||
def _first_line(cap: Capture) -> str:
|
||||
"""The most informative thing about a capture, in one line."""
|
||||
if cap.image_headline:
|
||||
return f"{cap.image_headline} {cap.image_path}".strip()
|
||||
if cap.transcript:
|
||||
return cap.transcript.splitlines()[0]
|
||||
if cap.morse:
|
||||
|
|
|
|||
|
|
@ -38,7 +38,7 @@ LOOKUP_URL = "https://callook.info/{call}/json"
|
|||
# rather than being kept forever with a field permanently empty -- which is
|
||||
# how a warm cache would otherwise pin every station to its grid square for
|
||||
# good after coordinates were added.
|
||||
CACHE_VERSION = 2
|
||||
CACHE_VERSION = 3
|
||||
|
||||
# The NATO alphabet, with the spellings a speech recogniser actually writes
|
||||
# down beside the ones the ITU prints. A recogniser has never heard of the
|
||||
|
|
@ -413,6 +413,7 @@ class Callsign:
|
|||
|
||||
call: str
|
||||
name: str = ""
|
||||
street: str = "" # the first line of the licensed address
|
||||
location: str = "" # town and state, as licensed
|
||||
country: str = ""
|
||||
district: str = ""
|
||||
|
|
@ -476,9 +477,24 @@ class Callsign:
|
|||
return f"not looked up ({tail})" if bits else "not looked up"
|
||||
return f"unlisted ({tail})" if bits else "no matching licence"
|
||||
|
||||
@property
|
||||
def address(self) -> str:
|
||||
"""The whole licensed address, on one line.
|
||||
|
||||
US amateur licence records are public by law and carry the street the
|
||||
licence was issued to, not merely the town. It is shown and it is
|
||||
recorded, because the alternative -- holding it and not saying so --
|
||||
is worse than either showing it or not fetching it, and
|
||||
``--no-lookup`` is there for anyone who would rather not have it.
|
||||
"""
|
||||
parts = [self.street, self.location, self.postcode]
|
||||
return ", ".join(p for p in parts if p)
|
||||
|
||||
def details(self) -> list[str]:
|
||||
"""The rest of it, for anywhere with room to show more than one line."""
|
||||
out = []
|
||||
if self.street:
|
||||
out.append(self.street)
|
||||
if self.oper_class:
|
||||
out.append(self.oper_class.title())
|
||||
if self.licence_type and self.licence_type.upper() != "PERSON":
|
||||
|
|
@ -500,7 +516,12 @@ class Callsign:
|
|||
# be wrong in title case: "PAUL PAKES COOK, III." must not become "Iii.".
|
||||
_KEEP_CAPS = frozenset({"II", "III", "IV", "V", "VI", "VII", "VIII", "JR",
|
||||
"SR", "MD", "DDS", "PHD", "HQ", "ARRL", "USA", "US",
|
||||
"LLC", "INC", "LTD", "ARC", "EMA", "RACES", "ARES"})
|
||||
"LLC", "INC", "LTD", "ARC", "EMA", "RACES", "ARES",
|
||||
# Address abbreviations, which a licence writes in
|
||||
# capitals and which title case would spoil: "PO Box"
|
||||
# is right and "Po Box" is not.
|
||||
"PO", "RR", "HC", "NE", "NW", "SE", "SW", "N", "S",
|
||||
"E", "W", "APO", "FPO"})
|
||||
|
||||
|
||||
def person_case(name: str) -> str:
|
||||
|
|
@ -516,7 +537,14 @@ def person_case(name: str) -> str:
|
|||
out = []
|
||||
for word in name.split():
|
||||
bare = word.strip(".,")
|
||||
out.append(word if bare in _KEEP_CAPS else word.title())
|
||||
if bare in _KEEP_CAPS:
|
||||
out.append(word)
|
||||
elif word[:1].isdigit():
|
||||
# A word that starts with a digit is a number or an ordinal, and
|
||||
# title case makes "5TH AVE" into "5Th Ave".
|
||||
out.append(word.lower())
|
||||
else:
|
||||
out.append(word.title())
|
||||
return " ".join(out)
|
||||
|
||||
|
||||
|
|
@ -709,6 +737,7 @@ class CallsignBook:
|
|||
previous = body.get("previous") or {}
|
||||
trustee = body.get("trustee") or {}
|
||||
entry.name = person_case(str(body.get("name") or ""))
|
||||
entry.street = person_case(str(address.get("line1") or "").strip())
|
||||
entry.location, entry.postcode = split_postcode(
|
||||
str(address.get("line2") or ""))
|
||||
entry.grid = str(location.get("gridsquare") or "")
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ from pathlib import Path
|
|||
from rich.console import Console
|
||||
from rich.prompt import Confirm
|
||||
from rich.live import Live
|
||||
from rich.markup import escape
|
||||
from rich.panel import Panel
|
||||
from rich.table import Table
|
||||
from rich.text import Text
|
||||
|
|
@ -21,7 +22,8 @@ from .bandplan import CATEGORIES, PRESETS, fmt_hz, in_category, search
|
|||
from .config import (DEFAULT_CONFIG_DIR, DEFAULT_CONFIG_PATH, ScanConfig,
|
||||
is_first_run, list_profiles, load_config, load_default,
|
||||
save_config, save_default)
|
||||
from .device import RtlSdrError, list_devices, set_driver_messages
|
||||
from .device import (RtlSdrDevice, RtlSdrError, list_devices,
|
||||
set_driver_messages)
|
||||
from .librtlsdr import load_error
|
||||
from . import settings as st
|
||||
from .ranges import (RangeError, ScanRange, build_plan, parse_range_list)
|
||||
|
|
@ -135,6 +137,20 @@ examples:
|
|||
pr.add_argument("--show", metavar="NAME", help="print one profile")
|
||||
|
||||
# -- analyse ------------------------------------------------------------
|
||||
ad = sub.add_parser("adsb",
|
||||
help="listen to aircraft on 1090 MHz")
|
||||
ad.add_argument("--seconds", type=float, default=0.0,
|
||||
help="stop after this long (default: until interrupted)")
|
||||
ad.add_argument("--rate", type=float, default=2_000_000.0,
|
||||
help="sample rate in Hz; two megasamples is the minimum")
|
||||
ad.add_argument("--gain", default="auto", help="tuner gain in dB, or auto")
|
||||
ad.add_argument("--device", type=int, default=0, help="which receiver")
|
||||
ad.add_argument("--frames", action="store_true",
|
||||
help="print every frame as it arrives, not a summary")
|
||||
ad.add_argument("--kml", nargs="?", const="aircraft.kml", default=None,
|
||||
metavar="FILE",
|
||||
help="write what was heard as a map and exit")
|
||||
|
||||
a = sub.add_parser("analyze", aliases=["analyse"],
|
||||
help="identify a signal in a recorded file")
|
||||
a.add_argument("path", help=".cf32/.cs16 IQ file or a .wav from a recording")
|
||||
|
|
@ -779,6 +795,112 @@ def cmd_profiles(args) -> int:
|
|||
return 0
|
||||
|
||||
|
||||
def cmd_adsb(args) -> int:
|
||||
"""Park the receiver on 1090 MHz and read the aircraft overhead.
|
||||
|
||||
A command of its own because ADS-B does not fit through the scanner. It
|
||||
is a megabit a second, which needs two megasamples a second of raw
|
||||
receiver output; the scan path decimates everything to a channel twelve
|
||||
and a half kilohertz wide before anything sees it, and a megabit will not
|
||||
go through that.
|
||||
"""
|
||||
from .adsb import ADSB_HZ, AircraftRegistry, SAMPLE_RATE, decode_adsb
|
||||
|
||||
if args.rate < SAMPLE_RATE:
|
||||
console.print(f"[red]ADS-B needs at least {SAMPLE_RATE/1e6:g} MS/s; "
|
||||
f"{args.rate/1e6:g} is not enough to see a bit."
|
||||
"[/red]")
|
||||
return 2
|
||||
try:
|
||||
device = RtlSdrDevice(index=args.device, sample_rate=int(args.rate),
|
||||
gain=args.gain, agc=args.gain == "auto")
|
||||
device.open()
|
||||
except RtlSdrError as exc:
|
||||
console.print(Panel(Text(str(exc)), title="[red]cannot open the receiver",
|
||||
border_style="red"))
|
||||
return 1
|
||||
|
||||
registry = AircraftRegistry()
|
||||
total = 0
|
||||
started = time.time()
|
||||
console.print(f"[grey62]listening on {ADSB_HZ/1e6:g} MHz at "
|
||||
f"{args.rate/1e6:g} MS/s — control-C to stop[/grey62]")
|
||||
try:
|
||||
device.tune(ADSB_HZ)
|
||||
block = int(args.rate) # a second at a time
|
||||
while True:
|
||||
samples = device.read_samples(block)
|
||||
if samples is None or samples.size == 0:
|
||||
break
|
||||
frames, registry = decode_adsb(samples, args.rate, registry)
|
||||
total += len(frames)
|
||||
if args.frames:
|
||||
for frame in frames:
|
||||
console.print(f"[cyan]{frame.icao}[/cyan] "
|
||||
f"{escape(frame.describe())}",
|
||||
highlight=False)
|
||||
elif frames:
|
||||
console.print(f"[grey62]{len(registry)} aircraft, "
|
||||
f"{total} frames[/grey62]", highlight=False)
|
||||
if args.seconds and time.time() - started >= args.seconds:
|
||||
break
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
device.close()
|
||||
|
||||
if not registry:
|
||||
console.print("[yellow]nothing heard. ADS-B needs an aerial cut for "
|
||||
"1090 MHz; the whip that came with the dongle will "
|
||||
"hear the airport and not much else.[/yellow]")
|
||||
return 1
|
||||
t = Table(title=f"{len(registry)} aircraft, {total} frames", box=None,
|
||||
header_style="bold")
|
||||
for column in ("ICAO", "callsign", "altitude", "position", "speed",
|
||||
"frames"):
|
||||
t.add_column(column)
|
||||
for craft in sorted(registry.aircraft.values(), key=lambda a: a.icao):
|
||||
t.add_row(craft.icao, craft.callsign or "",
|
||||
f"{craft.altitude_ft} ft" if craft.altitude_ft else "",
|
||||
(f"{craft.latitude:.4f}, {craft.longitude:.4f}"
|
||||
if craft.located else ""),
|
||||
(f"{craft.ground_speed_kt:.0f} kt {craft.track_deg:.0f}°"
|
||||
if craft.ground_speed_kt else ""),
|
||||
str(craft.messages))
|
||||
console.print(t)
|
||||
if args.kml is not None:
|
||||
written = _write_aircraft_kml(Path(args.kml).expanduser(), registry)
|
||||
console.print(f"[green]{written}[/green]" if written
|
||||
else "[red]could not write the map[/red]")
|
||||
return 0
|
||||
|
||||
|
||||
def _write_aircraft_kml(path: Path, registry) -> Path | None:
|
||||
"""Every located aircraft as a placemark, for Google Earth."""
|
||||
from xml.sax.saxutils import escape as xml_escape
|
||||
located = [a for a in registry.aircraft.values() if a.located]
|
||||
if not located:
|
||||
return None
|
||||
out = ['<?xml version="1.0" encoding="UTF-8"?>',
|
||||
'<kml xmlns="http://www.opengis.net/kml/2.2">', " <Document>",
|
||||
" <name>bandsaunter — aircraft heard</name>"]
|
||||
for craft in sorted(located, key=lambda a: a.icao):
|
||||
name = f"{craft.icao} {craft.callsign}".strip()
|
||||
out += [" <Placemark>",
|
||||
f" <name>{xml_escape(name)}</name>",
|
||||
f" <description>{xml_escape(craft.describe())}</description>",
|
||||
" <Point><coordinates>"
|
||||
f"{craft.longitude:.6f},{craft.latitude:.6f},"
|
||||
f"{craft.altitude_ft * 0.3048:.0f}</coordinates></Point>",
|
||||
" </Placemark>"]
|
||||
out += [" </Document>", "</kml>", ""]
|
||||
try:
|
||||
path.write_text("\n".join(out))
|
||||
except OSError:
|
||||
return None
|
||||
return path
|
||||
|
||||
|
||||
def cmd_analyze(args) -> int:
|
||||
import numpy as np
|
||||
from .classify import classify
|
||||
|
|
@ -811,6 +933,23 @@ def cmd_analyze(args) -> int:
|
|||
audio = np.frombuffer(raw, dtype="<i2").astype(np.float64) / 32768.0
|
||||
console.print(f"[grey62]{path.name}: {audio.size/rate:.1f} s of audio "
|
||||
f"at {rate:g} Hz[/grey62]")
|
||||
from .pictures import find_image
|
||||
picture = find_image(audio, rate, frequency=freq)
|
||||
if picture is not None and picture.ok:
|
||||
out = path.with_suffix(".png")
|
||||
picture.save(out)
|
||||
console.print(Panel(
|
||||
Text.from_markup(
|
||||
f"[bold]{escape(picture.summary())}[/bold]\n\n"
|
||||
f"[green]{escape(str(out))}[/green]"),
|
||||
title="picture", border_style="magenta"))
|
||||
for name, pixels in picture.channels.items():
|
||||
from .images import ImageDecode
|
||||
extra = ImageDecode(ok=True, pixels=pixels)
|
||||
where = path.with_name(path.stem + f"_{name}.png")
|
||||
extra.save(where)
|
||||
console.print(f"[grey62]{where}[/grey62]", highlight=False)
|
||||
return 0
|
||||
m = decode_morse(audio, rate)
|
||||
if m.is_morse or args.morse:
|
||||
console.print(Panel(
|
||||
|
|
@ -922,6 +1061,7 @@ def main(argv=None) -> int:
|
|||
"scan": cmd_scan, "bands": cmd_bands, "devices": cmd_devices,
|
||||
"config": cmd_config, "transcribe": cmd_transcribe,
|
||||
"profiles": cmd_profiles, "analyze": cmd_analyze, "analyse": cmd_analyze,
|
||||
"adsb": cmd_adsb,
|
||||
}
|
||||
try:
|
||||
return handlers[args.command](args)
|
||||
|
|
|
|||
|
|
@ -109,6 +109,7 @@ class ScanConfig:
|
|||
classify: bool = True
|
||||
decode_morse: bool = True
|
||||
decode_data: bool = True # read packets out of data signals
|
||||
decode_images: bool = True # SSTV, weather satellites and shortwave fax
|
||||
|
||||
# -- one file per frequency ------------------------------------------
|
||||
combine_by_frequency: bool = False
|
||||
|
|
|
|||
|
|
@ -45,6 +45,9 @@ from dataclasses import dataclass, field
|
|||
|
||||
import numpy as np
|
||||
|
||||
from .ism import decode_ism
|
||||
from .payload import MIN_TEXT, hexdump, read_text
|
||||
|
||||
__all__ = ["decode_data", "DataDecode", "Burst", "PulseTrain", "Encoding",
|
||||
"slice_ook", "slice_fsk", "split_bursts", "decode_train",
|
||||
"decode_four_level", "level_count", "nrz_bits", "bits_to_hex",
|
||||
|
|
@ -1026,6 +1029,7 @@ class DataDecode:
|
|||
checks: list[str] = field(default_factory=list)
|
||||
bursts: list[Burst] = field(default_factory=list)
|
||||
messages: list[str] = field(default_factory=list) # decoded text, if any
|
||||
readings: list[str] = field(default_factory=list) # the payload, readably
|
||||
confidence: float = 0.0
|
||||
note: str = ""
|
||||
|
||||
|
|
@ -1059,9 +1063,11 @@ class DataDecode:
|
|||
more = len(self.messages) - limit
|
||||
if more > 0:
|
||||
out.append(f"... and {more} more")
|
||||
if not self.messages and self.bits:
|
||||
out.append(self.hex[:width])
|
||||
out.append(self.bits[:width])
|
||||
# The payload itself, laid out to be read. Underneath whatever was
|
||||
# decoded rather than instead of it: a packet whose fields have been
|
||||
# named still has bytes in it, and the bytes are what somebody works
|
||||
# from when the naming turns out to be wrong.
|
||||
out += self.readings
|
||||
if self.checks:
|
||||
out.append("checks: " + ", ".join(self.checks))
|
||||
return out
|
||||
|
|
@ -1088,14 +1094,51 @@ def _score(decode: DataDecode, fit: float) -> float:
|
|||
return float(max(0.0, min(0.99, score)))
|
||||
|
||||
|
||||
def _interpret(out: DataDecode, frequency: float) -> DataDecode:
|
||||
"""Turn a decoded packet into something a person can read.
|
||||
|
||||
Three tries, and the first that succeeds is what gets said. A meter or a
|
||||
weather sensor has a checksum of its own, so where one of those matches
|
||||
there is nothing left to guess. Failing that, a packet with text in it
|
||||
has the text pulled out. And underneath either, always, the bytes
|
||||
themselves in groups with their printable characters beside them --
|
||||
because "4A 3F 1B 22" on one long line is a true answer to "what did that
|
||||
say" and not a useful one.
|
||||
"""
|
||||
if not out.ok or not out.bits:
|
||||
return out
|
||||
# Trusted means the packet is known good: it arrived identically several
|
||||
# times, or its own checksum came out right. It lowers the bar on the
|
||||
# text search, which is otherwise set for a single unrepeated burst.
|
||||
trusted = bool(out.checks) or (out.repeats >= 2 and out.agreement >= 0.9)
|
||||
if not out.messages:
|
||||
reading = decode_ism(out.bits, frequency)
|
||||
if reading is not None:
|
||||
out.protocol = f"{reading.device} ({reading.kind})"
|
||||
out.messages = [reading.describe()]
|
||||
out.checks = list(dict.fromkeys(out.checks + reading.checks))
|
||||
out.confidence = max(out.confidence, 0.9)
|
||||
else:
|
||||
text = read_text(out.bits,
|
||||
minimum=MIN_TEXT - 1 if trusted else MIN_TEXT)
|
||||
if text is not None:
|
||||
out.messages = [f'"{text.text}"']
|
||||
out.protocol = out.protocol or f"text, {text.how}"
|
||||
out.readings = hexdump(out.bits)
|
||||
return out
|
||||
|
||||
|
||||
def decode_data(x: np.ndarray, sample_rate: float, family: str = "",
|
||||
baud_hint: float = 0.0) -> DataDecode:
|
||||
baud_hint: float = 0.0,
|
||||
frequency: float = 0.0) -> DataDecode:
|
||||
"""Recover whatever data is in ``x``.
|
||||
|
||||
``x`` is complex baseband centred on the signal. ``family`` is the
|
||||
classifier's opinion of what it is, used only to decide which way to
|
||||
slice it first; both ways are tried regardless, because the classifier
|
||||
is working from statistics and this is working from the bits.
|
||||
``frequency`` is the dial reading, which decides which of the named ISM
|
||||
formats is tried first and nothing else.
|
||||
"""
|
||||
x = np.asarray(x)
|
||||
if x.size < 256:
|
||||
|
|
@ -1105,7 +1148,7 @@ def decode_data(x: np.ndarray, sample_rate: float, family: str = "",
|
|||
# where one of them matches there is nothing to guess at.
|
||||
named = _try_protocols(x, sample_rate, family, baud_hint)
|
||||
if named is not None and named.ok:
|
||||
return named
|
||||
return _interpret(named, frequency)
|
||||
|
||||
# Before anything two-level. A four-level signal sliced down the middle
|
||||
# yields runs, and runs yield bits, and the bits mean nothing -- so where
|
||||
|
|
@ -1113,7 +1156,7 @@ def decode_data(x: np.ndarray, sample_rate: float, family: str = "",
|
|||
# even when the symbols could not be framed into anything named.
|
||||
multi = decode_four_level(x, sample_rate, baud_hint)
|
||||
if multi is not None:
|
||||
return multi
|
||||
return _interpret(multi, frequency)
|
||||
|
||||
order = ["fsk", "ook"] if family in ("fsk", "psk", "digital", "control") \
|
||||
else ["ook", "fsk"]
|
||||
|
|
@ -1130,9 +1173,9 @@ def decode_data(x: np.ndarray, sample_rate: float, family: str = "",
|
|||
if best is not None and best.confidence > 0.8:
|
||||
break
|
||||
if best is not None:
|
||||
return best
|
||||
return _interpret(best, frequency)
|
||||
if named is not None:
|
||||
return named
|
||||
return _interpret(named, frequency)
|
||||
return DataDecode(note="nothing in it decoded as data")
|
||||
|
||||
|
||||
|
|
|
|||
199
bandsaunter/fax.py
Normal file
199
bandsaunter/fax.py
Normal file
|
|
@ -0,0 +1,199 @@
|
|||
"""Radiofax: the weather charts the shortwave stations have always sent.
|
||||
|
||||
Northwood, Hamburg, Kodiak and a dozen others send synoptic charts, ice maps
|
||||
and wave forecasts on shortwave all day, and have done since long before any
|
||||
of it was digital. A ship with an SSB receiver and a printer got a paper
|
||||
chart; the same signal into a sound card gets a PNG.
|
||||
|
||||
The signal is an FM subcarrier inside an SSB channel: 1500 Hz is black,
|
||||
2300 Hz is white. A transmission opens with a start tone, then a phasing
|
||||
signal -- twenty or so lines that are black except for a short white pulse at
|
||||
the start of each -- and then the chart, at 120 lines a minute for almost
|
||||
everybody. The phasing signal is there so a mechanical drum could be got
|
||||
into step, and it is exactly as useful here: it says where a line begins and
|
||||
how long one is, which is the whole of what a decoder needs.
|
||||
|
||||
The width comes from the index of cooperation, which is a drum circumference
|
||||
divided by pi and has survived into a world with no drums. IOC 576 means
|
||||
pi times 576, or 1809 pixels across.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
import numpy as np
|
||||
from scipy import ndimage as _ndi
|
||||
|
||||
from .images import ImageDecode, instantaneous_frequency, resample_to
|
||||
|
||||
__all__ = ["decode_fax", "LINE_RATES", "IOC_STANDARD", "find_phasing",
|
||||
"BLACK_HZ", "WHITE_HZ"]
|
||||
|
||||
BLACK_HZ = 1500.0
|
||||
WHITE_HZ = 2300.0
|
||||
|
||||
# Lines per minute. 120 is what almost every station uses; the others are in
|
||||
# the standard and turn up on a few.
|
||||
LINE_RATES: tuple[float, ...] = (120.0, 60.0, 90.0, 180.0, 240.0)
|
||||
|
||||
IOC_STANDARD = 576 # pi * 576 = 1809 pixels across
|
||||
IOC_SMALL = 288
|
||||
|
||||
# The phasing pulse is 5% of a line wide by the standard. Both ends of this
|
||||
# are generous, because what arrives has been through a receiver's filters.
|
||||
PULSE_SHARE = (0.02, 0.12)
|
||||
|
||||
# How strongly the phasing has to show before a chart is believed. A phasing
|
||||
# line is black apart from one bright pulse, so folding a few seconds of it
|
||||
# at the line rate gives a peak several times the rest of the line; SSB voice
|
||||
# and static fold flat. Measured over speech, noise and steady tones, none
|
||||
# reached 2.0; a fax at 6 dB is above 6.
|
||||
MIN_PHASING_PEAK = 3.5
|
||||
|
||||
# Folded at any rate at all, noise converges on mid-grey. A phasing line is
|
||||
# black but for its pulse, so these two are what actually separate a
|
||||
# transmission from a band with nothing on it.
|
||||
MAX_PHASING_FLOOR = 0.35
|
||||
MIN_PHASING_PULSE = 0.60
|
||||
|
||||
MIN_LINES = 8
|
||||
|
||||
|
||||
@dataclass
|
||||
class Phasing:
|
||||
"""What the phasing signal said about the transmission that follows."""
|
||||
|
||||
lpm: float
|
||||
period: int # samples per line
|
||||
phase: int # sample the first line starts at
|
||||
peak: float # how strongly it showed
|
||||
last: int # sample the phasing signal ends at
|
||||
|
||||
|
||||
def _levels(freq: np.ndarray) -> np.ndarray:
|
||||
"""Frequency as brightness, 0 for black and 1 for white."""
|
||||
return np.clip((freq - BLACK_HZ) / (WHITE_HZ - BLACK_HZ), 0.0, 1.0)
|
||||
|
||||
|
||||
def find_phasing(level: np.ndarray, rate: float) -> Phasing | None:
|
||||
"""Find the phasing signal, and with it the line rate and where lines start.
|
||||
|
||||
Every candidate line rate is tried and the signal folded at each: the
|
||||
phasing pulse lands on the same place in every line, so folding stacks it
|
||||
up while everything else averages out.
|
||||
|
||||
What makes this safe to run over any shortwave capture is that the test
|
||||
is on absolute brightness rather than on contrast. Folded at any rate,
|
||||
noise converges on mid-grey and stays there; a phasing signal converges
|
||||
on black with one white pulse standing out of it, and nothing else on the
|
||||
air does that.
|
||||
"""
|
||||
best: Phasing | None = None
|
||||
for lpm in LINE_RATES:
|
||||
period = int(round(60.0 / lpm * rate))
|
||||
if period < 16 or level.size < 4 * period:
|
||||
continue
|
||||
lines = min(40, level.size // period)
|
||||
folded = level[:lines * period].reshape(lines, period).mean(axis=0)
|
||||
floor = float(np.median(folded))
|
||||
top = float(folded.max())
|
||||
# Black everywhere but the pulse, and the pulse actually white.
|
||||
if floor > MAX_PHASING_FLOOR or top < MIN_PHASING_PULSE:
|
||||
continue
|
||||
above = folded > floor + 0.5 * (top - floor)
|
||||
share = float(above.mean())
|
||||
if not (PULSE_SHARE[0] <= share <= PULSE_SHARE[1]):
|
||||
continue
|
||||
# One pulse, not a scatter of them.
|
||||
edges = np.flatnonzero(np.diff(above.astype(np.int8)))
|
||||
runs = (edges.size + (1 if above[0] else 0) + 1) // 2
|
||||
if runs > 2:
|
||||
continue
|
||||
spread = float(np.median(np.abs(folded - floor))) or 1e-9
|
||||
peak = (top - floor) / spread
|
||||
# The line starts where the pulse does, not at its middle.
|
||||
lit = np.flatnonzero(above)
|
||||
phase = int(lit[0]) if lit.size else int(np.argmax(folded))
|
||||
if best is None or peak > best.peak:
|
||||
best = Phasing(lpm=lpm, period=period, phase=phase, peak=peak,
|
||||
last=0)
|
||||
if best is None or best.peak < MIN_PHASING_PEAK:
|
||||
return None
|
||||
return _end_of_phasing(level, best)
|
||||
|
||||
|
||||
def _end_of_phasing(level: np.ndarray, found: Phasing) -> Phasing:
|
||||
"""Where the phasing stops and the chart starts.
|
||||
|
||||
The last phasing line, not the first line that is not one: a transmission
|
||||
opens with a start tone that is a black-and-white alternation, and its
|
||||
average is mid-grey, so "the first line that is not black" is the very
|
||||
first line and the chart comes out as thirty seconds of tone.
|
||||
"""
|
||||
lines = (level.size - found.phase) // found.period
|
||||
phasing = []
|
||||
for line in range(lines):
|
||||
start = found.phase + line * found.period
|
||||
body = level[start + int(0.15 * found.period):start + found.period]
|
||||
phasing.append(bool(body.size and float(np.mean(body)) < 0.25))
|
||||
# The end of the longest run of phasing lines.
|
||||
best_end, best_len, run = 0, 0, 0
|
||||
for line, is_phasing in enumerate(phasing):
|
||||
run = run + 1 if is_phasing else 0
|
||||
if run > best_len:
|
||||
best_len, best_end = run, line + 1
|
||||
if best_len < 3:
|
||||
found.last = found.phase
|
||||
else:
|
||||
found.last = found.phase + best_end * found.period
|
||||
return found
|
||||
|
||||
|
||||
def decode_fax(audio: np.ndarray, rate: float, ioc: int = IOC_STANDARD,
|
||||
) -> ImageDecode | None:
|
||||
"""Decode a weather fax from the audio of an SSB receiver.
|
||||
|
||||
Returns None when there is no phasing signal, which is the usual answer.
|
||||
The phasing is what makes this safe to run over any shortwave capture: it
|
||||
is twenty seconds of a pattern nothing else sends, and without it a fax
|
||||
decoder draws static beautifully.
|
||||
"""
|
||||
audio = np.asarray(audio, dtype=np.float64).ravel()
|
||||
if audio.size < int(rate * 8.0):
|
||||
return None
|
||||
freq = instantaneous_frequency(audio, rate, low=1200.0, high=2600.0)
|
||||
if freq.size == 0:
|
||||
return None
|
||||
# A fax line is half a second and a pixel is a third of a millisecond, so
|
||||
# a filter a tenth of that wide costs no detail and takes the scatter off
|
||||
# a weak signal.
|
||||
window = int(round(0.0001 * rate)) | 1
|
||||
if window >= 3:
|
||||
freq = _ndi.median_filter(freq, size=window, mode="nearest")
|
||||
level = _levels(freq)
|
||||
|
||||
found = find_phasing(level, rate)
|
||||
if found is None:
|
||||
return None
|
||||
|
||||
width = int(round(np.pi * ioc))
|
||||
start = found.last
|
||||
lines = max(0, (level.size - start) // found.period)
|
||||
if lines < MIN_LINES:
|
||||
return ImageDecode(kind="HF fax",
|
||||
mode=f"{found.lpm:.0f} lpm IOC {ioc}",
|
||||
note=f"phasing signal, but only {lines} line(s) "
|
||||
"of chart followed it")
|
||||
picture = np.zeros((lines, width), dtype=np.uint8)
|
||||
for line in range(lines):
|
||||
at = start + line * found.period
|
||||
row = resample_to(level[at:at + found.period], width)
|
||||
picture[line] = np.clip(row * 255.0, 0, 255).astype(np.uint8)
|
||||
|
||||
return ImageDecode(
|
||||
ok=True, kind="HF fax", mode=f"{found.lpm:.0f} lpm IOC {ioc}",
|
||||
width=width, height=lines, pixels=picture, complete=True,
|
||||
seconds=lines * 60.0 / found.lpm,
|
||||
confidence=float(min(0.95, 0.45 + found.peak / 25.0)),
|
||||
note="")
|
||||
211
bandsaunter/images.py
Normal file
211
bandsaunter/images.py
Normal file
|
|
@ -0,0 +1,211 @@
|
|||
"""Pictures off the air, and the file they get written to.
|
||||
|
||||
Three of the things a receiver can hear are images rather than sound or
|
||||
bits: weather satellites in the 137 MHz band sending APT, amateurs sending
|
||||
SSTV on 14.230 MHz and 144.5 MHz, and the marine weather fax stations that
|
||||
have been sending charts on shortwave since before any of this was digital.
|
||||
All three are analogue -- brightness is a frequency, and a picture is a very
|
||||
long single tone that wobbles -- so all three come out of the same place:
|
||||
the audio the scanner already records.
|
||||
|
||||
This module is what they have in common. The decoders themselves are in
|
||||
:mod:`sstv`, :mod:`apt` and :mod:`fax`.
|
||||
|
||||
PNG is written here rather than pulled in from Pillow. A scanner that
|
||||
cannot start because an imaging library is missing is worse than one that
|
||||
cannot draw, the format's own container is a dozen lines of zlib and
|
||||
struct, and this way the pictures work on a machine with nothing installed
|
||||
but numpy.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import struct
|
||||
import zlib
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
from scipy import signal as sps
|
||||
|
||||
__all__ = ["ImageDecode", "write_png", "instantaneous_frequency",
|
||||
"tone_amplitude", "resample_to", "PNG_SIGNATURE"]
|
||||
|
||||
PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# What a decoder gives back
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class ImageDecode:
|
||||
"""One picture recovered from a capture."""
|
||||
|
||||
ok: bool = False
|
||||
kind: str = "" # APT / SSTV / HF fax
|
||||
mode: str = "" # NOAA APT, Martin M1, 120 lpm IOC 576, ...
|
||||
width: int = 0
|
||||
height: int = 0
|
||||
pixels: np.ndarray | None = field(default=None, repr=False)
|
||||
path: str = "" # where it was written, once it has been
|
||||
complete: bool = True # False when the capture ran out partway
|
||||
seconds: float = 0.0 # of signal that went into it
|
||||
confidence: float = 0.0
|
||||
channels: dict[str, str] = field(default_factory=dict) # extra images
|
||||
note: str = ""
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return f"{self.kind} ({self.mode})" if self.mode else self.kind
|
||||
|
||||
def summary(self) -> str:
|
||||
if not self.ok:
|
||||
return self.note or "no image decoded"
|
||||
size = f"{self.width}x{self.height}"
|
||||
bits = [self.name, size]
|
||||
if not self.complete:
|
||||
bits.append("partial")
|
||||
if self.path:
|
||||
bits.append(Path(self.path).name)
|
||||
return " ".join(bits)
|
||||
|
||||
def save(self, path) -> str:
|
||||
"""Write the picture, and remember where it went."""
|
||||
if self.pixels is None:
|
||||
return ""
|
||||
written = write_png(path, self.pixels)
|
||||
self.path = str(written)
|
||||
return self.path
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# PNG
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _chunk(tag: bytes, data: bytes) -> bytes:
|
||||
return (struct.pack(">I", len(data)) + tag + data
|
||||
+ struct.pack(">I", zlib.crc32(tag + data) & 0xFFFFFFFF))
|
||||
|
||||
|
||||
def write_png(path, pixels: np.ndarray) -> Path:
|
||||
"""Write a greyscale or RGB array as a PNG.
|
||||
|
||||
``pixels`` is ``(height, width)`` for grey or ``(height, width, 3)`` for
|
||||
colour, in any numeric type; it is clipped into a byte rather than scaled,
|
||||
because a decoder that has already decided what is black and what is white
|
||||
should not have that decision quietly changed here.
|
||||
"""
|
||||
data = np.asarray(pixels)
|
||||
if data.ndim == 2:
|
||||
colour, depth = 0, 1
|
||||
elif data.ndim == 3 and data.shape[2] == 3:
|
||||
colour, depth = 2, 3
|
||||
else:
|
||||
raise ValueError(f"cannot write an array of shape {data.shape} as PNG")
|
||||
if data.dtype != np.uint8:
|
||||
data = np.clip(data, 0, 255).astype(np.uint8)
|
||||
height, width = data.shape[0], data.shape[1]
|
||||
|
||||
raw = bytearray()
|
||||
flat = data.reshape(height, width * depth)
|
||||
for row in flat:
|
||||
raw.append(0) # filter: none
|
||||
raw.extend(row.tobytes())
|
||||
|
||||
out = bytearray(PNG_SIGNATURE)
|
||||
out += _chunk(b"IHDR", struct.pack(">IIBBBBB", width, height, 8, colour,
|
||||
0, 0, 0))
|
||||
out += _chunk(b"IDAT", zlib.compress(bytes(raw), 6))
|
||||
out += _chunk(b"IEND", b"")
|
||||
|
||||
path = Path(path)
|
||||
tmp = path.with_suffix(path.suffix + ".tmp")
|
||||
tmp.write_bytes(bytes(out))
|
||||
tmp.replace(path) # never a half-written picture
|
||||
return path
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The measurements every one of these decoders needs
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def instantaneous_frequency(audio: np.ndarray, rate: float,
|
||||
low: float = 800.0,
|
||||
high: float = 2800.0) -> np.ndarray:
|
||||
"""The frequency of an audio tone, sample by sample, in hertz.
|
||||
|
||||
All three picture modes encode brightness as a frequency, so this is the
|
||||
whole of the demodulation: band-pass to the range the tone lives in, take
|
||||
the analytic signal, and differentiate its phase.
|
||||
"""
|
||||
audio = np.asarray(audio, dtype=np.float64).ravel()
|
||||
if audio.size < 64:
|
||||
return np.zeros(0)
|
||||
nyquist = rate / 2.0
|
||||
lo = max(1e-4, low / nyquist)
|
||||
hi = min(0.999, high / nyquist)
|
||||
if hi <= lo:
|
||||
return np.zeros(0)
|
||||
taps = sps.firwin(129, [lo, hi], pass_zero=False)
|
||||
filtered = sps.lfilter(taps, [1.0], audio)
|
||||
# Undo the filter's delay. A linear-phase FIR of 129 taps holds its
|
||||
# output back by 64 samples, which is four milliseconds at 16 kHz -- and
|
||||
# four milliseconds is most of an SSTV sync pulse and a fifteen-pixel
|
||||
# shift across a Martin M2 line. It cancels only if every time in the
|
||||
# decode is measured from this same trace, and the line timings are not:
|
||||
# they come from the mode specification, in real seconds.
|
||||
delay = 64
|
||||
filtered = np.concatenate((filtered[delay:],
|
||||
np.zeros(delay, dtype=filtered.dtype)))
|
||||
analytic = sps.hilbert(filtered)
|
||||
phase = np.unwrap(np.angle(analytic))
|
||||
freq = np.diff(phase) * rate / (2.0 * np.pi)
|
||||
# The first samples are the filter filling up and mean nothing.
|
||||
freq[:65] = freq[65] if freq.size > 65 else 0.0
|
||||
return freq
|
||||
|
||||
|
||||
def tone_amplitude(audio: np.ndarray, rate: float, centre: float,
|
||||
width: float = 1200.0) -> np.ndarray:
|
||||
"""The envelope of an amplitude-modulated subcarrier.
|
||||
|
||||
APT is a 2400 Hz tone whose loudness is the brightness of the picture, so
|
||||
for that one the frequency is fixed and it is the envelope that carries
|
||||
everything.
|
||||
"""
|
||||
audio = np.asarray(audio, dtype=np.float64).ravel()
|
||||
if audio.size < 64:
|
||||
return np.zeros(0)
|
||||
nyquist = rate / 2.0
|
||||
lo = max(1e-4, (centre - width) / nyquist)
|
||||
hi = min(0.999, (centre + width) / nyquist)
|
||||
if hi <= lo:
|
||||
return np.abs(sps.hilbert(audio))
|
||||
taps = sps.firwin(129, [lo, hi], pass_zero=False)
|
||||
return np.abs(sps.hilbert(sps.lfilter(taps, [1.0], audio)))
|
||||
|
||||
|
||||
def resample_to(values: np.ndarray, count: int) -> np.ndarray:
|
||||
"""Stretch or squash a run of samples to exactly ``count`` of them.
|
||||
|
||||
Linear rather than a proper resampler: a picture line is a few hundred
|
||||
samples wide and the difference is invisible, while a polyphase filter
|
||||
per line over a fifteen-minute pass is not.
|
||||
"""
|
||||
values = np.asarray(values, dtype=np.float64)
|
||||
if values.size == 0 or count <= 0:
|
||||
return np.zeros(max(0, count))
|
||||
if values.size == count:
|
||||
return values
|
||||
source = np.linspace(0.0, 1.0, values.size)
|
||||
target = np.linspace(0.0, 1.0, count)
|
||||
return np.interp(target, source, values)
|
||||
|
||||
|
||||
def stretch(values: np.ndarray, low: float, high: float) -> np.ndarray:
|
||||
"""Map a frequency range onto 0-255, clipped at both ends."""
|
||||
if high <= low:
|
||||
return np.zeros_like(values, dtype=np.uint8)
|
||||
scaled = (np.asarray(values, dtype=np.float64) - low) / (high - low)
|
||||
return np.clip(scaled * 255.0, 0, 255).astype(np.uint8)
|
||||
240
bandsaunter/ism.py
Normal file
240
bandsaunter/ism.py
Normal file
|
|
@ -0,0 +1,240 @@
|
|||
"""Two things worth naming on the ISM bands: utility meters and weather sensors.
|
||||
|
||||
Both are on the air constantly and both come out of the generic decoder as
|
||||
hexadecimal, which is a true and useless answer to "what did that say".
|
||||
|
||||
**Utility meters.** The Itron ERT modules fitted to electricity, gas and
|
||||
water meters across North America broadcast their reading every thirty
|
||||
seconds or so on 902-928 MHz, in the clear, so that a van can drive past and
|
||||
read a street. The message says which meter, what kind, what the register
|
||||
reads, and whether the tamper switches have been tripped.
|
||||
|
||||
**AcuRite weather sensors.** The 433.92 MHz outdoor sensors sold with every
|
||||
consumer weather station send temperature, humidity, battery state and a
|
||||
channel letter every sixteen seconds.
|
||||
|
||||
Neither is guessed at. A meter message carries a sixteen-bit BCH checksum
|
||||
and a sensor message carries a checksum and four parity bits, and nothing is
|
||||
reported here that has not satisfied them -- which is what makes it safe to
|
||||
try these on every burst that a 433 or 900 MHz capture yields.
|
||||
|
||||
Both formats are implemented from their published descriptions and are
|
||||
checked here against frames built from the same descriptions. That proves
|
||||
the framing and the arithmetic; it is not the same as having held a meter.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
__all__ = ["decode_ism", "IsmReading", "decode_scm", "decode_acurite",
|
||||
"scm_frame", "acurite_frame", "SCM_PREAMBLE", "ERT_TYPES"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Itron ERT / SCM
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Twenty-one bits, sent before every standard consumption message.
|
||||
SCM_PREAMBLE = "111110010101001100000"
|
||||
SCM_BITS = 96
|
||||
|
||||
# What the four-bit type field means. Only the common ones; anything else is
|
||||
# reported by number rather than guessed at.
|
||||
ERT_TYPES = {
|
||||
4: "electricity", 5: "gas", 7: "electricity", 8: "electricity",
|
||||
11: "water", 12: "gas", 13: "water",
|
||||
}
|
||||
|
||||
|
||||
def _crc16(bits: str, poly: int = 0x6F63, init: int = 0) -> int:
|
||||
"""The BCH check an SCM message carries, run bit by bit.
|
||||
|
||||
Bit by bit rather than byte by byte because the message is not byte
|
||||
aligned: the preamble is twenty-one bits long, so everything after it
|
||||
sits five bits into a byte and packing it first would check the wrong
|
||||
eighty bits.
|
||||
"""
|
||||
crc = init
|
||||
for bit in bits:
|
||||
top = (crc >> 15) & 1
|
||||
crc = (crc << 1) & 0xFFFF
|
||||
if top ^ (1 if bit == "1" else 0):
|
||||
crc ^= poly
|
||||
return crc
|
||||
|
||||
|
||||
@dataclass
|
||||
class IsmReading:
|
||||
"""One meter or sensor message that satisfied its own checksum."""
|
||||
|
||||
kind: str = "" # SCM / AcuRite
|
||||
device: str = "" # what sort of thing it is
|
||||
identifier: str = "" # the number on the meter, or the sensor id
|
||||
fields: list[tuple[str, str]] = field(default_factory=list)
|
||||
bits: str = ""
|
||||
checks: list[str] = field(default_factory=list)
|
||||
|
||||
def describe(self) -> str:
|
||||
head = f"{self.device} {self.identifier}".strip()
|
||||
rest = " ".join(f"{name} {value}" for name, value in self.fields)
|
||||
return f"{head} {rest}".strip()
|
||||
|
||||
|
||||
def decode_scm(bits: str) -> IsmReading | None:
|
||||
"""Read one Itron ERT standard consumption message.
|
||||
|
||||
The layout after the preamble, in bits: two of the meter number, one
|
||||
reserved, two physical tamper, four for what kind of meter it is, two
|
||||
encoder tamper, twenty-four of consumption, twenty-four more of the meter
|
||||
number, and sixteen of checksum.
|
||||
"""
|
||||
at = bits.find(SCM_PREAMBLE)
|
||||
if at < 0 or len(bits) - at < SCM_BITS:
|
||||
return None
|
||||
frame = bits[at:at + SCM_BITS]
|
||||
# The check covers everything from bit 16: the tail of the preamble, the
|
||||
# message, and the checksum itself, and comes out at zero.
|
||||
if _crc16(frame[16:]) != 0:
|
||||
return None
|
||||
|
||||
body = frame[len(SCM_PREAMBLE):]
|
||||
id_high = int(body[0:2], 2)
|
||||
tamper_physical = int(body[3:5], 2)
|
||||
ert_type = int(body[5:9], 2)
|
||||
tamper_encoder = int(body[9:11], 2)
|
||||
consumption = int(body[11:35], 2)
|
||||
id_low = int(body[35:59], 2)
|
||||
meter = (id_high << 24) | id_low
|
||||
|
||||
kind = ERT_TYPES.get(ert_type, f"type {ert_type}")
|
||||
fields = [("reading", str(consumption))]
|
||||
if tamper_physical:
|
||||
fields.append(("physical tamper", str(tamper_physical)))
|
||||
if tamper_encoder:
|
||||
fields.append(("encoder tamper", str(tamper_encoder)))
|
||||
return IsmReading(kind="SCM", device=f"{kind} meter",
|
||||
identifier=str(meter), fields=fields, bits=frame,
|
||||
checks=["BCH(255,239)"])
|
||||
|
||||
|
||||
def scm_frame(meter: int, consumption: int, ert_type: int = 4,
|
||||
tamper_physical: int = 0, tamper_encoder: int = 0) -> str:
|
||||
"""Build one SCM message, checksum included.
|
||||
|
||||
Kept beside the decoder so the two cannot drift apart, and so a test can
|
||||
put a meter reading in and take the same one out.
|
||||
"""
|
||||
if not 0 <= meter < (1 << 26):
|
||||
raise ValueError("a meter number is 26 bits")
|
||||
if not 0 <= consumption < (1 << 24):
|
||||
raise ValueError("a consumption reading is 24 bits")
|
||||
body = (format(meter >> 24, "02b") + "0"
|
||||
+ format(tamper_physical, "02b") + format(ert_type, "04b")
|
||||
+ format(tamper_encoder, "02b")
|
||||
+ format(consumption, "024b")
|
||||
+ format(meter & 0xFFFFFF, "024b"))
|
||||
head = SCM_PREAMBLE + body
|
||||
return head + format(_crc16(head[16:]), "016b")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AcuRite
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
ACURITE_BYTES = 7
|
||||
ACURITE_CHANNELS = "ABCD"
|
||||
|
||||
|
||||
def _parity(value: int) -> int:
|
||||
value ^= value >> 4
|
||||
value ^= value >> 2
|
||||
value ^= value >> 1
|
||||
return value & 1
|
||||
|
||||
|
||||
def decode_acurite(bits: str) -> IsmReading | None:
|
||||
"""Read one AcuRite 592TXR / Tower outdoor sensor message.
|
||||
|
||||
Seven bytes: fourteen bits of sensor number with the channel above them,
|
||||
a status byte, humidity, temperature in tenths of a degree offset by a
|
||||
hundred, and a checksum that is the sum of the six bytes before it. The
|
||||
four middle bytes each carry odd parity in their top bit, which is what
|
||||
makes a seven-byte message safe to accept on a band this crowded.
|
||||
"""
|
||||
need = ACURITE_BYTES * 8
|
||||
if len(bits) < need:
|
||||
return None
|
||||
# Every offset, because what reaches here has a sync pattern of some
|
||||
# length in front of it and the message does not begin on a byte
|
||||
# boundary of the recovered bits. The checksum and the four parity bits
|
||||
# are what make that affordable.
|
||||
for at in range(len(bits) - need + 1):
|
||||
data = [int(bits[at + i * 8:at + (i + 1) * 8], 2)
|
||||
for i in range(ACURITE_BYTES)]
|
||||
if (sum(data[:6]) & 0xFF) == data[6] and any(data) and \
|
||||
all(_parity(byte) == 1 for byte in data[2:6]):
|
||||
break
|
||||
else:
|
||||
return None
|
||||
bits = bits[at:at + need]
|
||||
|
||||
channel = ACURITE_CHANNELS[(data[0] >> 6) & 0x03]
|
||||
sensor = ((data[0] & 0x3F) << 8) | data[1]
|
||||
humidity = data[3] & 0x7F
|
||||
raw = ((data[4] & 0x0F) << 7) | (data[5] & 0x7F)
|
||||
celsius = raw / 10.0 - 100.0
|
||||
if not (-40.0 <= celsius <= 70.0) or humidity > 100:
|
||||
return None # outside what the sensor can report
|
||||
|
||||
fields = [("temperature", f"{celsius:.1f} C"),
|
||||
("humidity", f"{humidity}%"),
|
||||
("channel", channel)]
|
||||
if data[2] & 0x40:
|
||||
fields.append(("battery", "low"))
|
||||
return IsmReading(kind="AcuRite", device="AcuRite sensor",
|
||||
identifier=f"{sensor:04X}", fields=fields,
|
||||
bits=bits,
|
||||
checks=["checksum-8", "parity"])
|
||||
|
||||
|
||||
def acurite_frame(sensor: int, celsius: float, humidity: int,
|
||||
channel: str = "A", battery_low: bool = False) -> str:
|
||||
"""Build one AcuRite sensor message, checksum and parity included."""
|
||||
raw = int(round((celsius + 100.0) * 10.0))
|
||||
data = [((ACURITE_CHANNELS.index(channel) & 3) << 6) | ((sensor >> 8) & 0x3F),
|
||||
sensor & 0xFF,
|
||||
0x04 | (0x40 if battery_low else 0x00),
|
||||
humidity & 0x7F,
|
||||
(raw >> 7) & 0x0F,
|
||||
raw & 0x7F]
|
||||
for i in range(2, 6):
|
||||
if _parity(data[i]) != 1:
|
||||
data[i] |= 0x80
|
||||
data.append(sum(data[:6]) & 0xFF)
|
||||
return "".join(format(byte, "08b") for byte in data)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Both together
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def decode_ism(bits: str, frequency: float = 0.0) -> IsmReading | None:
|
||||
"""Whichever of these a run of bits turns out to be, or None.
|
||||
|
||||
``frequency`` narrows the search where it is known -- meters are a 900 MHz
|
||||
thing and these sensors a 433 MHz one -- but neither decoder will accept a
|
||||
message that fails its own checksum, so getting the band wrong costs
|
||||
nothing but the work.
|
||||
"""
|
||||
order = [decode_scm, decode_acurite]
|
||||
if 420e6 <= frequency <= 450e6:
|
||||
order.reverse()
|
||||
for decoder in order:
|
||||
try:
|
||||
got = decoder(bits)
|
||||
except (ValueError, IndexError):
|
||||
continue
|
||||
if got is not None:
|
||||
return got
|
||||
return None
|
||||
|
|
@ -119,6 +119,8 @@ class Contact:
|
|||
|
||||
call: str
|
||||
name: str = ""
|
||||
street: str = "" # the first line of the licensed address
|
||||
postcode: str = ""
|
||||
location: str = ""
|
||||
country: str = ""
|
||||
grid: str = ""
|
||||
|
|
@ -132,6 +134,12 @@ class Contact:
|
|||
def located(self) -> bool:
|
||||
return bool(self.latitude or self.longitude)
|
||||
|
||||
@property
|
||||
def address(self) -> str:
|
||||
"""The whole licensed address on one line, as far as it is known."""
|
||||
parts = [self.street, self.location, self.postcode]
|
||||
return ", ".join(p for p in parts if p)
|
||||
|
||||
@property
|
||||
def first_heard(self) -> float:
|
||||
times = [h.at for h in self.hearings if h.at]
|
||||
|
|
@ -157,7 +165,8 @@ class Contact:
|
|||
successful lookup wrote last time, which is why every field is only
|
||||
overwritten when the new value is not empty.
|
||||
"""
|
||||
for attr in ("name", "location", "country", "grid", "oper_class"):
|
||||
for attr in ("name", "street", "postcode", "location", "country",
|
||||
"grid", "oper_class"):
|
||||
value = getattr(entry, attr, "")
|
||||
if value:
|
||||
setattr(self, attr, value)
|
||||
|
|
@ -180,8 +189,8 @@ class Contact:
|
|||
rows = []
|
||||
if self.name:
|
||||
rows.append(("Licensee", self.name))
|
||||
if self.location:
|
||||
rows.append(("Licensed at", self.location))
|
||||
if self.address:
|
||||
rows.append(("Licensed at", self.address))
|
||||
elif self.country:
|
||||
rows.append(("Country", self.country))
|
||||
if self.oper_class:
|
||||
|
|
@ -226,7 +235,10 @@ class Contact:
|
|||
out.append(" <ExtendedData>")
|
||||
for key, value in (("callsign", self.call),
|
||||
("licensee", self.name),
|
||||
("address", self.address),
|
||||
("street", self.street),
|
||||
("location", self.location),
|
||||
("postcode", self.postcode),
|
||||
("country", self.country),
|
||||
("grid", self.grid),
|
||||
("class", self.oper_class),
|
||||
|
|
@ -275,6 +287,8 @@ def _read_placemark(element) -> Contact | None:
|
|||
return None
|
||||
contact = Contact(call=call,
|
||||
name=fields.get("licensee", ""),
|
||||
street=fields.get("street", ""),
|
||||
postcode=fields.get("postcode", ""),
|
||||
location=fields.get("location", ""),
|
||||
country=fields.get("country", ""),
|
||||
grid=fields.get("grid", ""),
|
||||
|
|
|
|||
326
bandsaunter/payload.py
Normal file
326
bandsaunter/payload.py
Normal file
|
|
@ -0,0 +1,326 @@
|
|||
"""Turning recovered bits into something a person can read.
|
||||
|
||||
A decoder that stops at a bit string has done half the job. ``4A 3F 1B 22``
|
||||
is a true statement about a transmission and tells nobody anything, and the
|
||||
answer to "what did the doorbell say" is not sixteen hexadecimal digits.
|
||||
|
||||
Three things happen here, in decreasing order of how much they claim.
|
||||
|
||||
Where the packet is a shape somebody standardised -- twenty address bits and
|
||||
four for the button, which is most of what a 433 MHz band contains -- its
|
||||
fields are named. That claims a lot and is safe to claim, because the shape
|
||||
itself is the evidence.
|
||||
|
||||
Where there is text in it, the text is read out. That claims a lot and is
|
||||
*not* safe to claim, because a long enough string of random bits contains a
|
||||
short readable word by accident; so the bar is set from measurement rather
|
||||
than from taste, and the measurement is in ``MIN_TEXT`` below.
|
||||
|
||||
And where neither applies the bytes are laid out the way a person reads
|
||||
bytes -- in groups, with the printable characters beside them -- which claims
|
||||
nothing at all but is still worth more than one long line of hex.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
__all__ = ["Reading", "interpret", "readable", "read_text", "hexdump",
|
||||
"describe_fields", "MIN_TEXT", "TEXT_CHARS"]
|
||||
|
||||
|
||||
# The alphabet a message is written in. Deliberately not "every printable
|
||||
# byte": the control codes and the high half of Latin-1 are printable in the
|
||||
# sense that a terminal will draw something, and letting them count turns one
|
||||
# byte in three of random data into evidence of text.
|
||||
TEXT_CHARS = frozenset(
|
||||
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
|
||||
"abcdefghijklmnopqrstuvwxyz"
|
||||
"0123456789"
|
||||
" .,:;!?'\"-/()[]@#%&+=*_<>$\n\t")
|
||||
|
||||
# What separates a message from a coincidence.
|
||||
#
|
||||
# Printability is not it, and this is the trap the first version fell into.
|
||||
# Every framing below is tried at every bit offset in both bit orders, which
|
||||
# is some forty readings of each packet, and seven-bit values are printable
|
||||
# three times in four -- so on random payloads a "readable" run of six turned
|
||||
# up 64% of the time. It has to be text, not merely typeable.
|
||||
#
|
||||
# So the run is measured the way text differs from noise. Real text is
|
||||
# nearly all one case; random letters are half and half. Real text is about
|
||||
# two fifths vowels; random letters over an alphabet of 52 are a fifth. And
|
||||
# real text is mostly letters, digits and spaces, with punctuation as the
|
||||
# exception rather than one character in four. Together, and with a length
|
||||
# bar, those take random payloads from 64% to under half a percent -- which
|
||||
# is measured in test_payload.py, and fails there if it ever stops being
|
||||
# true.
|
||||
MIN_TEXT = 8
|
||||
MIN_CASE = 0.9 # share of letters in the majority case
|
||||
VOWELS = (0.22, 0.60) # share of letters that are vowels
|
||||
MIN_WORDLIKE = 0.75 # share that is a letter, a digit or a space
|
||||
MIN_COVERAGE = 0.5 # share of that framing's characters the run is
|
||||
|
||||
|
||||
@dataclass
|
||||
class Reading:
|
||||
"""One way of reading a payload, and what it said."""
|
||||
|
||||
kind: str # "text" / "fields" / "bytes"
|
||||
text: str = ""
|
||||
how: str = "" # the framing it was read under
|
||||
at_bit: int = 0
|
||||
n_bits: int = 0
|
||||
confidence: float = 0.0
|
||||
fields: list[tuple[str, str]] = field(default_factory=list)
|
||||
|
||||
def line(self) -> str:
|
||||
if self.kind == "fields":
|
||||
return " ".join(f"{name} {value}" for name, value in self.fields)
|
||||
return self.text
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Text
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _chars(bits: str, width: int, msb_first: bool, offset: int) -> str:
|
||||
"""Read ``bits`` as characters of ``width`` bits, starting at ``offset``."""
|
||||
out = []
|
||||
for i in range(offset, len(bits) - width + 1, width):
|
||||
chunk = bits[i:i + width]
|
||||
value = int(chunk if msb_first else chunk[::-1], 2)
|
||||
out.append(chr(value))
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def _async_chars(bits: str, offset: int) -> str:
|
||||
"""Read 8N1 serial framing: a start bit, eight data bits, a stop bit.
|
||||
|
||||
Cheap radio modules are a UART with an aerial on the end, so a great deal
|
||||
of what turns up on 433 and 915 MHz is a serial line and nothing more.
|
||||
Reading it as bare bytes gets the data bits shifted by one and produces
|
||||
hex; reading the framing gets the message.
|
||||
"""
|
||||
out = []
|
||||
i = offset
|
||||
while i + 10 <= len(bits):
|
||||
if bits[i] != "0" or bits[i + 9] != "1":
|
||||
return "".join(out) # framing lost: stop where it was still good
|
||||
out.append(chr(int(bits[i + 1:i + 9][::-1], 2)))
|
||||
i += 10
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def _wordlike(text: str) -> float:
|
||||
"""The share of the run that is a letter, a digit or a space."""
|
||||
if not text:
|
||||
return 0.0
|
||||
good = sum(1 for c in text if c.isalnum() or c == " ")
|
||||
return good / len(text)
|
||||
|
||||
|
||||
def _reads_as_text(run: str) -> bool:
|
||||
"""Whether a run of typeable characters is actually a message.
|
||||
|
||||
Three measurements, each of which random data fails in a different way.
|
||||
See the note on MIN_TEXT above for where the numbers come from.
|
||||
"""
|
||||
if len(run) < MIN_TEXT or _wordlike(run) < MIN_WORDLIKE:
|
||||
return False
|
||||
letters = [c for c in run if c.isalpha()]
|
||||
if not letters:
|
||||
# All digits and punctuation. It may well be a reading, but nothing
|
||||
# here can tell it from a number that happened to land in the range.
|
||||
return False
|
||||
upper = sum(1 for c in letters if c.isupper())
|
||||
if max(upper, len(letters) - upper) / len(letters) < MIN_CASE:
|
||||
return False
|
||||
vowels = sum(1 for c in letters if c in "AEIOUaeiou") / len(letters)
|
||||
return VOWELS[0] <= vowels <= VOWELS[1]
|
||||
|
||||
|
||||
def _best_run(text: str) -> str:
|
||||
"""The longest unbroken stretch of the text alphabet."""
|
||||
best = ""
|
||||
for run in re.split(f"[^{re.escape(''.join(sorted(TEXT_CHARS)))}]+", text):
|
||||
if len(run) > len(best):
|
||||
best = run
|
||||
return best
|
||||
|
||||
|
||||
# (name, width, msb_first) -- every way a byte gets onto the air.
|
||||
FRAMINGS: tuple[tuple[str, int, bool], ...] = (
|
||||
("8-bit ASCII", 8, True),
|
||||
("8-bit ASCII, LSB first", 8, False),
|
||||
("7-bit ASCII", 7, True),
|
||||
("7-bit ASCII, LSB first", 7, False),
|
||||
)
|
||||
|
||||
|
||||
def read_text(bits: str, minimum: int = MIN_TEXT) -> Reading | None:
|
||||
"""The best readable text in ``bits``, or None if there is none.
|
||||
|
||||
Every framing at every bit offset, because a packet's text almost never
|
||||
starts on a byte boundary of the recovered bit string: the preamble and
|
||||
the address come first and they are whatever length the designer felt
|
||||
like. The cost of trying them all is that some of them will read as text
|
||||
by accident, which is what the length bar is for.
|
||||
"""
|
||||
best: Reading | None = None
|
||||
candidates: list[tuple[str, int, str]] = []
|
||||
for how, width, msb in FRAMINGS:
|
||||
for offset in range(width):
|
||||
candidates.append((how, offset, _chars(bits, width, msb, offset)))
|
||||
for offset in range(10):
|
||||
got = _async_chars(bits, offset)
|
||||
if got:
|
||||
candidates.append(("8N1 serial", offset, got))
|
||||
|
||||
for how, offset, decoded in candidates:
|
||||
run = _best_run(decoded)
|
||||
if len(run) < minimum or not _reads_as_text(run):
|
||||
continue
|
||||
# And it has to be most of what that framing produced. A message
|
||||
# sits between a preamble and a checksum, so it is the bulk of the
|
||||
# packet; a coincidence is a fragment of one.
|
||||
if len(run) / max(1, len(decoded)) < MIN_COVERAGE:
|
||||
continue
|
||||
share = _wordlike(run)
|
||||
# Longer wins, and a tie goes to the more word-like reading: the same
|
||||
# bits read two ways will both contain something, and the one that
|
||||
# reads as a sentence is the one that was sent.
|
||||
score = (len(run), share)
|
||||
if best is None or score > (len(best.text), best.confidence):
|
||||
best = Reading(kind="text", text=run.strip(), how=how,
|
||||
at_bit=offset, n_bits=len(bits), confidence=share)
|
||||
return best
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Shapes somebody standardised
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _ev1527(bits: str) -> list[tuple[str, str]]:
|
||||
"""Twenty address bits and four for the button.
|
||||
|
||||
The chip in every cheap doorbell, gate remote, PIR sensor and window
|
||||
contact on 433 MHz, and in the clones of it that fill the band. The
|
||||
address is set at the factory or by solder blobs and identifies the
|
||||
transmitter; the low four bits say which button, one bit per button, so a
|
||||
remote with four of them sends one of 1000, 0100, 0010, 0001.
|
||||
"""
|
||||
address, buttons = bits[:20], bits[20:24]
|
||||
out = [("address", f"0x{int(address, 2):05X}")]
|
||||
pressed = [i for i, b in enumerate(buttons) if b == "1"]
|
||||
if len(pressed) == 1:
|
||||
out.append(("button", "ABCD"[pressed[0]]))
|
||||
else:
|
||||
out.append(("data", f"0b{buttons}"))
|
||||
return out
|
||||
|
||||
|
||||
# File formats that announce themselves in their first four bytes. None of
|
||||
# these is a modulation and none is decoded here; the point is to say what a
|
||||
# byte stream is rather than let it come out as hex. GRIB in particular gets
|
||||
# asked about as though it were something a receiver demodulates: it is the
|
||||
# format weather models are published in, and it travels by satellite data
|
||||
# link and by e-mail.
|
||||
MAGIC: tuple[tuple[bytes, str], ...] = (
|
||||
(b"GRIB", "GRIB, the gridded weather-model format"),
|
||||
(b"BUFR", "BUFR, the WMO observation format"),
|
||||
(b"\x89PNG", "a PNG image"),
|
||||
(b"\xff\xd8\xff", "a JPEG image"),
|
||||
(b"%PDF", "a PDF document"),
|
||||
)
|
||||
|
||||
|
||||
def _named_format(bits: str) -> str:
|
||||
"""What a payload says it is, where its first bytes say anything."""
|
||||
head = bytes(int(bits[i:i + 8], 2) for i in range(0, min(len(bits) - len(bits) % 8, 32), 8))
|
||||
for magic, name in MAGIC:
|
||||
if head.startswith(magic):
|
||||
return name
|
||||
return ""
|
||||
|
||||
|
||||
def describe_fields(bits: str, encoding: str = "") -> Reading | None:
|
||||
"""Break a packet into named fields where its shape says what they are."""
|
||||
if encoding == "PWM" and len(bits) == 24:
|
||||
return Reading(kind="fields", how="EV1527 / PT2262", n_bits=24,
|
||||
confidence=0.9, fields=_ev1527(bits))
|
||||
named = _named_format(bits)
|
||||
if named:
|
||||
return Reading(kind="fields", how="file format", n_bits=len(bits),
|
||||
confidence=0.95, fields=[("this is", named)])
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The bytes themselves, laid out to be read
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def hexdump(bits: str, per_line: int = 8) -> list[str]:
|
||||
"""The payload as bytes, in groups, with the printable characters beside.
|
||||
|
||||
What ``hexdump -C`` does, and for the same reason: a wall of hex hides
|
||||
the one field that is a word, and the gutter is where a person spots it.
|
||||
Any trailing bits that do not fill a byte are shown as bits, because
|
||||
padding them to a byte would invent four zeroes nobody sent.
|
||||
"""
|
||||
whole = len(bits) - len(bits) % 8
|
||||
data = bytes(int(bits[i:i + 8], 2) for i in range(0, whole, 8))
|
||||
out = []
|
||||
for start in range(0, len(data), per_line):
|
||||
chunk = data[start:start + per_line]
|
||||
cells = " ".join(f"{b:02X}" for b in chunk)
|
||||
gutter = "".join(chr(b) if chr(b) in TEXT_CHARS and b >= 0x20 else "."
|
||||
for b in chunk)
|
||||
out.append(f"{start:04X} {cells:<{per_line * 3 - 1}} |{gutter}|")
|
||||
if whole < len(bits):
|
||||
out.append(f"{len(data):04X} + {len(bits) - whole} bit(s): "
|
||||
f"{bits[whole:]}")
|
||||
return out
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# All of it together
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def interpret(bits: str, encoding: str = "", protocol: str = "",
|
||||
trusted: bool = False) -> list[Reading]:
|
||||
"""Every reading of ``bits`` worth showing, most informative first.
|
||||
|
||||
``trusted`` says the packet is known good -- it arrived identically
|
||||
several times, or its checksum came out right -- which lowers the bar on
|
||||
the text search. A five-character word inside a packet that arrived six
|
||||
times over is not a coincidence; the same five characters inside a single
|
||||
unrepeated burst may well be.
|
||||
"""
|
||||
if not bits:
|
||||
return []
|
||||
out: list[Reading] = []
|
||||
fields = describe_fields(bits, encoding)
|
||||
if fields is not None:
|
||||
out.append(fields)
|
||||
text = read_text(bits, minimum=MIN_TEXT - 1 if trusted else MIN_TEXT)
|
||||
if text is not None:
|
||||
out.append(text)
|
||||
out.append(Reading(kind="bytes", n_bits=len(bits),
|
||||
text="\n".join(hexdump(bits))))
|
||||
return out
|
||||
|
||||
|
||||
def readable(bits: str, encoding: str = "", protocol: str = "",
|
||||
trusted: bool = False) -> list[str]:
|
||||
"""``interpret`` as lines of text, ready to print or write to a file."""
|
||||
lines: list[str] = []
|
||||
for reading in interpret(bits, encoding, protocol, trusted):
|
||||
if reading.kind == "fields":
|
||||
lines.append(f"{reading.how}: {reading.line()}")
|
||||
elif reading.kind == "text":
|
||||
lines.append(f'text ({reading.how}): "{reading.text}"')
|
||||
else:
|
||||
lines.extend(reading.text.split("\n"))
|
||||
return lines
|
||||
63
bandsaunter/pictures.py
Normal file
63
bandsaunter/pictures.py
Normal file
|
|
@ -0,0 +1,63 @@
|
|||
"""Which picture decoder to offer a capture to, and in what order.
|
||||
|
||||
The three image modes are unlike each other in every respect except that all
|
||||
of them are sound, so this is where a capture is handed round until one of
|
||||
them recognises it. Each refuses on its own evidence -- SSTV wants a VIS
|
||||
header, APT wants both line syncs at the right distance, fax wants a phasing
|
||||
signal -- so the order is about not wasting time rather than about resolving
|
||||
disagreements between them.
|
||||
|
||||
The dial reading narrows it where it can. APT is only ever in the 137 MHz
|
||||
satellite band, and weather fax is a shortwave thing sent in single sideband,
|
||||
so neither is worth trying anywhere else; SSTV turns up on HF and on two
|
||||
metres alike and is offered everything.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
|
||||
from .apt import apt_frequency, decode_apt
|
||||
from .fax import decode_fax
|
||||
from .images import ImageDecode
|
||||
from .sstv import decode_sstv
|
||||
|
||||
__all__ = ["find_image", "IMAGE_KINDS"]
|
||||
|
||||
IMAGE_KINDS = ("SSTV", "APT", "HF fax")
|
||||
|
||||
# Above this, shortwave fax is not a thing. Weather fax stations are all
|
||||
# between 2 and 20 MHz; the ceiling is generous because a receiver's dial and
|
||||
# a station's schedule do not always agree.
|
||||
HF_CEILING = 30e6
|
||||
|
||||
|
||||
def find_image(audio: np.ndarray, rate: float, frequency: float = 0.0,
|
||||
mode: str = "") -> ImageDecode | None:
|
||||
"""The picture in a capture, if it is one.
|
||||
|
||||
Returns None when no decoder recognised it, which is the answer for
|
||||
almost every capture ever made. ``frequency`` and ``mode`` only decide
|
||||
what is worth trying; nothing here accepts a picture it was not shown.
|
||||
"""
|
||||
audio = np.asarray(audio, dtype=np.float64).ravel()
|
||||
if audio.size < int(rate * 2.0):
|
||||
return None
|
||||
|
||||
# SSTV first, and everywhere. Its header is the cheapest thing to look
|
||||
# for and the least likely to be found by accident.
|
||||
found = decode_sstv(audio, rate)
|
||||
if found is not None and found.ok:
|
||||
return found
|
||||
|
||||
if not frequency or apt_frequency(frequency):
|
||||
got = decode_apt(audio, rate, frequency)
|
||||
if got is not None and got.ok:
|
||||
return got
|
||||
|
||||
if not frequency or frequency <= HF_CEILING or mode in ("usb", "lsb"):
|
||||
got = decode_fax(audio, rate)
|
||||
if got is not None and got.ok:
|
||||
return got
|
||||
|
||||
return found # the SSTV note, where there was one, or None
|
||||
|
|
@ -85,6 +85,17 @@ class HitRecord:
|
|||
data_confidence: float = 0.0
|
||||
data_path: str = ""
|
||||
|
||||
# A picture, where the capture turned out to be one. Weather satellites,
|
||||
# SSTV and shortwave fax are all images sent as sound, so they come out of
|
||||
# the same audio everything else does and land beside it as a PNG.
|
||||
image_kind: str = "" # APT / SSTV / HF fax
|
||||
image_mode: str = "" # NOAA APT, Martin M1, 120 lpm IOC 576, ...
|
||||
image_path: str = ""
|
||||
image_width: int = 0
|
||||
image_height: int = 0
|
||||
image_complete: bool = True
|
||||
image_paths: list[str] = field(default_factory=list) # the extra channels
|
||||
|
||||
category: str = "" # voice / cw / digital / carrier / noise
|
||||
signal_score: float = 0.0
|
||||
voice_score: float = 0.0
|
||||
|
|
|
|||
|
|
@ -28,6 +28,8 @@ from .decode import decode_data
|
|||
from .demod import make_demodulator
|
||||
from .device import RtlSdrDevice, RtlSdrError
|
||||
from .kml import KmlLog
|
||||
from .images import ImageDecode
|
||||
from .pictures import find_image
|
||||
from .morse import decode_morse
|
||||
from .quality import Assessment, assess
|
||||
from .ranges import Lockout, TuneStep, build_plan
|
||||
|
|
@ -890,11 +892,18 @@ class Scanner:
|
|||
self._note_control(det.frequency, verdict)
|
||||
self._reject(rec, hit)
|
||||
return hit
|
||||
# Before the content check, because a picture has to survive it. A
|
||||
# weather satellite is a steady tone with a wobble on it and an SSTV
|
||||
# transmission is a whistle: neither is speech, neither has symbol
|
||||
# structure, and both were being thrown away as "no signal content"
|
||||
# having already been recognised.
|
||||
picture = self._decode_images(rec, hit)
|
||||
if cfg.require_signal and verdict is not None and not verdict.accept \
|
||||
and not hit.data_messages:
|
||||
and not hit.data_messages and not hit.image_kind:
|
||||
# A decoded packet outranks the content check. That test works
|
||||
# from statistics -- how noise-like, how speech-like -- and a
|
||||
# frame whose own checksum came out right is not a statistic.
|
||||
# Nor is a picture that came with its own header.
|
||||
hit.kept = False
|
||||
hit.stop_reason = f"no signal content: {verdict.reason}"
|
||||
self._reject(rec, hit)
|
||||
|
|
@ -907,6 +916,7 @@ class Scanner:
|
|||
hit.audio_path = str(rec.audio_path) if cfg.save_audio else ""
|
||||
hit.iq_path = str(rec.iq_path) if cfg.save_iq else ""
|
||||
self._write_data_file(rec, hit)
|
||||
self._save_image(rec, hit, picture)
|
||||
|
||||
# Add this transmission to the running file for its frequency, after
|
||||
# a spoken timestamp, so one channel plays back as one recording.
|
||||
|
|
@ -1044,6 +1054,41 @@ class Scanner:
|
|||
self._error(exc)
|
||||
return None
|
||||
|
||||
def _morse_from_recording(self, rec: Recording, morse):
|
||||
"""Read the Morse again, from the whole recording this time.
|
||||
|
||||
The decode above works from the classifier's buffer, which holds a
|
||||
few seconds -- enough to say "this is Morse", and not always enough
|
||||
to catch a callsign whole between two word gaps. A beacon repeating
|
||||
every eight seconds through a buffer eight seconds wide is caught
|
||||
mid-message every time, and the truncated words are dropped rather
|
||||
than reported, so the station is never identified.
|
||||
|
||||
A capture recorded in cw mode has the beat note in its .wav from end
|
||||
to end, so where that file exists it is worth a second look. The
|
||||
longer identifiable reading wins; neither is trusted more than the
|
||||
other, they are the same decoder over different amounts of signal.
|
||||
"""
|
||||
if not rec.audio_path.exists():
|
||||
return morse
|
||||
try:
|
||||
audio, rate = read_wav(rec.audio_path)
|
||||
except (OSError, ValueError):
|
||||
return morse
|
||||
if audio is None or audio.size < int(rate * 0.5):
|
||||
return morse
|
||||
try:
|
||||
longer = decode_morse(audio, rate)
|
||||
except Exception as exc:
|
||||
self._error(exc)
|
||||
return morse
|
||||
if not longer.is_morse:
|
||||
return morse
|
||||
if morse is None or not morse.is_morse:
|
||||
return longer
|
||||
return (longer if len(longer.complete_text) > len(morse.complete_text)
|
||||
else morse)
|
||||
|
||||
def _submit_transcription(self, rec: Recording, hit: HitRecord) -> None:
|
||||
"""Queue a voice capture for speech recognition.
|
||||
|
||||
|
|
@ -1100,7 +1145,7 @@ class Scanner:
|
|||
return
|
||||
try:
|
||||
got = decode_data(iq, demod.if_rate, family=cls.family,
|
||||
baud_hint=hit.baud)
|
||||
baud_hint=hit.baud, frequency=hit.frequency)
|
||||
except Exception as exc:
|
||||
self._error(exc)
|
||||
return
|
||||
|
|
@ -1147,6 +1192,70 @@ class Scanner:
|
|||
except OSError as exc:
|
||||
self._error(exc)
|
||||
|
||||
# Modes that are a picture rather than a sound or a stream of bits. All
|
||||
# three ride on the audio the scanner already records, so they are tried
|
||||
# once the capture is closed and the .wav is on disk -- which is also the
|
||||
# only place the whole of a fifteen-minute satellite pass exists.
|
||||
def _decode_images(self, rec: Recording, hit: HitRecord):
|
||||
"""Look for a picture in what was recorded.
|
||||
|
||||
Read from the .wav rather than from what is still in memory: a
|
||||
satellite pass is fifteen minutes long and the classifier's buffer
|
||||
holds seconds, and the file is the only place the whole of it exists.
|
||||
Nothing is written here -- the capture has not been renamed yet -- so
|
||||
this only decides, and :meth:`_save_image` writes.
|
||||
"""
|
||||
if not self.cfg.decode_images or not rec.audio_path.exists():
|
||||
return None
|
||||
try:
|
||||
audio, rate = read_wav(rec.audio_path)
|
||||
except (OSError, ValueError) as exc:
|
||||
self._error(exc)
|
||||
return None
|
||||
if audio is None or audio.size < int(rate * 2.0):
|
||||
return None
|
||||
try:
|
||||
got = find_image(audio, rate, frequency=hit.frequency,
|
||||
mode=hit.mode)
|
||||
except Exception as exc:
|
||||
self._error(exc)
|
||||
return None
|
||||
if got is None or not got.ok:
|
||||
return None
|
||||
|
||||
# After _identify, which set the category from the content check. A
|
||||
# picture is what the capture is, whatever the statistics made of it.
|
||||
hit.category = "image"
|
||||
hit.image_kind = got.kind
|
||||
hit.image_mode = got.mode
|
||||
hit.image_width = got.width
|
||||
hit.image_height = got.height
|
||||
hit.image_complete = got.complete
|
||||
hit.reasons.insert(0, f"decoded an image: {got.summary()}")
|
||||
hit.classification = got.name
|
||||
# The family is left as the demodulator that received it, because
|
||||
# that is what the filename is made of and "how it was received" is
|
||||
# still true of a picture. What it is goes in the classification.
|
||||
hit.confidence = max(hit.confidence, got.confidence)
|
||||
return got
|
||||
|
||||
def _save_image(self, rec: Recording, hit: HitRecord, picture) -> None:
|
||||
"""Write the picture beside the recording, once both have their name."""
|
||||
if picture is None:
|
||||
return
|
||||
try:
|
||||
hit.image_path = picture.save(rec.dir / f"{rec.stem}.png")
|
||||
for name, pixels in picture.channels.items():
|
||||
extra = ImageDecode(ok=True, pixels=pixels)
|
||||
path = extra.save(rec.dir / f"{rec.stem}_{name}.png")
|
||||
if path:
|
||||
hit.image_paths.append(path)
|
||||
except (OSError, ValueError) as exc:
|
||||
self._error(exc)
|
||||
return
|
||||
self._status(f"picture from {fmt_hz(hit.frequency)}: "
|
||||
f"{picture.summary()}")
|
||||
|
||||
def _announce_data(self, hit: HitRecord, got) -> None:
|
||||
"""Say on the display what was just read off the air."""
|
||||
self._status(f"decoded {fmt_hz(hit.frequency)}: {got.summary()}")
|
||||
|
|
@ -1302,6 +1411,8 @@ class Scanner:
|
|||
# symbol-rate estimate rather than a zero.
|
||||
self._decode_payload(rec, hit, demod, cls)
|
||||
|
||||
if hit.mode == "cw":
|
||||
morse = self._morse_from_recording(rec, morse)
|
||||
if morse is not None and morse.is_morse:
|
||||
hit.morse_text = morse.text
|
||||
hit.morse_complete = morse.complete_text
|
||||
|
|
|
|||
|
|
@ -313,6 +313,11 @@ _TABLE: tuple[Setting, ...] = (
|
|||
"Turning this off also turns off the content check, since that is what "
|
||||
"decides whether a capture is worth keeping.",
|
||||
flags=("--classify",), off_flags=("--no-classify",)),
|
||||
S("decode_images", "Decode pictures", "Output", "bool",
|
||||
"save SSTV, weather satellite and shortwave fax pictures as PNG",
|
||||
"Some transmissions are pictures rather than sounds. This looks for "
|
||||
"them in every recording and writes what it finds beside the audio.",
|
||||
flags=("--images",), off_flags=("--no-images",)),
|
||||
S("decode_morse", "Decode CW to text", "Output", "bool",
|
||||
"decode keyed carriers as Morse",
|
||||
"Speed is measured from the signal, so nothing needs configuring. "
|
||||
|
|
@ -707,6 +712,14 @@ _GUIDANCE: dict[str, str] = {
|
|||
"single sideband, Morse, a paging system or a digital voice mode, "
|
||||
"and write it into the log and the filename. Turning it off saves a "
|
||||
"little processing and leaves you to identify things by ear.",
|
||||
"decode_images":
|
||||
"Three of the things a receiver can hear are pictures: the weather "
|
||||
"satellites on 137 MHz, amateur slow-scan television, and the "
|
||||
"shortwave weather fax stations. All three are images sent as sound, "
|
||||
"so they arrive in the same recordings everything else does. Each is "
|
||||
"recognised by its own header rather than guessed at, so this costs "
|
||||
"a moment per recording and finds nothing where there is nothing. "
|
||||
"What it does find is written as a PNG beside the audio.",
|
||||
"decode_morse":
|
||||
"Turn keyed carriers into readable text, with the sending speed. "
|
||||
"Morse is still in daily use by amateurs and by beacons, and this "
|
||||
|
|
|
|||
|
|
@ -107,7 +107,7 @@ class VirtualTransmitter:
|
|||
"""One synthetic signal on the air."""
|
||||
|
||||
frequency: float
|
||||
mode: str = "nfm" # nfm wfm am usb cw fsk2 fsk4 psk
|
||||
mode: str = "nfm" # nfm wfm am usb cw sstv fsk2 fsk4 psk
|
||||
# carrier ook packet pocsag
|
||||
power: float = 0.35 # linear amplitude
|
||||
bandwidth: float = 12_500.0
|
||||
|
|
@ -176,6 +176,14 @@ class VirtualTransmitter:
|
|||
from scipy.signal import hilbert
|
||||
an = hilbert(audio)
|
||||
out = an if m == "usb" else np.conj(an)
|
||||
elif m == "sstv":
|
||||
# The picture is an audio tone, and the audio tone is what
|
||||
# modulates the carrier -- exactly as it would through a
|
||||
# microphone socket. Anything else produces a signal that
|
||||
# demodulates to a slowly drifting baseband rather than to the
|
||||
# whistle a decoder is listening for.
|
||||
audio = self._sstv_audio(t, fs)
|
||||
out = np.exp(1j * self._advance(3_000.0 * audio, fs))
|
||||
elif m == "cw":
|
||||
env = self._morse_envelope(t, fs)
|
||||
out = env * np.exp(1j * self._advance(np.zeros(n), fs))
|
||||
|
|
@ -323,6 +331,67 @@ class VirtualTransmitter:
|
|||
taps /= taps.sum()
|
||||
return np.convolve(lv, taps, mode="same")[half:half + n]
|
||||
|
||||
def _sstv_frequency(self, t: np.ndarray) -> np.ndarray:
|
||||
"""The audio frequency of a Martin M1 transmission, sample by sample.
|
||||
|
||||
Built from the mode's own line structure, so the demo band carries a
|
||||
real transmission rather than a whistle that decodes to nothing. The
|
||||
picture is a set of colour bars over a grey wedge, which is what a
|
||||
test card is for: wrong colours, wrong geometry and lines out of
|
||||
order are all visible at a glance.
|
||||
"""
|
||||
from .sstv import (BIT_ONE_HZ, BIT_ZERO_HZ, BLACK_HZ, BREAK_HZ,
|
||||
LEADER_HZ, SYNC_HZ, VIS_CODES, WHITE_HZ)
|
||||
mode = VIS_CODES[44] # Martin M1
|
||||
header = [(LEADER_HZ, 0.300), (BREAK_HZ, 0.010), (LEADER_HZ, 0.300),
|
||||
(SYNC_HZ, 0.030)]
|
||||
vis = [(mode.vis >> i) & 1 for i in range(7)]
|
||||
vis.append(sum(vis) % 2)
|
||||
header += [(BIT_ONE_HZ if b else BIT_ZERO_HZ, 0.030) for b in vis]
|
||||
header.append((SYNC_HZ, 0.030))
|
||||
|
||||
# One line of the picture, as a function of how far across it we are.
|
||||
def bar(where: np.ndarray) -> np.ndarray:
|
||||
return (np.floor(where * 8) % 8) / 7.0
|
||||
|
||||
begin = 0.0
|
||||
if self.period_seconds > 0 and self.on_seconds > 0:
|
||||
begin = (((t + self.phase_offset) // self.period_seconds)
|
||||
* self.period_seconds - self.phase_offset)
|
||||
into = t - begin
|
||||
out = np.full(t.size, BLACK_HZ)
|
||||
at = 0.0
|
||||
for hz, seconds in header:
|
||||
here = (into >= at) & (into < at + seconds)
|
||||
out[here] = hz
|
||||
at += seconds
|
||||
head = at
|
||||
line_at = np.maximum(0.0, into - head)
|
||||
line = np.floor(line_at / mode.line_seconds)
|
||||
within = line_at - line * mode.line_seconds
|
||||
cursor = 0.0
|
||||
for what, seconds in mode.segments():
|
||||
here = (into >= head) & (within >= cursor) & \
|
||||
(within < cursor + seconds)
|
||||
if what == "sync":
|
||||
out[here] = SYNC_HZ
|
||||
elif len(what) == 1:
|
||||
across = (within[here] - cursor) / max(seconds, 1e-9)
|
||||
shade = bar(across) if what == "G" else \
|
||||
(1.0 - bar(across) if what == "B" else across)
|
||||
out[here] = BLACK_HZ + shade * (WHITE_HZ - BLACK_HZ)
|
||||
cursor += seconds
|
||||
return out
|
||||
|
||||
def _sstv_audio(self, t: np.ndarray, fs: float) -> np.ndarray:
|
||||
"""The SSTV tone itself, as audio between -1 and 1."""
|
||||
freq = self._sstv_frequency(t)
|
||||
self._sstv_phase = getattr(self, "_sstv_phase", 0.0)
|
||||
phase = self._sstv_phase + np.cumsum(2.0 * np.pi * freq / fs)
|
||||
if phase.size:
|
||||
self._sstv_phase = float(phase[-1] % (2.0 * np.pi))
|
||||
return np.sin(phase)
|
||||
|
||||
def _morse_envelope(self, t: np.ndarray, fs: float) -> np.ndarray:
|
||||
key = self._morse_key(self.message, self.wpm)
|
||||
dot = 1.2 / self.wpm
|
||||
|
|
@ -405,6 +474,11 @@ def default_transmitters() -> list[VirtualTransmitter]:
|
|||
deviation=4_500, period_seconds=11, on_seconds=2.5, phase_offset=5,
|
||||
pages=((1234568, 3, "ENGINE 4 RESPOND"), (98765, 0, "CALL EXT 4412"))),
|
||||
V(446_000_000, "carrier", 0.25, 1_000, "unmodulated carrier"),
|
||||
# A picture, because the pictures are the hardest part to believe
|
||||
# without seeing one come out. Slow-scan television on the calling
|
||||
# frequency, sent over and over the way a net does.
|
||||
V(144_500_000, "sstv", 0.38, 3_000, "2 m SSTV", period_seconds=40,
|
||||
on_seconds=14.0, phase_offset=3),
|
||||
V(433_920_000, "packet", 0.30, 40_000, "ISM remote", baud=2000,
|
||||
period_seconds=9, on_seconds=1.2,
|
||||
payload="101100100011010101001110", repeats=5),
|
||||
|
|
|
|||
451
bandsaunter/sstv.py
Normal file
451
bandsaunter/sstv.py
Normal file
|
|
@ -0,0 +1,451 @@
|
|||
"""Slow-scan television: pictures sent as audio, one line at a time.
|
||||
|
||||
An SSTV transmission is a single tone that wobbles. Brightness is frequency
|
||||
-- 1500 Hz is black, 2300 Hz is white -- and a 1200 Hz pulse marks the start
|
||||
of each line. A picture takes one to two minutes to arrive, which is where
|
||||
the name comes from and why nobody sends more than one.
|
||||
|
||||
The transmission announces itself. Before the picture comes a VIS header: a
|
||||
leader tone, a break, another leader, then eight bits at 30 ms each that say
|
||||
which mode follows. That header is what makes automatic decoding possible at
|
||||
all, and it is what is looked for here -- no header, no picture, because a
|
||||
decoder that starts drawing whenever the audio happens to be in the right
|
||||
range fills a directory with noise.
|
||||
|
||||
The modes below are the ones actually used on the air. Martin and Scottie
|
||||
send green, blue and red as three separate scans of each line; Robot sends
|
||||
luminance every line and the two colour differences on alternate lines, which
|
||||
is why its colour is softer and its files are smaller.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
import numpy as np
|
||||
from scipy import ndimage as _ndi
|
||||
|
||||
from .images import ImageDecode, instantaneous_frequency, resample_to
|
||||
|
||||
__all__ = ["decode_sstv", "SSTVMode", "MODES", "find_vis", "VIS_CODES"]
|
||||
|
||||
|
||||
# Frequencies, all in hertz. These are the standard and are not adjustable:
|
||||
# a transmission that used different ones would not be SSTV.
|
||||
LEADER_HZ = 1900.0
|
||||
BREAK_HZ = 1200.0
|
||||
SYNC_HZ = 1200.0
|
||||
BIT_ONE_HZ = 1100.0
|
||||
BIT_ZERO_HZ = 1300.0
|
||||
BLACK_HZ = 1500.0
|
||||
WHITE_HZ = 2300.0
|
||||
|
||||
VIS_BIT_SECONDS = 0.030
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SSTVMode:
|
||||
"""One SSTV mode, as the ordered pieces a line is actually made of.
|
||||
|
||||
Written this way rather than as three equal scans because the modes are
|
||||
not built alike, and guessing wrong walks the picture off the screen
|
||||
within a few lines. Martin puts its sync pulse at the head of the line;
|
||||
Scottie puts it two thirds of the way in, between blue and red. Robot
|
||||
sends luminance at full width and its colour differences at half, so a
|
||||
Robot 36 line is a scan and a half rather than three.
|
||||
|
||||
``line_ms`` is the published length of a line in each mode, and is here so
|
||||
that :func:`segments` can be checked against it rather than believed.
|
||||
"""
|
||||
|
||||
vis: int
|
||||
name: str
|
||||
width: int
|
||||
height: int
|
||||
scan: float # seconds per full-width colour scan
|
||||
sync: float # seconds of the 1200 Hz line pulse
|
||||
porch: float # seconds between sync and the scan after it
|
||||
separator: float # seconds between scans
|
||||
line_ms: float # the published line time, for checking
|
||||
layout: str = "GBR" # GBR / YUV / YUV2 / mono
|
||||
sync_first: bool = True # False for Scottie, whose sync is mid-line
|
||||
tail: float = 0.0 # trailing porch some modes have
|
||||
|
||||
def segments(self) -> tuple[tuple[str, float], ...]:
|
||||
"""One line, in order, as ``(what, seconds)``.
|
||||
|
||||
The names that are single letters are the scans a decoder reads; the
|
||||
rest is timing it has to step over.
|
||||
"""
|
||||
sync = ("sync", self.sync)
|
||||
porch = ("porch", self.porch)
|
||||
sep = ("sep", self.separator)
|
||||
if self.layout == "mono":
|
||||
return (sync, porch, ("Y", self.scan))
|
||||
if self.layout == "YUV": # Robot 36
|
||||
return (sync, porch, ("Y", self.scan), sep,
|
||||
("C", self.scan / 2), ("tail", self.tail))
|
||||
if self.layout == "YUV2": # Robot 72
|
||||
return (sync, porch, ("Y", self.scan), sep,
|
||||
("U", self.scan / 2), sep, ("V", self.scan / 2),
|
||||
("tail", self.tail))
|
||||
if self.sync_first: # Martin
|
||||
return (sync, porch, ("G", self.scan), sep, ("B", self.scan),
|
||||
sep, ("R", self.scan), sep)
|
||||
return (sep, ("G", self.scan), sep, ("B", self.scan), # Scottie
|
||||
sync, porch, ("R", self.scan))
|
||||
|
||||
@property
|
||||
def line_seconds(self) -> float:
|
||||
return sum(seconds for _, seconds in self.segments())
|
||||
|
||||
def offset_of(self, name: str) -> float:
|
||||
"""Seconds from the start of a line to the start of a named piece."""
|
||||
at = 0.0
|
||||
for what, seconds in self.segments():
|
||||
if what == name:
|
||||
return at
|
||||
at += seconds
|
||||
return 0.0
|
||||
|
||||
def length_of(self, name: str) -> float:
|
||||
for what, seconds in self.segments():
|
||||
if what == name:
|
||||
return seconds
|
||||
return 0.0
|
||||
|
||||
|
||||
# The VIS codes are assigned; the timings are from the mode specifications,
|
||||
# and test_images.py checks that each adds up to the published line time.
|
||||
MODES: tuple[SSTVMode, ...] = (
|
||||
SSTVMode(44, "Martin M1", 320, 256, 0.146432, 0.004862, 0.000572,
|
||||
0.000572, 446.446),
|
||||
SSTVMode(40, "Martin M2", 320, 256, 0.073216, 0.004862, 0.000572,
|
||||
0.000572, 226.798),
|
||||
SSTVMode(60, "Scottie S1", 320, 256, 0.138240, 0.009000, 0.001500,
|
||||
0.001500, 428.220, sync_first=False),
|
||||
SSTVMode(56, "Scottie S2", 320, 256, 0.088064, 0.009000, 0.001500,
|
||||
0.001500, 277.692, sync_first=False),
|
||||
SSTVMode(76, "Scottie DX", 320, 256, 0.345600, 0.009000, 0.001500,
|
||||
0.001500, 1050.300, sync_first=False),
|
||||
SSTVMode(8, "Robot 36", 320, 240, 0.088000, 0.009000, 0.003000,
|
||||
0.004500, 150.000, layout="YUV", tail=0.0015),
|
||||
SSTVMode(12, "Robot 72", 320, 240, 0.138000, 0.009000, 0.003000,
|
||||
0.004500, 300.000, layout="YUV2", tail=0.003),
|
||||
)
|
||||
|
||||
VIS_CODES = {mode.vis: mode for mode in MODES}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The header
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _nearest(freq: float, *targets: float) -> float:
|
||||
return min(targets, key=lambda t: abs(freq - t))
|
||||
|
||||
|
||||
def find_vis(freq: np.ndarray, rate: float,
|
||||
tolerance: float = 60.0) -> tuple[int, int] | None:
|
||||
"""Find the VIS header. Returns ``(code, sample the picture starts at)``.
|
||||
|
||||
Looked for by its shape rather than by correlation: a 1200 Hz break at
|
||||
least 5 ms long with a 1900 Hz leader on each side of it, then the start
|
||||
bit, then eight data bits. The parity bit is checked, because a header
|
||||
read wrongly names the wrong mode and produces a picture that is
|
||||
recognisably a picture and completely wrong.
|
||||
"""
|
||||
if freq.size < int(rate * 0.5):
|
||||
return None
|
||||
# The header is looked for in a median-filtered copy. Every tone in it
|
||||
# lasts at least ten milliseconds, so a filter a millisecond wide costs
|
||||
# nothing and takes out the sample-by-sample scatter that a weak signal
|
||||
# puts on a frequency estimate -- without it, a header at 15 dB is a
|
||||
# 1200 Hz run that keeps breaking in half and is never long enough.
|
||||
window = int(round(0.001 * rate)) | 1
|
||||
if window >= 3:
|
||||
freq = _ndi.median_filter(freq, size=window, mode="nearest")
|
||||
bit = int(round(VIS_BIT_SECONDS * rate))
|
||||
# The break is nominally 10 ms and never shorter, but the band-pass that
|
||||
# produced these frequencies takes a moment to swing between two tones,
|
||||
# so a few milliseconds at each end of it read as neither. Half the
|
||||
# nominal length is still far longer than anything else at 1200 Hz.
|
||||
break_len = int(round(0.005 * rate))
|
||||
leader = int(round(0.100 * rate)) # of the 300 ms, enough to be sure
|
||||
|
||||
def near(values, target):
|
||||
return np.abs(values - target) < tolerance
|
||||
|
||||
# Where the audio sits on 1200 Hz for about ten milliseconds.
|
||||
on_break = near(freq, BREAK_HZ)
|
||||
# A cheap run-length scan for a stretch of break long enough.
|
||||
edges = np.flatnonzero(np.diff(on_break.astype(np.int8)))
|
||||
starts = np.concatenate(([0], edges + 1))
|
||||
ends = np.concatenate((edges + 1, [on_break.size]))
|
||||
for start, end in zip(starts, ends):
|
||||
if not on_break[start] or (end - start) < break_len:
|
||||
continue
|
||||
if start < leader or end + int(11.5 * bit) > freq.size:
|
||||
continue
|
||||
before = freq[start - leader:start]
|
||||
after = freq[end:end + leader]
|
||||
# By the middle of each leader rather than by how much of it is
|
||||
# within tolerance. A frequency estimate on a weak signal is noisy
|
||||
# sample by sample and steady in the median, and counting samples
|
||||
# threw away every header the band was not kind to.
|
||||
if not (abs(float(np.median(before)) - LEADER_HZ) < tolerance
|
||||
and abs(float(np.median(after)) - LEADER_HZ) < tolerance):
|
||||
continue
|
||||
# The leader after the break runs 300 ms, then a 30 ms start bit at
|
||||
# 1200 Hz, then the eight data bits.
|
||||
code = _read_vis(freq, rate, end)
|
||||
if code is not None:
|
||||
start_bit = _find_start_bit(freq, rate, end)
|
||||
if start_bit is not None:
|
||||
return code, start_bit + 10 * bit
|
||||
return None
|
||||
|
||||
|
||||
def _find_start_bit(freq: np.ndarray, rate: float, after: int) -> int | None:
|
||||
"""Where the 1200 Hz start bit begins, to the sample.
|
||||
|
||||
Everything downstream is measured from here -- eight VIS bits, then every
|
||||
line of the picture -- so a coarse answer is not good enough. Fifteen
|
||||
milliseconds out is half a bit, and on Martin M1 it is a picture shifted
|
||||
thirty pixels sideways. So the search is coarse to find the bit and then
|
||||
walks back to the first sample that is actually on the tone.
|
||||
"""
|
||||
bit = int(round(VIS_BIT_SECONDS * rate))
|
||||
limit = min(freq.size - 10 * bit, after + int(0.6 * rate))
|
||||
i = after + int(0.20 * rate) # into the 300 ms leader
|
||||
while i < limit:
|
||||
window = freq[i:i + bit]
|
||||
if window.size and abs(float(np.median(window)) - BREAK_HZ) < 60.0:
|
||||
found = _edge_down(freq, i, bit, LEADER_HZ, BREAK_HZ)
|
||||
return i if found is None else found
|
||||
i += max(1, bit // 6)
|
||||
return None
|
||||
|
||||
|
||||
def _edge_down(freq: np.ndarray, near: int, span: int,
|
||||
high: float, low: float) -> int | None:
|
||||
"""Where the tone crossed from ``high`` to ``low``, to the sample.
|
||||
|
||||
A band-pass does not switch between two tones, it slides between them
|
||||
over a few milliseconds, so "the first sample near 1200 Hz" lands wherever
|
||||
the slide happens to cross the tolerance. The midpoint of the slide does
|
||||
not move, so that is what is measured.
|
||||
|
||||
Searched both ways from ``near``, because the coarse scan that found the
|
||||
tone steps in strides and lands on either side of the edge as often as
|
||||
not; looking only backwards found the edge when it overshot and returned
|
||||
the stride itself when it undershot, which is a picture shifted sideways
|
||||
by a whole colour bar.
|
||||
"""
|
||||
midpoint = (high + low) / 2.0
|
||||
start = max(0, near - span)
|
||||
stop = min(freq.size, near + span)
|
||||
if stop - start < 4:
|
||||
return None
|
||||
below = (freq[start:stop] < midpoint).astype(np.int8)
|
||||
# The first crossing that stays crossed, so a single noisy sample in the
|
||||
# middle of the leader is not mistaken for the edge.
|
||||
hold = max(2, min(8, span // 8))
|
||||
kernel = np.ones(hold, dtype=np.int8)
|
||||
steady = np.convolve(below, kernel, mode="valid") == hold
|
||||
where = np.flatnonzero(steady)
|
||||
return start + int(where[0]) if where.size else None
|
||||
|
||||
|
||||
def _read_vis(freq: np.ndarray, rate: float, after: int) -> int | None:
|
||||
start = _find_start_bit(freq, rate, after)
|
||||
if start is None:
|
||||
return None
|
||||
bit = int(round(VIS_BIT_SECONDS * rate))
|
||||
values = []
|
||||
for n in range(1, 9): # bit 0 is the start bit
|
||||
window = freq[start + n * bit:start + (n + 1) * bit]
|
||||
if window.size < bit // 2:
|
||||
return None
|
||||
core = window[bit // 4:3 * bit // 4]
|
||||
middle = float(np.median(core if core.size else window))
|
||||
if abs(middle - BIT_ONE_HZ) < 80.0:
|
||||
values.append(1)
|
||||
elif abs(middle - BIT_ZERO_HZ) < 80.0:
|
||||
values.append(0)
|
||||
else:
|
||||
return None
|
||||
code = sum(v << i for i, v in enumerate(values[:7]))
|
||||
parity = values[7]
|
||||
if (sum(values[:7]) % 2) != parity:
|
||||
return None # even parity, as the standard says
|
||||
return code
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The picture
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _scan(freq: np.ndarray, rate: float, at: float, seconds: float,
|
||||
width: int) -> np.ndarray:
|
||||
"""One colour scan, as ``width`` samples of 0-255."""
|
||||
start = int(round(at * rate))
|
||||
stop = int(round((at + seconds) * rate))
|
||||
if start < 0 or stop > freq.size or stop <= start:
|
||||
return np.zeros(width, dtype=np.uint8)
|
||||
row = resample_to(freq[start:stop], width)
|
||||
scaled = (row - BLACK_HZ) / (WHITE_HZ - BLACK_HZ) * 255.0
|
||||
return np.clip(scaled, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def _yuv_to_rgb(y: np.ndarray, u: np.ndarray, v: np.ndarray) -> np.ndarray:
|
||||
"""The conversion Robot modes are defined against (CCIR 601)."""
|
||||
y = y.astype(np.float64)
|
||||
u = u.astype(np.float64) - 128.0
|
||||
v = v.astype(np.float64) - 128.0
|
||||
r = y + 1.402 * v
|
||||
g = y - 0.344136 * u - 0.714136 * v
|
||||
b = y + 1.772 * u
|
||||
return np.clip(np.stack([r, g, b], axis=-1), 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def _decode_lines(freq: np.ndarray, rate: float, mode: SSTVMode,
|
||||
start: int) -> tuple[np.ndarray, int]:
|
||||
"""Read as many lines as the capture holds. Returns ``(image, lines)``."""
|
||||
rows = np.zeros((mode.height, mode.width, 3), dtype=np.uint8)
|
||||
at = start / rate
|
||||
# Robot 36 sends only one colour difference per line, so a line cannot be
|
||||
# reconstructed until its neighbour has arrived. The scans are collected
|
||||
# first and turned into colour afterwards.
|
||||
held: list[tuple[np.ndarray, np.ndarray]] = []
|
||||
done = 0
|
||||
for line in range(mode.height):
|
||||
if (at + mode.line_seconds) * rate > freq.size:
|
||||
break
|
||||
at = _resync(freq, rate, mode, at)
|
||||
scans: dict[str, np.ndarray] = {}
|
||||
cursor = at
|
||||
for what, seconds in mode.segments():
|
||||
if len(what) == 1:
|
||||
scans[what] = _scan(freq, rate, cursor, seconds, mode.width)
|
||||
cursor += seconds
|
||||
if mode.layout == "GBR":
|
||||
rows[line] = np.stack([scans["R"], scans["G"], scans["B"]],
|
||||
axis=-1)
|
||||
elif mode.layout == "YUV2":
|
||||
rows[line] = _yuv_to_rgb(scans["Y"], scans["U"], scans["V"])
|
||||
elif mode.layout == "YUV":
|
||||
held.append((scans["Y"], scans["C"]))
|
||||
else:
|
||||
grey = scans["Y"]
|
||||
rows[line] = np.stack([grey, grey, grey], axis=-1)
|
||||
at += mode.line_seconds
|
||||
done = line + 1
|
||||
|
||||
if mode.layout == "YUV":
|
||||
# Even lines carry the red difference, odd lines the blue, so each
|
||||
# line borrows the one it is missing from the neighbour that has it.
|
||||
# Both directions, because the neighbour that has it is below on an
|
||||
# even line and above on an odd one.
|
||||
grey = np.full(mode.width, 128, dtype=np.uint8)
|
||||
for line, (luma, chroma) in enumerate(held):
|
||||
if line % 2 == 0:
|
||||
other = held[line + 1][1] if line + 1 < len(held) else grey
|
||||
rows[line] = _yuv_to_rgb(luma, other, chroma)
|
||||
else:
|
||||
other = held[line - 1][1] if line else grey
|
||||
rows[line] = _yuv_to_rgb(luma, chroma, other)
|
||||
return rows, done
|
||||
|
||||
|
||||
def _resync(freq: np.ndarray, rate: float, mode: SSTVMode,
|
||||
at: float) -> float:
|
||||
"""Nudge the line start onto the 1200 Hz sync pulse.
|
||||
|
||||
A receiver's clock and a transmitter's are never the same, and over two
|
||||
minutes a few parts per million becomes a picture that leans. Looking
|
||||
for the pulse at the start of each line costs almost nothing and takes
|
||||
the lean out.
|
||||
"""
|
||||
slack = int(round(0.004 * rate))
|
||||
sync = int(round(mode.sync * rate))
|
||||
# Where in the line the pulse is: at the head for Martin, two thirds of
|
||||
# the way in for Scottie.
|
||||
offset = int(round(mode.offset_of("sync") * rate))
|
||||
start = int(round(at * rate)) + offset - slack
|
||||
stop = start + 2 * slack + sync
|
||||
if start < 0 or stop > freq.size or sync < 4:
|
||||
return at
|
||||
piece = freq[start:stop]
|
||||
# The pulse is the only thing down at 1200 Hz, so find its longest run and
|
||||
# take the far edge of the slide into it, the same way the start bit is
|
||||
# found. Anything shorter than half a pulse is filter wobble.
|
||||
keyed = np.abs(piece - SYNC_HZ) < 150.0
|
||||
if keyed.sum() < sync // 2:
|
||||
return at
|
||||
edges = np.flatnonzero(np.diff(keyed.astype(np.int8)))
|
||||
bounds = np.concatenate(([0], edges + 1, [keyed.size]))
|
||||
best, best_len = None, 0
|
||||
for lo, hi in zip(bounds[:-1], bounds[1:]):
|
||||
if keyed[lo] and hi - lo > best_len:
|
||||
best, best_len = int(lo), int(hi - lo)
|
||||
# Long as well as short is wrong. The VIS stop bit is thirty
|
||||
# milliseconds of the same 1200 Hz and it runs straight into the first
|
||||
# line's pulse, so on line one the two merge into one stretch seven times
|
||||
# the length of a sync -- and following its leading edge starts the whole
|
||||
# picture a colour bar early.
|
||||
if best is None or not (sync // 2 <= best_len <= 2 * sync):
|
||||
return at
|
||||
found = _edge_down(freq, start + best, slack, BLACK_HZ, SYNC_HZ)
|
||||
if found is None:
|
||||
return at
|
||||
# Only ever a nudge. A resync that moved further than the slack allowed
|
||||
# has locked onto something that is not this line's pulse, and following
|
||||
# it would tear the picture rather than straighten it.
|
||||
if abs(found - (int(round(at * rate)) + offset)) > slack:
|
||||
return at
|
||||
return (found - offset) / rate
|
||||
|
||||
|
||||
def decode_sstv(audio: np.ndarray, rate: float) -> ImageDecode | None:
|
||||
"""Find and decode an SSTV picture in a block of audio.
|
||||
|
||||
Returns None when there is no VIS header, which is the usual answer:
|
||||
almost nothing on the air is SSTV, and a decoder without a header to go
|
||||
on has no way to tell a picture from a fax from a whistle.
|
||||
"""
|
||||
audio = np.asarray(audio, dtype=np.float64).ravel()
|
||||
if audio.size < int(rate * 2.0):
|
||||
return None
|
||||
freq = instantaneous_frequency(audio, rate, low=900.0, high=2600.0)
|
||||
if freq.size == 0:
|
||||
return None
|
||||
found = find_vis(freq, rate)
|
||||
if found is None:
|
||||
return None
|
||||
code, start = found
|
||||
mode = VIS_CODES.get(code)
|
||||
if mode is not None and not mode.sync_first:
|
||||
# Scottie sends one pulse before the first line, because its per-line
|
||||
# pulse sits two thirds of the way in and the first line has none
|
||||
# before it otherwise.
|
||||
start += int(round(mode.sync * rate))
|
||||
if mode is None:
|
||||
return ImageDecode(kind="SSTV", note=f"VIS code {code}, "
|
||||
"which is a mode not decoded here")
|
||||
|
||||
rows, lines = _decode_lines(freq, rate, mode, start)
|
||||
if lines < 8:
|
||||
return ImageDecode(kind="SSTV", mode=mode.name,
|
||||
note=f"{mode.name} header, but only {lines} line(s) "
|
||||
"of picture arrived")
|
||||
complete = lines >= mode.height
|
||||
if not complete:
|
||||
rows = rows[:lines]
|
||||
return ImageDecode(
|
||||
ok=True, kind="SSTV", mode=mode.name, width=mode.width,
|
||||
height=rows.shape[0], pixels=rows, complete=complete,
|
||||
seconds=lines * mode.line_seconds,
|
||||
confidence=0.9 if complete else 0.6 + 0.3 * lines / mode.height,
|
||||
note="" if complete else
|
||||
f"the capture ended after {lines} of {mode.height} lines")
|
||||
|
|
@ -600,6 +600,26 @@ never say a word, and identify themselves in a short burst of CW instead.
|
|||
Turn 'Look callsigns up' off to keep the scan entirely offline; callsigns are
|
||||
still found, and named by country from their prefix. Clear 'Map file' to stop
|
||||
writing the map. US licence records are public, and include addresses."""),
|
||||
"12": ("Pictures, aircraft, meters", """
|
||||
Three of the things a receiver can hear are images rather than sounds, and all
|
||||
three arrive as the audio a scan already records: slow-scan television, the
|
||||
NOAA weather satellites on 137 MHz, and the shortwave weather fax stations.
|
||||
Each announces itself -- a VIS header, a pair of line syncs, a phasing signal
|
||||
-- so none of them is guessed at, and what is decoded is written as a PNG
|
||||
beside the recording. saunterbrowse marks those in the list and gives the
|
||||
path. Turn 'Decode pictures' off to skip the work.
|
||||
|
||||
A picture takes minutes rather than seconds, so 'Max record time' has to be
|
||||
long enough or what arrives is the top of one. A partial picture is kept and
|
||||
labelled partial.
|
||||
|
||||
Utility meters on 900 MHz and AcuRite weather sensors on 433 MHz are named
|
||||
rather than reported as hexadecimal, and neither is believed without its own
|
||||
checksum.
|
||||
|
||||
Aircraft are a separate command: `bandsaunter adsb` parks the receiver on
|
||||
1090 MHz. ADS-B is a megabit a second and will not go through a channel
|
||||
twelve and a half kilohertz wide, which is what a scan is made of."""),
|
||||
"11": ("Keys during a scan", """
|
||||
q stop the scan
|
||||
p pause and resume
|
||||
|
|
|
|||
|
|
@ -451,7 +451,10 @@ class ScanDisplay:
|
|||
shown = list(self.hits)[:capacity]
|
||||
for h in shown:
|
||||
extra = ""
|
||||
if h.morse_text:
|
||||
if h.image_kind:
|
||||
extra = (f' [magenta]{escape(h.image_kind)} '
|
||||
f'{escape(h.image_mode)}[/magenta]')
|
||||
elif h.morse_text:
|
||||
extra = f' [yellow]"{escape(h.morse_text.strip()[:32])}"[/yellow]'
|
||||
elif h.data_messages:
|
||||
# What was decoded is the most interesting thing about a data
|
||||
|
|
@ -539,6 +542,12 @@ def print_hit(console: Console, hit: HitRecord) -> None:
|
|||
f"[green]{hit.classification or 'unclassified'}[/green] ({conf})",
|
||||
highlight=False)
|
||||
indent = 26 + _PLAIN_BAND + 2
|
||||
if hit.image_path:
|
||||
console.print(f'{"":>{indent}}[magenta]picture: '
|
||||
f'{escape(hit.image_kind)} {escape(hit.image_mode)} '
|
||||
f'{hit.image_width}x{hit.image_height} → '
|
||||
f'{escape(hit.image_path)}[/magenta]',
|
||||
highlight=False, soft_wrap=True)
|
||||
if hit.morse_text:
|
||||
console.print(f'{"":>{indent}}[yellow]Morse @ {hit.morse_wpm:.0f} WPM: '
|
||||
f'"{escape(hit.morse_text.strip())}"[/yellow]',
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
.\" Generated by packaging/make-man.py -- do not edit by hand.
|
||||
.TH BANDSAUNTER 1 "2026-08-29" "bandsaunter 2026-08-29_01" "User Commands"
|
||||
.TH BANDSAUNTER 1 "2026-08-29" "bandsaunter 2026-08-29_03" "User Commands"
|
||||
.SH NAME
|
||||
bandsaunter \- scan, record and identify radio signals with an RTL-SDR
|
||||
.SH SYNOPSIS
|
||||
|
|
@ -68,8 +68,14 @@ List attached receivers.
|
|||
.B profiles
|
||||
List saved profiles.
|
||||
.TP
|
||||
.B adsb
|
||||
Listen to aircraft on 1090 MHz. See
|
||||
.B AIRCRAFT
|
||||
below.
|
||||
.TP
|
||||
.B analyze
|
||||
Identify a signal in an already-recorded file, or decode Morse from it.
|
||||
Identify a signal in an already-recorded file, decode Morse from it, or write
|
||||
out the picture it turns out to be.
|
||||
.SH OPTIONS
|
||||
.TP
|
||||
.BI \-r " RANGE\fR, \fP" \-\-range " RANGE"
|
||||
|
|
@ -534,6 +540,15 @@ Setting name \fBclassify\fR, default \fByes\fR.
|
|||
Work out what each recording actually is, whether FM voice, AM, single sideband, Morse, a paging system or a digital voice mode, and write it into the log and the filename. Turning it off saves a little processing and leaves you to identify things by ear.
|
||||
.RE
|
||||
.TP
|
||||
.B --images / --no-images
|
||||
Decode pictures \[em] save SSTV, weather satellite and shortwave fax pictures as PNG.
|
||||
.br
|
||||
Setting name \fBdecode_images\fR, default \fByes\fR.
|
||||
.RS
|
||||
.PP
|
||||
Three of the things a receiver can hear are pictures: the weather satellites on 137 MHz, amateur slow-scan television, and the shortwave weather fax stations. All three are images sent as sound, so they arrive in the same recordings everything else does. Each is recognised by its own header rather than guessed at, so this costs a moment per recording and finds nothing where there is nothing. What it does find is written as a PNG beside the audio.
|
||||
.RE
|
||||
.TP
|
||||
.B --morse / --no-morse
|
||||
Decode CW to text \[em] decode keyed carriers as Morse.
|
||||
.br
|
||||
|
|
@ -1079,6 +1094,96 @@ A decode that does have repeats or a checksum behind it outranks the content
|
|||
check: a burst of keying demodulated as FM audio is a buzz, and the speech
|
||||
detector likes a buzz, but a frame whose own checksum came out right is not a
|
||||
statistic.
|
||||
.SH PICTURES
|
||||
Three of the things a receiver can hear are images rather than sounds. All
|
||||
three are analogue, all three encode brightness as a frequency, and all three
|
||||
arrive as the audio the scanner already records \[em] so they are looked for in
|
||||
every recording and written out as PNG beside it.
|
||||
.TP
|
||||
.B SSTV
|
||||
Slow-scan television, on 14.230 MHz and 144.5 MHz and wherever else amateurs
|
||||
send it. A transmission opens with a VIS header that says which mode follows,
|
||||
and that header is what is looked for: no header, no picture. Martin M1 and
|
||||
M2, Scottie S1, S2 and DX, and Robot 36 and 72 are decoded, in colour.
|
||||
.TP
|
||||
.B "APT"
|
||||
The NOAA weather satellites on 137 MHz, which spend a fifteen-minute pass
|
||||
sending one continuous picture. A 2400 Hz tone carries the brightness, two
|
||||
lines a second, 2080 words to a line, with both of the satellite's sensors in
|
||||
every line. The whole frame is written, and each sensor again on its own.
|
||||
.TP
|
||||
.B "HF fax"
|
||||
The weather charts the shortwave stations have sent for decades, in single
|
||||
sideband between 2 and 20 MHz. A transmission opens with a phasing signal \[em]
|
||||
twenty or so lines that are black but for a pulse at the start of each \[em] and
|
||||
that is what says where a line begins and how long one is.
|
||||
.PP
|
||||
None of the three is guessed at, which is what makes it safe to try them on
|
||||
every recording: each is recognised by a header or a phasing signal that
|
||||
nothing else on the air sends. A decoder without one draws static beautifully,
|
||||
and a directory of beautifully rendered static is worse than an empty one.
|
||||
.PP
|
||||
A picture keeps its capture whatever the content check made of it. A satellite
|
||||
is a steady tone with a wobble on it and an SSTV transmission is a whistle:
|
||||
neither is speech and neither has symbol structure, so both were being thrown
|
||||
away as "no signal content" having already been recognised.
|
||||
.PP
|
||||
Pictures take minutes rather than seconds \[em] two minutes for SSTV, fifteen for
|
||||
a satellite pass \[em] so
|
||||
.B \-\-record
|
||||
has to be long enough or what arrives is the top of one. A partial picture is
|
||||
kept and labelled as partial rather than discarded.
|
||||
.PP
|
||||
.BR saunterbrowse (1)
|
||||
marks these in the list and gives the path of the file.
|
||||
.PP
|
||||
GRIB, which is sometimes asked about in the same breath, is not a modulation:
|
||||
it is the binary format the weather models are published in, and it travels by
|
||||
satellite data link and by e-mail rather than as something a receiver can
|
||||
demodulate. Where a decoded byte stream begins with its magic number it is
|
||||
named as such; nothing here fetches or renders one.
|
||||
.SH AIRCRAFT
|
||||
.B bandsaunter adsb
|
||||
parks the receiver on 1090 MHz and reads the Mode S extended squitter that
|
||||
every airliner overhead broadcasts twice a second: the aircraft's address, its
|
||||
callsign, its altitude, its position and its speed, unencrypted, to nobody in
|
||||
particular.
|
||||
.PP
|
||||
It is a command of its own because ADS-B does not fit through the scanner. The
|
||||
signalling is a megabit a second, which needs at least two megasamples a second
|
||||
of raw receiver output; the scan path decimates everything to a channel twelve
|
||||
and a half kilohertz wide long before any decoder sees it.
|
||||
.PP
|
||||
Every frame carries a 24-bit checksum, so there is no threshold here and
|
||||
nothing to disbelieve: a frame either passes or is dropped. A position takes
|
||||
two frames \[em] the encoding sends a fraction of a zone, and one frame alone is
|
||||
ambiguous by hundreds of miles \[em] so an aircraft is placed once an even and an
|
||||
odd frame have both arrived, about a second apart.
|
||||
.PP
|
||||
.B \-\-kml
|
||||
writes what was heard as a map.
|
||||
.B \-\-frames
|
||||
prints each frame as it arrives instead of a running count. An aerial cut for
|
||||
1090 MHz makes the difference between hearing the airport and hearing the
|
||||
county; the whip supplied with a dongle is a quarter of the length it wants.
|
||||
.SH METERS AND SENSORS
|
||||
Two things on the ISM bands are worth naming rather than reporting as
|
||||
hexadecimal.
|
||||
.PP
|
||||
The Itron ERT modules fitted to electricity, gas and water meters across North
|
||||
America broadcast their reading every thirty seconds or so on 902-928 MHz, in
|
||||
the clear, so that a van can drive past and read a street. The message says
|
||||
which meter, what kind, what the register reads and whether the tamper
|
||||
switches have been tripped, and carries a sixteen-bit BCH check.
|
||||
.PP
|
||||
The AcuRite 433.92 MHz outdoor sensors sold with every consumer weather
|
||||
station send temperature, humidity, battery state and a channel letter every
|
||||
sixteen seconds, with a checksum and four parity bits.
|
||||
.PP
|
||||
Neither is guessed at: nothing is reported that has not satisfied its own
|
||||
checksum. Both are implemented from their published descriptions and checked
|
||||
against frames built from the same descriptions, which proves the framing and
|
||||
the arithmetic and is not the same as having held a meter.
|
||||
.SH THE MAP
|
||||
A callsign is looked up in the FCC's published licence data, which gives the
|
||||
licensee, the town, and coordinates. They arrive from three directions and
|
||||
|
|
|
|||
|
|
@ -231,8 +231,12 @@ there is; callook.info holds US licences only.
|
|||
prints every callsign in the directory, who it belongs to, and each frequency
|
||||
and time it was heard on, then exits.
|
||||
.PP
|
||||
US amateur licence records are public by law, and include the licensee's
|
||||
address. That is what is shown.
|
||||
US amateur licence records are public by law, and include the street the
|
||||
licence was issued to, not merely the town. That is what is shown, and what
|
||||
is written into the map: holding it and not saying so would be worse than
|
||||
either showing it or not asking for it.
|
||||
.B \-\-no\-lookup
|
||||
asks for none of it.
|
||||
.SH THE MAP
|
||||
A licence says where its holder is, so a list of callsigns is also a map. The
|
||||
scanner writes one as it runs and
|
||||
|
|
@ -303,6 +307,30 @@ lock-out list in your settings file. The width comes from the
|
|||
setting, so a lock-out is a channel rather than a single point. Locking out a
|
||||
frequency does not delete what has already been recorded on it \[em] the two
|
||||
keys are separate on purpose, and pressing both is the usual thing to do.
|
||||
.SH PICTURES
|
||||
Some recordings are not sounds. Slow-scan television, the NOAA weather
|
||||
satellites and the shortwave weather fax stations all send images as audio,
|
||||
and
|
||||
.BR bandsaunter (1)
|
||||
writes what it decodes as a PNG beside the recording.
|
||||
.PP
|
||||
Those are marked in the list by what they are \[em] "SSTV Martin M1 320x256" \[em]
|
||||
and the panel at the top of the screen gives the path of the file in full,
|
||||
wrapped rather than cut off, because a terminal cannot draw a PNG and saying
|
||||
exactly what to open is the most useful thing left. A weather satellite pass
|
||||
has three files: the whole frame, and each of the satellite's two sensors on
|
||||
its own.
|
||||
.PP
|
||||
.B o
|
||||
prints the picture's path rather than the recording's, for piping into an
|
||||
image viewer:
|
||||
.IP
|
||||
.EX
|
||||
saunterbrowse \-\-filter sstv \-\-list
|
||||
xdg\-open "$(saunterbrowse)"
|
||||
.EE
|
||||
.PP
|
||||
Filing a recording moves its pictures with it, and deleting one deletes them.
|
||||
.SH CALLSIGNS IN MORSE
|
||||
Most stations on the air never say a word. A repeater, a beacon or an
|
||||
unattended transmitter sends its callsign in Morse and stops, and what it
|
||||
|
|
|
|||
|
|
@ -126,8 +126,14 @@ List attached receivers.
|
|||
.B profiles
|
||||
List saved profiles.
|
||||
.TP
|
||||
.B adsb
|
||||
Listen to aircraft on 1090 MHz. See
|
||||
.B AIRCRAFT
|
||||
below.
|
||||
.TP
|
||||
.B analyze
|
||||
Identify a signal in an already-recorded file, or decode Morse from it.
|
||||
Identify a signal in an already-recorded file, decode Morse from it, or write
|
||||
out the picture it turns out to be.
|
||||
.SH OPTIONS
|
||||
.TP
|
||||
.BI \-r " RANGE\fR, \fP" \-\-range " RANGE"
|
||||
|
|
@ -501,6 +507,96 @@ A decode that does have repeats or a checksum behind it outranks the content
|
|||
check: a burst of keying demodulated as FM audio is a buzz, and the speech
|
||||
detector likes a buzz, but a frame whose own checksum came out right is not a
|
||||
statistic.
|
||||
.SH PICTURES
|
||||
Three of the things a receiver can hear are images rather than sounds. All
|
||||
three are analogue, all three encode brightness as a frequency, and all three
|
||||
arrive as the audio the scanner already records \[em] so they are looked for in
|
||||
every recording and written out as PNG beside it.
|
||||
.TP
|
||||
.B SSTV
|
||||
Slow-scan television, on 14.230 MHz and 144.5 MHz and wherever else amateurs
|
||||
send it. A transmission opens with a VIS header that says which mode follows,
|
||||
and that header is what is looked for: no header, no picture. Martin M1 and
|
||||
M2, Scottie S1, S2 and DX, and Robot 36 and 72 are decoded, in colour.
|
||||
.TP
|
||||
.B "APT"
|
||||
The NOAA weather satellites on 137 MHz, which spend a fifteen-minute pass
|
||||
sending one continuous picture. A 2400 Hz tone carries the brightness, two
|
||||
lines a second, 2080 words to a line, with both of the satellite's sensors in
|
||||
every line. The whole frame is written, and each sensor again on its own.
|
||||
.TP
|
||||
.B "HF fax"
|
||||
The weather charts the shortwave stations have sent for decades, in single
|
||||
sideband between 2 and 20 MHz. A transmission opens with a phasing signal \[em]
|
||||
twenty or so lines that are black but for a pulse at the start of each \[em] and
|
||||
that is what says where a line begins and how long one is.
|
||||
.PP
|
||||
None of the three is guessed at, which is what makes it safe to try them on
|
||||
every recording: each is recognised by a header or a phasing signal that
|
||||
nothing else on the air sends. A decoder without one draws static beautifully,
|
||||
and a directory of beautifully rendered static is worse than an empty one.
|
||||
.PP
|
||||
A picture keeps its capture whatever the content check made of it. A satellite
|
||||
is a steady tone with a wobble on it and an SSTV transmission is a whistle:
|
||||
neither is speech and neither has symbol structure, so both were being thrown
|
||||
away as "no signal content" having already been recognised.
|
||||
.PP
|
||||
Pictures take minutes rather than seconds \[em] two minutes for SSTV, fifteen for
|
||||
a satellite pass \[em] so
|
||||
.B \-\-record
|
||||
has to be long enough or what arrives is the top of one. A partial picture is
|
||||
kept and labelled as partial rather than discarded.
|
||||
.PP
|
||||
.BR saunterbrowse (1)
|
||||
marks these in the list and gives the path of the file.
|
||||
.PP
|
||||
GRIB, which is sometimes asked about in the same breath, is not a modulation:
|
||||
it is the binary format the weather models are published in, and it travels by
|
||||
satellite data link and by e-mail rather than as something a receiver can
|
||||
demodulate. Where a decoded byte stream begins with its magic number it is
|
||||
named as such; nothing here fetches or renders one.
|
||||
.SH AIRCRAFT
|
||||
.B bandsaunter adsb
|
||||
parks the receiver on 1090 MHz and reads the Mode S extended squitter that
|
||||
every airliner overhead broadcasts twice a second: the aircraft's address, its
|
||||
callsign, its altitude, its position and its speed, unencrypted, to nobody in
|
||||
particular.
|
||||
.PP
|
||||
It is a command of its own because ADS-B does not fit through the scanner. The
|
||||
signalling is a megabit a second, which needs at least two megasamples a second
|
||||
of raw receiver output; the scan path decimates everything to a channel twelve
|
||||
and a half kilohertz wide long before any decoder sees it.
|
||||
.PP
|
||||
Every frame carries a 24-bit checksum, so there is no threshold here and
|
||||
nothing to disbelieve: a frame either passes or is dropped. A position takes
|
||||
two frames \[em] the encoding sends a fraction of a zone, and one frame alone is
|
||||
ambiguous by hundreds of miles \[em] so an aircraft is placed once an even and an
|
||||
odd frame have both arrived, about a second apart.
|
||||
.PP
|
||||
.B \-\-kml
|
||||
writes what was heard as a map.
|
||||
.B \-\-frames
|
||||
prints each frame as it arrives instead of a running count. An aerial cut for
|
||||
1090 MHz makes the difference between hearing the airport and hearing the
|
||||
county; the whip supplied with a dongle is a quarter of the length it wants.
|
||||
.SH METERS AND SENSORS
|
||||
Two things on the ISM bands are worth naming rather than reporting as
|
||||
hexadecimal.
|
||||
.PP
|
||||
The Itron ERT modules fitted to electricity, gas and water meters across North
|
||||
America broadcast their reading every thirty seconds or so on 902-928 MHz, in
|
||||
the clear, so that a van can drive past and read a street. The message says
|
||||
which meter, what kind, what the register reads and whether the tamper
|
||||
switches have been tripped, and carries a sixteen-bit BCH check.
|
||||
.PP
|
||||
The AcuRite 433.92 MHz outdoor sensors sold with every consumer weather
|
||||
station send temperature, humidity, battery state and a channel letter every
|
||||
sixteen seconds, with a checksum and four parity bits.
|
||||
.PP
|
||||
Neither is guessed at: nothing is reported that has not satisfied its own
|
||||
checksum. Both are implemented from their published descriptions and checked
|
||||
against frames built from the same descriptions, which proves the framing and
|
||||
the arithmetic and is not the same as having held a meter.
|
||||
.SH THE MAP
|
||||
A callsign is looked up in the FCC's published licence data, which gives the
|
||||
licensee, the town, and coordinates. They arrive from three directions and
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
.\" Generated by packaging/make-browse-man.py -- do not edit by hand.
|
||||
.TH SAUNTERBROWSE 1 "2026-08-29" "bandsaunter 2026-08-29_01" "User Commands"
|
||||
.TH SAUNTERBROWSE 1 "2026-08-29" "bandsaunter 2026-08-29_03" "User Commands"
|
||||
.SH NAME
|
||||
saunterbrowse \- read and listen to what a bandsaunter scan collected
|
||||
.SH SYNOPSIS
|
||||
|
|
@ -205,8 +205,12 @@ there is; callook.info holds US licences only.
|
|||
prints every callsign in the directory, who it belongs to, and each frequency
|
||||
and time it was heard on, then exits.
|
||||
.PP
|
||||
US amateur licence records are public by law, and include the licensee's
|
||||
address. That is what is shown.
|
||||
US amateur licence records are public by law, and include the street the
|
||||
licence was issued to, not merely the town. That is what is shown, and what
|
||||
is written into the map: holding it and not saying so would be worse than
|
||||
either showing it or not asking for it.
|
||||
.B \-\-no\-lookup
|
||||
asks for none of it.
|
||||
.SH THE MAP
|
||||
A licence says where its holder is, so a list of callsigns is also a map. The
|
||||
scanner writes one as it runs and
|
||||
|
|
@ -291,6 +295,30 @@ lock-out list in your settings file. The width comes from the
|
|||
setting, so a lock-out is a channel rather than a single point. Locking out a
|
||||
frequency does not delete what has already been recorded on it \[em] the two
|
||||
keys are separate on purpose, and pressing both is the usual thing to do.
|
||||
.SH PICTURES
|
||||
Some recordings are not sounds. Slow-scan television, the NOAA weather
|
||||
satellites and the shortwave weather fax stations all send images as audio,
|
||||
and
|
||||
.BR bandsaunter (1)
|
||||
writes what it decodes as a PNG beside the recording.
|
||||
.PP
|
||||
Those are marked in the list by what they are \[em] "SSTV Martin M1 320x256" \[em]
|
||||
and the panel at the top of the screen gives the path of the file in full,
|
||||
wrapped rather than cut off, because a terminal cannot draw a PNG and saying
|
||||
exactly what to open is the most useful thing left. A weather satellite pass
|
||||
has three files: the whole frame, and each of the satellite's two sensors on
|
||||
its own.
|
||||
.PP
|
||||
.B o
|
||||
prints the picture's path rather than the recording's, for piping into an
|
||||
image viewer:
|
||||
.IP
|
||||
.EX
|
||||
saunterbrowse \-\-filter sstv \-\-list
|
||||
xdg\-open "$(saunterbrowse)"
|
||||
.EE
|
||||
.PP
|
||||
Filing a recording moves its pictures with it, and deleting one deletes them.
|
||||
.SH CALLSIGNS IN MORSE
|
||||
Most stations on the air never say a word. A repeater, a beacon or an
|
||||
unattended transmitter sends its callsign in Morse and stops, and what it
|
||||
|
|
|
|||
118
tests/adsb_gen.py
Normal file
118
tests/adsb_gen.py
Normal file
|
|
@ -0,0 +1,118 @@
|
|||
"""Generate Mode S extended squitter frames, so the ADS-B decoder can be tested.
|
||||
|
||||
Built from the standard rather than from the decoder: the parity is computed
|
||||
here with the same polynomial but the frames are assembled independently, and
|
||||
a decoder that reads back what was put in has read the format rather than
|
||||
agreed with itself.
|
||||
"""
|
||||
import math
|
||||
|
||||
import numpy as np
|
||||
|
||||
from bandsaunter.adsb import CALLSIGN_CHARS, PREAMBLE_US, crc24
|
||||
|
||||
|
||||
def with_parity(payload: bytes) -> bytes:
|
||||
"""A frame with its 24 parity bits appended, as a transmitter sends it."""
|
||||
return payload + crc24(payload).to_bytes(3, "big")
|
||||
|
||||
|
||||
def identification(icao: int, callsign: str, category: int = 0) -> bytes:
|
||||
"""A DF17 type-4 frame: the aircraft saying what it is called."""
|
||||
me = bytearray(7)
|
||||
me[0] = (4 << 3) | (category & 0x07)
|
||||
text = callsign.upper().ljust(8)[:8]
|
||||
bits = ""
|
||||
for ch in text:
|
||||
index = CALLSIGN_CHARS.find(ch)
|
||||
bits += format(index if index >= 0 else 32, "06b")
|
||||
packed = int(bits, 2).to_bytes(6, "big")
|
||||
me[1:7] = packed
|
||||
return with_parity(bytes([17 << 3 | 5]) + icao.to_bytes(3, "big")
|
||||
+ bytes(me))
|
||||
|
||||
|
||||
def _cpr(lat: float, lon: float, odd: bool) -> tuple[int, int]:
|
||||
"""Compact position reporting, the encoding side."""
|
||||
def nl(latitude):
|
||||
if abs(latitude) >= 87.0:
|
||||
return 1
|
||||
if latitude == 0:
|
||||
return 59
|
||||
inner = 1 - (1 - math.cos(math.pi / 30)) / \
|
||||
math.cos(math.radians(abs(latitude))) ** 2
|
||||
return int(math.floor(2 * math.pi / math.acos(max(-1.0, min(1.0, inner)))))
|
||||
|
||||
i = 1 if odd else 0
|
||||
d_lat = 360.0 / (60 - i)
|
||||
j = math.floor(lat / d_lat) + math.floor(
|
||||
0.5 + (lat % d_lat) / d_lat)
|
||||
y = int(round(131072 * ((lat % d_lat) / d_lat)))
|
||||
zones = nl(lat) - i
|
||||
d_lon = 360.0 / zones if zones > 0 else 360.0
|
||||
x = int(round(131072 * ((lon % d_lon) / d_lon)))
|
||||
del j
|
||||
return y & 0x1FFFF, x & 0x1FFFF
|
||||
|
||||
|
||||
def airborne_position(icao: int, lat: float, lon: float, altitude_ft: int,
|
||||
odd: bool) -> bytes:
|
||||
"""A DF17 type-11 frame: where the aircraft is and how high."""
|
||||
encoded = int(round((altitude_ft + 1000) / 25.0))
|
||||
field = format(encoded, "011b")
|
||||
alt = field[:7] + "1" + field[7:] # the Q bit, 25-foot steps
|
||||
y, x = _cpr(lat, lon, odd)
|
||||
me_bits = (format(11, "05b") + "000" + alt + "0"
|
||||
+ ("1" if odd else "0") + format(y, "017b")
|
||||
+ format(x, "017b"))
|
||||
me = int(me_bits, 2).to_bytes(7, "big")
|
||||
return with_parity(bytes([17 << 3 | 5]) + icao.to_bytes(3, "big") + me)
|
||||
|
||||
|
||||
def velocity(icao: int, east_kt: int, north_kt: int,
|
||||
vertical_fpm: int = 0) -> bytes:
|
||||
"""A DF17 type-19 frame: ground speed and climb rate."""
|
||||
ew_sign = "1" if east_kt < 0 else "0"
|
||||
ns_sign = "1" if north_kt < 0 else "0"
|
||||
ew = min(1023, abs(east_kt) + 1)
|
||||
ns = min(1023, abs(north_kt) + 1)
|
||||
rate = min(511, abs(vertical_fpm) // 64 + 1) if vertical_fpm else 0
|
||||
me_bits = (format(19, "05b") + "001" + "00000"
|
||||
+ ew_sign + format(ew, "010b")
|
||||
+ ns_sign + format(ns, "010b")
|
||||
+ "0" + ("1" if vertical_fpm < 0 else "0")
|
||||
+ format(rate, "09b") + "0" * 10)
|
||||
me = int(me_bits[:56].ljust(56, "0"), 2).to_bytes(7, "big")
|
||||
return with_parity(bytes([17 << 3 | 5]) + icao.to_bytes(3, "big") + me)
|
||||
|
||||
|
||||
def modulate(frames, sample_rate: float = 2_000_000.0, gap_us: float = 60.0,
|
||||
amplitude: float = 1.0, noise: float = 0.0,
|
||||
seed: int = 0) -> np.ndarray:
|
||||
"""Turn frames into the magnitude a receiver would see at 1090 MHz.
|
||||
|
||||
Pulse-position: a preamble of four pulses, then one microsecond per bit
|
||||
with the energy in the first half for a one and the second half for a
|
||||
zero.
|
||||
"""
|
||||
rng = np.random.default_rng(seed)
|
||||
per_us = sample_rate / 1e6
|
||||
out = [np.zeros(int(round(gap_us * per_us)))]
|
||||
for frame in frames:
|
||||
bits = "".join(format(b, "08b") for b in frame)
|
||||
span = np.zeros(int(round((8 + len(bits)) * per_us)))
|
||||
for at in PREAMBLE_US:
|
||||
lo = int(round(at * per_us))
|
||||
span[lo:lo + int(round(0.5 * per_us))] = amplitude
|
||||
for i, bit in enumerate(bits):
|
||||
base = (8 + i) * per_us
|
||||
lo = int(round(base if bit == "1" else base + 0.5 * per_us))
|
||||
span[lo:lo + int(round(0.5 * per_us))] = amplitude
|
||||
out.append(span)
|
||||
out.append(np.zeros(int(round(gap_us * per_us))))
|
||||
signal = np.concatenate(out)
|
||||
if noise:
|
||||
signal = signal + noise * np.abs(
|
||||
rng.standard_normal(signal.size)
|
||||
+ 1j * rng.standard_normal(signal.size)) / math.sqrt(2)
|
||||
return signal.astype(np.complex64)
|
||||
198
tests/image_gen.py
Normal file
198
tests/image_gen.py
Normal file
|
|
@ -0,0 +1,198 @@
|
|||
"""Generate SSTV, APT and HF fax audio, so the decoders can be tested.
|
||||
|
||||
Written from the mode specifications rather than from the decoders, so that a
|
||||
decoder agreeing with one of these is evidence rather than a tautology.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
from bandsaunter.sstv import (BIT_ONE_HZ, BIT_ZERO_HZ, BLACK_HZ, BREAK_HZ,
|
||||
LEADER_HZ, SYNC_HZ, WHITE_HZ, SSTVMode)
|
||||
|
||||
|
||||
def fm(segments, rate: float) -> np.ndarray:
|
||||
"""Turn ``(hertz, seconds)`` pairs into one continuous-phase tone."""
|
||||
phase = 0.0
|
||||
out = []
|
||||
for hz, seconds in segments:
|
||||
n = max(1, int(round(seconds * rate)))
|
||||
freq = np.full(n, float(hz)) if np.isscalar(hz) else np.asarray(hz)
|
||||
if freq.size != n:
|
||||
freq = np.interp(np.linspace(0, 1, n),
|
||||
np.linspace(0, 1, freq.size), freq)
|
||||
step = 2.0 * np.pi * freq / rate
|
||||
angles = phase + np.cumsum(step)
|
||||
phase = float(angles[-1] % (2.0 * np.pi))
|
||||
out.append(np.sin(angles))
|
||||
return np.concatenate(out) if out else np.zeros(0)
|
||||
|
||||
|
||||
def _levels(row: np.ndarray) -> np.ndarray:
|
||||
"""A row of 0-255 as the frequencies that carry it."""
|
||||
return BLACK_HZ + np.asarray(row, dtype=np.float64) / 255.0 * (
|
||||
WHITE_HZ - BLACK_HZ)
|
||||
|
||||
|
||||
def vis_header(code: int) -> list:
|
||||
bits = [(code >> i) & 1 for i in range(7)]
|
||||
bits.append(sum(bits) % 2) # even parity
|
||||
out = [(LEADER_HZ, 0.300), (BREAK_HZ, 0.010), (LEADER_HZ, 0.300),
|
||||
(SYNC_HZ, 0.030)]
|
||||
out += [(BIT_ONE_HZ if b else BIT_ZERO_HZ, 0.030) for b in bits]
|
||||
out.append((SYNC_HZ, 0.030)) # stop bit
|
||||
return out
|
||||
|
||||
|
||||
def sstv_audio(mode: SSTVMode, image: np.ndarray, rate: float = 16000.0,
|
||||
lines: int | None = None, lead: float = 0.2,
|
||||
snr_db: float = 40.0, seed: int = 0) -> np.ndarray:
|
||||
"""A whole SSTV transmission: header, then the picture, line by line.
|
||||
|
||||
Built from the mode's own segment list, so that a decoder agreeing with
|
||||
this is agreeing about the published line structure rather than about a
|
||||
second copy of the same guess.
|
||||
"""
|
||||
rng = np.random.default_rng(seed)
|
||||
height = lines if lines is not None else min(mode.height, image.shape[0])
|
||||
segments = [(0.0, lead)] + vis_header(mode.vis)
|
||||
if not mode.sync_first:
|
||||
# Scottie sends one sync pulse before the first line, because its
|
||||
# per-line pulse comes two thirds of the way through.
|
||||
segments.append((SYNC_HZ, mode.sync))
|
||||
for y in range(height):
|
||||
row = image[y % image.shape[0]].astype(np.float64)
|
||||
red, green, blue = row[:, 0], row[:, 1], row[:, 2]
|
||||
luma = 0.299 * red + 0.587 * green + 0.114 * blue
|
||||
chroma_u = 0.564 * (blue - luma) + 128.0
|
||||
chroma_v = 0.713 * (red - luma) + 128.0
|
||||
scans = {"R": red, "G": green, "B": blue, "Y": luma,
|
||||
"U": chroma_u, "V": chroma_v,
|
||||
"C": chroma_v if y % 2 == 0 else chroma_u}
|
||||
for what, seconds in mode.segments():
|
||||
if seconds <= 0:
|
||||
continue
|
||||
if what == "sync":
|
||||
segments.append((SYNC_HZ, seconds))
|
||||
elif len(what) == 1:
|
||||
segments.append((_levels(scans[what]), seconds))
|
||||
else:
|
||||
segments.append((BLACK_HZ, seconds))
|
||||
audio = fm(segments, rate)
|
||||
if snr_db < 60:
|
||||
noise = np.sqrt(np.mean(audio ** 2) / (10 ** (snr_db / 10.0)))
|
||||
audio = audio + noise * rng.standard_normal(audio.size)
|
||||
return audio
|
||||
|
||||
|
||||
def colour_card(width: int = 320, height: int = 256) -> np.ndarray:
|
||||
"""A picture with structure a decoder can be checked against.
|
||||
|
||||
Colour bars across the top, a grey wedge down the middle and a border, so
|
||||
that a picture decoded with the colours swapped, the lines out of order or
|
||||
the geometry wrong looks wrong in a way a test can measure.
|
||||
"""
|
||||
image = np.zeros((height, width, 3), dtype=np.uint8)
|
||||
bars = [(255, 255, 255), (255, 255, 0), (0, 255, 255), (0, 255, 0),
|
||||
(255, 0, 255), (255, 0, 0), (0, 0, 255), (0, 0, 0)]
|
||||
band = height // 2
|
||||
for i, colour in enumerate(bars):
|
||||
lo = i * width // len(bars)
|
||||
hi = (i + 1) * width // len(bars)
|
||||
image[:band, lo:hi] = colour
|
||||
wedge = np.linspace(0, 255, width).astype(np.uint8)
|
||||
image[band:, :] = wedge[None, :, None]
|
||||
image[:, :2] = 255
|
||||
image[:, -2:] = 255
|
||||
return image
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# APT
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
APT_SYNC_A = "0" + "1100" * 7 + "0" * 8 # 1040 Hz square, 39 words
|
||||
APT_SYNC_B = "0" + "11100" * 7 + "0" * 3
|
||||
|
||||
|
||||
def apt_audio(image: np.ndarray, rate: float = 16000.0,
|
||||
snr_db: float = 40.0, seed: int = 0,
|
||||
subcarrier: float = 2400.0) -> np.ndarray:
|
||||
"""A NOAA APT transmission: two lines a second, 2080 words each.
|
||||
|
||||
``image`` is greyscale and becomes channel A; channel B is the same
|
||||
picture inverted, which is what a real pass looks like when one channel is
|
||||
visible and the other infrared.
|
||||
"""
|
||||
rng = np.random.default_rng(seed)
|
||||
words_per_line = 2080
|
||||
lines = image.shape[0]
|
||||
frame = np.zeros((lines, words_per_line), dtype=np.float64)
|
||||
for y in range(lines):
|
||||
row = np.interp(np.linspace(0, 1, 909),
|
||||
np.linspace(0, 1, image.shape[1]),
|
||||
image[y].astype(np.float64))
|
||||
line = np.zeros(words_per_line)
|
||||
line[0:39] = np.array([255.0 if c == "1" else 11.0
|
||||
for c in APT_SYNC_A.ljust(39, "0")])
|
||||
line[39:86] = 11.0 # space A
|
||||
line[86:995] = row
|
||||
line[995:1040] = 128.0 # telemetry A
|
||||
line[1040:1079] = np.array([255.0 if c == "1" else 11.0
|
||||
for c in APT_SYNC_B.ljust(39, "0")])
|
||||
line[1079:1126] = 11.0
|
||||
line[1126:2035] = 255.0 - row
|
||||
line[2035:2080] = 128.0
|
||||
frame[y] = line
|
||||
|
||||
words = frame.reshape(-1)
|
||||
word_rate = 4160.0
|
||||
n = int(round(words.size / word_rate * rate))
|
||||
envelope = np.interp(np.linspace(0, 1, n), np.linspace(0, 1, words.size),
|
||||
words) / 255.0
|
||||
t = np.arange(n) / rate
|
||||
audio = (0.1 + 0.9 * envelope) * np.sin(2 * np.pi * subcarrier * t)
|
||||
if snr_db < 60:
|
||||
noise = np.sqrt(np.mean(audio ** 2) / (10 ** (snr_db / 10.0)))
|
||||
audio = audio + noise * rng.standard_normal(audio.size)
|
||||
return audio
|
||||
|
||||
|
||||
def grey_card(width: int = 909, height: int = 40) -> np.ndarray:
|
||||
"""A greyscale picture with a hard edge and a ramp, for APT and fax."""
|
||||
image = np.zeros((height, width), dtype=np.uint8)
|
||||
image[:, :] = np.linspace(0, 255, width).astype(np.uint8)[None, :]
|
||||
image[height // 3:2 * height // 3, width // 4:width // 2] = 255
|
||||
image[:, ::128] = 0
|
||||
return image
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# HF fax
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def fax_audio(image: np.ndarray, rate: float = 16000.0, lpm: float = 120.0,
|
||||
ioc: int = 576, black: float = 1500.0, white: float = 2300.0,
|
||||
start_seconds: float = 5.0, phasing_lines: int = 20,
|
||||
snr_db: float = 40.0, seed: int = 0) -> np.ndarray:
|
||||
"""A weather fax transmission: start tone, phasing, then the chart."""
|
||||
rng = np.random.default_rng(seed)
|
||||
pixels = int(round(np.pi * ioc))
|
||||
line_seconds = 60.0 / lpm
|
||||
segments = [(black, 0.5)]
|
||||
# The start tone: 300 Hz for 120 lpm, sent as black/white alternation.
|
||||
tone_period = 1.0 / 300.0
|
||||
for _ in range(int(start_seconds / tone_period)):
|
||||
segments += [(white, tone_period / 2), (black, tone_period / 2)]
|
||||
# Phasing: a white pulse at the start of each line, black for the rest.
|
||||
for _ in range(phasing_lines):
|
||||
segments += [(white, line_seconds * 0.05),
|
||||
(black, line_seconds * 0.95)]
|
||||
for y in range(image.shape[0]):
|
||||
row = np.interp(np.linspace(0, 1, pixels),
|
||||
np.linspace(0, 1, image.shape[1]),
|
||||
image[y].astype(np.float64))
|
||||
segments.append((black + row / 255.0 * (white - black), line_seconds))
|
||||
audio = fm(segments, rate)
|
||||
if snr_db < 60:
|
||||
noise = np.sqrt(np.mean(audio ** 2) / (10 ** (snr_db / 10.0)))
|
||||
audio = audio + noise * rng.standard_normal(audio.size)
|
||||
return audio
|
||||
|
|
@ -515,3 +515,59 @@ def test_an_unlisted_callsign_is_cached_too(tmp_path, monkeypatch):
|
|||
CallsignBook._apply(entry, {"status": "INVALID"})
|
||||
assert entry.status == "unlisted"
|
||||
assert entry.version == CACHE_VERSION
|
||||
|
||||
|
||||
# -- the whole address -------------------------------------------------------
|
||||
#
|
||||
# US amateur licence records are public by law and carry the street the
|
||||
# licence was issued to. It is recorded because holding it and not saying so
|
||||
# would be worse than either showing it or not asking for it, and --no-lookup
|
||||
# asks for none of it.
|
||||
|
||||
def test_the_street_is_read_off_the_licence(tmp_path):
|
||||
b = book(tmp_path, {"W1AW": {
|
||||
"status": "VALID", "current": {"callsign": "W1AW"},
|
||||
"name": "ARRL HQ OPERATORS CLUB",
|
||||
"address": {"line1": "225 MAIN ST", "line2": "NEWINGTON, CT 06111"},
|
||||
"location": {"latitude": "41.714", "longitude": "-72.727"}}})
|
||||
entry = b.get("W1AW")
|
||||
b.wait(5.0)
|
||||
assert entry.street == "225 Main St"
|
||||
assert entry.location == "Newington, CT"
|
||||
assert entry.postcode == "06111"
|
||||
assert entry.address == "225 Main St, Newington, CT, 06111"
|
||||
|
||||
|
||||
def test_the_street_is_shown_under_the_callsign(tmp_path):
|
||||
b = book(tmp_path, {"W1AW": {
|
||||
"status": "VALID", "current": {"callsign": "W1AW", "operClass": "CLUB"},
|
||||
"name": "ARRL", "address": {"line1": "225 MAIN ST",
|
||||
"line2": "NEWINGTON, CT 06111"},
|
||||
"location": {"gridsquare": "FN31pr"}}})
|
||||
entry = b.get("W1AW")
|
||||
b.wait(5.0)
|
||||
assert "225 Main St" in entry.details()
|
||||
|
||||
|
||||
def test_an_address_with_nothing_in_it_is_an_empty_string(tmp_path):
|
||||
b = book(tmp_path, {"W1AW": {"status": "INVALID"}})
|
||||
entry = b.get("W1AW")
|
||||
b.wait(5.0)
|
||||
assert entry.address == ""
|
||||
|
||||
|
||||
def test_a_record_cached_before_the_street_is_asked_about_again(tmp_path):
|
||||
"""The cache is versioned so an old entry is not kept with a field empty."""
|
||||
import json
|
||||
import time as _time
|
||||
cache = tmp_path / "calls.json"
|
||||
cache.write_text(json.dumps({"W1AW": {
|
||||
"call": "W1AW", "name": "ARRL", "status": "found",
|
||||
"fetched_at": _time.time(), "version": 2}}))
|
||||
b = StubBook({"W1AW": {
|
||||
"status": "VALID", "current": {"callsign": "W1AW"}, "name": "ARRL",
|
||||
"address": {"line1": "225 MAIN ST", "line2": "NEWINGTON, CT 06111"},
|
||||
"location": {}}}, cache=cache)
|
||||
entry = b.get("W1AW")
|
||||
b.wait(5.0)
|
||||
assert entry.street == "225 Main St"
|
||||
|
|
|
|||
|
|
@ -55,7 +55,8 @@ class StubBook(CallsignBook):
|
|||
def _scanner(tmp_path, mhz: str, **over) -> Scanner:
|
||||
cfg = ScanConfig(ranges=parse_range_list(mhz),
|
||||
output_dir=str(tmp_path), transcribe=False,
|
||||
record_seconds=10, hang_seconds=1.5,
|
||||
record_seconds=over.pop("record_seconds", 10),
|
||||
hang_seconds=1.5,
|
||||
max_runtime_seconds=over.pop("seconds", 45),
|
||||
**over)
|
||||
scanner = Scanner(cfg, device=SimulatedDevice(realtime=False).open())
|
||||
|
|
@ -101,9 +102,14 @@ def test_a_beacon_is_identified_from_the_words_that_survived(tmp_path):
|
|||
"""A continuous beacon is always caught partway through.
|
||||
|
||||
Every capture of one begins and ends in the middle of the message, so
|
||||
what can be said about it is whatever lies between two word gaps.
|
||||
what can be said about it is whatever lies between two word gaps -- which
|
||||
means the capture has to be longer than one repeat before any word is
|
||||
certain to be bounded on both sides. This beacon repeats every eight
|
||||
seconds; a ten-second capture of it is identifiable only by luck, and a
|
||||
twenty-second one always.
|
||||
"""
|
||||
scanner = _scanner(tmp_path, "144.05M-144.15M", seconds=30)
|
||||
scanner = _scanner(tmp_path, "144.05M-144.15M", seconds=50,
|
||||
record_seconds=20)
|
||||
scanner.run()
|
||||
assert "W1AW" in scanner.heard
|
||||
assert set(scanner.heard) == {"W1AW"}, \
|
||||
|
|
@ -112,7 +118,8 @@ def test_a_beacon_is_identified_from_the_words_that_survived(tmp_path):
|
|||
|
||||
def test_the_hit_keeps_both_the_text_and_the_part_it_can_be_identified_from(
|
||||
tmp_path):
|
||||
scanner = _scanner(tmp_path, "144.05M-144.15M", seconds=30)
|
||||
scanner = _scanner(tmp_path, "144.05M-144.15M", seconds=30,
|
||||
record_seconds=20)
|
||||
scanner.run()
|
||||
cw = [h for h in scanner.hits if h.morse_text]
|
||||
assert cw
|
||||
|
|
|
|||
493
tests/test_images.py
Normal file
493
tests/test_images.py
Normal file
|
|
@ -0,0 +1,493 @@
|
|||
"""Pictures off the air: SSTV, weather satellites and shortwave fax.
|
||||
|
||||
Every one of these decoders is written against a published specification, and
|
||||
the generators in ``image_gen.py`` are written against the same one without
|
||||
reference to the decoders. So a picture that comes back matching the one
|
||||
that went in is evidence about the format rather than a decoder agreeing with
|
||||
itself.
|
||||
|
||||
Two things are measured throughout. How close the picture is -- as the share
|
||||
of pixels within a sixth of full scale, because a filter softens every edge
|
||||
and an exact match is not a thing analogue television does. And how often a
|
||||
decoder draws a picture from something that is not one, which has to be never:
|
||||
a scanner that fills a directory with beautifully rendered static is worse
|
||||
than one that draws nothing.
|
||||
"""
|
||||
import struct
|
||||
import zlib
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from bandsaunter.apt import decode_apt, CHANNEL_A, WORDS_PER_LINE
|
||||
from bandsaunter.fax import decode_fax, find_phasing, _levels
|
||||
from bandsaunter.images import (ImageDecode, PNG_SIGNATURE, instantaneous_frequency,
|
||||
resample_to, write_png)
|
||||
from bandsaunter.pictures import find_image
|
||||
from bandsaunter.sstv import MODES, VIS_CODES, decode_sstv, find_vis
|
||||
|
||||
from image_gen import (apt_audio, fax_audio, grey_card, sstv_audio, colour_card)
|
||||
|
||||
FS = 16000.0
|
||||
|
||||
|
||||
def close(got: np.ndarray, want: np.ndarray, within: int = 40) -> float:
|
||||
"""The share of pixels that agree to within ``within`` of 255."""
|
||||
got = np.asarray(got, dtype=float)
|
||||
want = np.asarray(want, dtype=float)[:got.shape[0]]
|
||||
return float((np.abs(got - want[:got.shape[0]]) < within).mean())
|
||||
|
||||
|
||||
def correlation(got: np.ndarray, want: np.ndarray) -> float:
|
||||
a = np.asarray(got, dtype=float)
|
||||
b = np.asarray(want, dtype=float)[:a.shape[0], :a.shape[1]]
|
||||
a = (a - a.mean()) / (a.std() or 1.0)
|
||||
b = (b - b.mean()) / (b.std() or 1.0)
|
||||
return float((a * b).mean())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# PNG
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _read_png(path):
|
||||
"""Read back a PNG this module wrote, so a test can check the pixels.
|
||||
|
||||
Only what write_png produces: eight bits, no interlacing, filter type
|
||||
zero on every row. A general reader would be a second implementation to
|
||||
get wrong.
|
||||
"""
|
||||
raw = path.read_bytes()
|
||||
assert raw[:8] == PNG_SIGNATURE
|
||||
at, chunks = 8, {}
|
||||
idat = b""
|
||||
while at < len(raw):
|
||||
length = struct.unpack(">I", raw[at:at + 4])[0]
|
||||
tag = raw[at + 4:at + 8]
|
||||
body = raw[at + 8:at + 8 + length]
|
||||
if tag == b"IDAT":
|
||||
idat += body
|
||||
else:
|
||||
chunks[tag] = body
|
||||
at += 12 + length
|
||||
width, height, depth, colour = struct.unpack(">IIBB", chunks[b"IHDR"][:10])
|
||||
assert depth == 8
|
||||
per = 3 if colour == 2 else 1
|
||||
data = zlib.decompress(idat)
|
||||
stride = width * per + 1
|
||||
rows = []
|
||||
for y in range(height):
|
||||
line = data[y * stride:(y + 1) * stride]
|
||||
assert line[0] == 0, "only the unfiltered form is written"
|
||||
rows.append(np.frombuffer(line[1:], dtype=np.uint8))
|
||||
out = np.stack(rows)
|
||||
return out.reshape(height, width, 3) if per == 3 else out
|
||||
|
||||
|
||||
def test_a_grey_png_round_trips(tmp_path):
|
||||
want = (np.arange(64 * 40).reshape(40, 64) % 256).astype(np.uint8)
|
||||
write_png(tmp_path / "g.png", want)
|
||||
assert np.array_equal(_read_png(tmp_path / "g.png"), want)
|
||||
|
||||
|
||||
def test_a_colour_png_round_trips(tmp_path):
|
||||
want = colour_card(64, 32)
|
||||
write_png(tmp_path / "c.png", want)
|
||||
assert np.array_equal(_read_png(tmp_path / "c.png"), want)
|
||||
|
||||
|
||||
def test_a_png_is_written_whole_or_not_at_all(tmp_path):
|
||||
"""Atomic, so an interrupted scan cannot leave half a picture."""
|
||||
path = tmp_path / "p.png"
|
||||
write_png(path, np.zeros((4, 4), dtype=np.uint8))
|
||||
assert path.exists()
|
||||
assert not list(tmp_path.glob("*.tmp"))
|
||||
|
||||
|
||||
def test_an_array_that_is_not_a_picture_is_refused(tmp_path):
|
||||
with pytest.raises(ValueError):
|
||||
write_png(tmp_path / "x.png", np.zeros((4, 4, 2)))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SSTV
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_every_mode_adds_up_to_its_published_line_time():
|
||||
"""The number the whole decode hangs on, checked rather than believed.
|
||||
|
||||
A line time a few milliseconds out walks the picture off the bottom of
|
||||
the screen; these are the published figures for each mode.
|
||||
"""
|
||||
for mode in MODES:
|
||||
assert mode.line_seconds * 1000 == pytest.approx(mode.line_ms,
|
||||
abs=0.001), mode.name
|
||||
|
||||
|
||||
def test_the_vis_codes_are_the_assigned_ones():
|
||||
assert VIS_CODES[44].name == "Martin M1"
|
||||
assert VIS_CODES[60].name == "Scottie S1"
|
||||
assert VIS_CODES[8].name == "Robot 36"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mode", MODES, ids=lambda m: m.name)
|
||||
def test_a_transmission_comes_back_as_the_picture_that_was_sent(mode):
|
||||
card = colour_card()
|
||||
got = decode_sstv(sstv_audio(mode, card, FS, lines=20, snr_db=30), FS)
|
||||
assert got is not None and got.ok, got and got.note
|
||||
assert got.mode == mode.name
|
||||
assert got.width == mode.width
|
||||
assert close(got.pixels, card) > 0.88, close(got.pixels, card)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("snr", [30, 20, 12, 9])
|
||||
def test_it_still_reads_through_noise(snr):
|
||||
mode = MODES[0]
|
||||
card = colour_card()
|
||||
got = decode_sstv(sstv_audio(mode, card, FS, lines=16, snr_db=snr), FS)
|
||||
assert got is not None and got.ok, f"{snr} dB: {got and got.note}"
|
||||
assert close(got.pixels, card) > 0.85
|
||||
|
||||
|
||||
def test_a_capture_that_ends_partway_keeps_what_arrived():
|
||||
"""Two minutes is longer than most captures, so partial is the normal case."""
|
||||
mode = MODES[0]
|
||||
got = decode_sstv(sstv_audio(mode, colour_card(), FS, lines=20), FS)
|
||||
assert got.ok and not got.complete
|
||||
assert got.height == 20
|
||||
assert "of 256 lines" in got.note
|
||||
|
||||
|
||||
def test_the_mode_is_read_from_the_header_not_guessed():
|
||||
"""A picture decoded as the wrong mode is a picture and is wrong."""
|
||||
card = colour_card()
|
||||
for mode in (MODES[0], MODES[3], MODES[6]):
|
||||
got = decode_sstv(sstv_audio(mode, card, FS, lines=12), FS)
|
||||
assert got.mode == mode.name
|
||||
|
||||
|
||||
@pytest.mark.parametrize("seed", range(8))
|
||||
def test_nothing_that_is_not_sstv_becomes_a_picture(seed):
|
||||
rng = np.random.default_rng(seed)
|
||||
t = np.arange(int(FS * 4)) / FS
|
||||
for signal in (rng.standard_normal(t.size),
|
||||
np.sin(2 * np.pi * 1900 * t),
|
||||
np.sin(2 * np.pi * 1500 * t) + 0.4 * rng.standard_normal(t.size),
|
||||
np.sin(2 * np.pi * (1700 + 400 * np.sin(2 * np.pi * 3 * t)) * t)):
|
||||
got = decode_sstv(signal, FS)
|
||||
assert got is None or not got.ok
|
||||
|
||||
|
||||
def test_a_header_for_a_mode_this_does_not_know_says_so():
|
||||
from image_gen import fm, vis_header
|
||||
audio = fm([(0.0, 0.2)] + vis_header(2) + [(1500.0, 2.0)], FS)
|
||||
got = decode_sstv(audio, FS)
|
||||
assert got is not None and not got.ok
|
||||
assert "not decoded here" in got.note
|
||||
|
||||
|
||||
def test_the_header_is_found_where_it_actually_is():
|
||||
mode = MODES[0]
|
||||
audio = sstv_audio(mode, colour_card(), FS, lines=4, lead=0.5)
|
||||
freq = instantaneous_frequency(audio, FS, low=900.0, high=2600.0)
|
||||
found = find_vis(freq, FS)
|
||||
assert found is not None
|
||||
code, start = found
|
||||
assert code == mode.vis
|
||||
# lead + leader + break + leader + start bit + 8 bits + stop bit
|
||||
want = 0.5 + 0.300 + 0.010 + 0.300 + 0.030 * 10
|
||||
assert start / FS == pytest.approx(want, abs=0.004)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# APT
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_a_satellite_pass_comes_back_as_the_picture():
|
||||
card = grey_card(909, 40)
|
||||
got = decode_apt(apt_audio(card, FS, snr_db=25), FS, frequency=137.1e6)
|
||||
assert got is not None and got.ok, got and got.note
|
||||
assert got.width == WORDS_PER_LINE
|
||||
channel = np.asarray(got.channels["A"], dtype=float)
|
||||
assert correlation(channel, card) > 0.9
|
||||
|
||||
|
||||
@pytest.mark.parametrize("snr", [30, 20, 12, 6])
|
||||
def test_the_pass_survives_a_weak_signal(snr):
|
||||
card = grey_card(909, 40)
|
||||
got = decode_apt(apt_audio(card, FS, snr_db=snr), FS, frequency=137.1e6)
|
||||
assert got is not None and got.ok
|
||||
assert correlation(np.asarray(got.channels["A"], float), card) > 0.75
|
||||
|
||||
|
||||
def test_both_sensors_are_cut_out_separately():
|
||||
card = grey_card(909, 30)
|
||||
got = decode_apt(apt_audio(card, FS), FS, frequency=137.1e6)
|
||||
assert set(got.channels) == {"A", "B"}
|
||||
assert got.channels["A"].shape[1] == CHANNEL_A[1]
|
||||
|
||||
|
||||
def test_it_is_only_tried_in_the_satellite_band():
|
||||
"""Nothing outside 137 MHz is APT, and looking anyway finds sync in static."""
|
||||
card = grey_card(909, 30)
|
||||
audio = apt_audio(card, FS)
|
||||
assert decode_apt(audio, FS, frequency=146.52e6) is None
|
||||
assert decode_apt(audio, FS, frequency=137.62e6).ok
|
||||
|
||||
|
||||
@pytest.mark.parametrize("seed", range(6))
|
||||
def test_static_does_not_become_a_satellite_pass(seed):
|
||||
rng = np.random.default_rng(seed)
|
||||
t = np.arange(int(FS * 12)) / FS
|
||||
for signal in (rng.standard_normal(t.size),
|
||||
np.sin(2 * np.pi * 2400 * t),
|
||||
np.sin(2 * np.pi * 2400 * t) * (1 + 0.5 * rng.standard_normal(t.size)),
|
||||
np.sin(2 * np.pi * 2400 * t) * (1 + 0.9 * np.sin(2 * np.pi * 7 * t))):
|
||||
got = decode_apt(signal, FS, frequency=137.1e6)
|
||||
assert got is None or not got.ok
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# HF fax
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_a_chart_comes_back_as_the_chart():
|
||||
card = grey_card(800, 30)
|
||||
got = decode_fax(fax_audio(card, FS, snr_db=25), FS)
|
||||
assert got is not None and got.ok, got and got.note
|
||||
assert got.mode.startswith("120 lpm")
|
||||
assert correlation(got.pixels, _stretched(card, got.width)) > 0.95
|
||||
|
||||
|
||||
def _stretched(card, width):
|
||||
return np.stack([resample_to(row.astype(float), width) for row in card])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("snr", [30, 20, 12, 6])
|
||||
def test_the_chart_survives_a_weak_signal(snr):
|
||||
card = grey_card(800, 30)
|
||||
got = decode_fax(fax_audio(card, FS, snr_db=snr), FS)
|
||||
assert got is not None and got.ok
|
||||
assert correlation(got.pixels, _stretched(card, got.width)) > 0.9
|
||||
|
||||
|
||||
def test_the_phasing_signal_gives_the_line_rate():
|
||||
card = grey_card(800, 20)
|
||||
for lpm in (60.0, 120.0, 180.0):
|
||||
audio = fax_audio(card, FS, lpm=lpm, snr_db=30)
|
||||
freq = instantaneous_frequency(audio, FS, low=1200.0, high=2600.0)
|
||||
found = find_phasing(_levels(freq), FS)
|
||||
assert found is not None and found.lpm == lpm
|
||||
|
||||
|
||||
def test_the_start_tone_is_not_mistaken_for_the_chart():
|
||||
"""It is a black-and-white alternation, so its average is mid-grey.
|
||||
|
||||
Taking "the first line that is not black" as the start of the picture
|
||||
made every chart thirty seconds of tone.
|
||||
"""
|
||||
card = grey_card(800, 20)
|
||||
got = decode_fax(fax_audio(card, FS, start_seconds=6.0), FS)
|
||||
assert got.ok
|
||||
assert got.height <= card.shape[0] + 2
|
||||
|
||||
|
||||
@pytest.mark.parametrize("seed", range(6))
|
||||
def test_a_band_with_nothing_on_it_produces_no_chart(seed):
|
||||
rng = np.random.default_rng(seed)
|
||||
t = np.arange(int(FS * 25)) / FS
|
||||
speech = (np.sin(2 * np.pi * (1800 + 300 * np.sin(2 * np.pi * 4 * t)) * t)
|
||||
* (0.5 + 0.5 * np.sin(2 * np.pi * 2.5 * t)))
|
||||
for signal in (rng.standard_normal(t.size), np.sin(2 * np.pi * 1900 * t),
|
||||
speech, speech + 0.3 * rng.standard_normal(t.size)):
|
||||
got = decode_fax(signal, FS)
|
||||
assert got is None or not got.ok
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Which decoder gets offered what
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_sstv_is_recognised_wherever_it_is_heard():
|
||||
audio = sstv_audio(MODES[0], colour_card(), FS, lines=12)
|
||||
for frequency in (14.230e6, 144.5e6, 0.0):
|
||||
got = find_image(audio, FS, frequency=frequency)
|
||||
assert got is not None and got.ok and got.kind == "SSTV"
|
||||
|
||||
|
||||
def test_a_satellite_is_only_looked_for_in_its_own_band():
|
||||
audio = apt_audio(grey_card(909, 30), FS)
|
||||
assert find_image(audio, FS, frequency=137.1e6).kind == "APT"
|
||||
assert find_image(audio, FS, frequency=433.92e6) is None
|
||||
|
||||
|
||||
def test_fax_is_looked_for_on_shortwave_and_in_sideband():
|
||||
audio = fax_audio(grey_card(800, 20), FS)
|
||||
assert find_image(audio, FS, frequency=8.040e6).kind == "HF fax"
|
||||
assert find_image(audio, FS, frequency=14.2e6, mode="usb").kind == "HF fax"
|
||||
assert find_image(audio, FS, frequency=146.52e6, mode="nfm") is None
|
||||
|
||||
|
||||
def test_an_ordinary_recording_is_not_a_picture():
|
||||
rng = np.random.default_rng(0)
|
||||
t = np.arange(int(FS * 6)) / FS
|
||||
speech = np.sin(2 * np.pi * 300 * t) * (0.5 + 0.5 * np.sin(2 * np.pi * 3 * t))
|
||||
assert find_image(speech + 0.1 * rng.standard_normal(t.size), FS,
|
||||
frequency=146.52e6) is None
|
||||
|
||||
|
||||
def test_a_decode_can_be_saved_and_read_back(tmp_path):
|
||||
got = find_image(sstv_audio(MODES[0], colour_card(), FS, lines=12), FS)
|
||||
path = got.save(tmp_path / "picture.png")
|
||||
assert path and (tmp_path / "picture.png").exists()
|
||||
assert _read_png(tmp_path / "picture.png").shape == (got.height, got.width, 3)
|
||||
|
||||
|
||||
def test_an_empty_decode_saves_nothing(tmp_path):
|
||||
assert ImageDecode().save(tmp_path / "nothing.png") == ""
|
||||
assert not (tmp_path / "nothing.png").exists()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Reading one back
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# The point of decoding a picture is that somebody looks at it, and the
|
||||
# browser cannot draw a PNG in a terminal. So what it owes them is to say
|
||||
# plainly that this recording is a picture and exactly where the file is.
|
||||
|
||||
import json
|
||||
import wave
|
||||
|
||||
from rich.console import Console
|
||||
|
||||
from bandsaunter.browse import Browser, Player, scan_directory
|
||||
|
||||
|
||||
def _picture_capture(directory, **over):
|
||||
stem = "0144.500000MHz--2026-08-29_10_00_00-nfm"
|
||||
with wave.open(str(directory / f"{stem}.wav"), "wb") as w:
|
||||
w.setnchannels(1)
|
||||
w.setsampwidth(2)
|
||||
w.setframerate(16000)
|
||||
w.writeframes(b"\0\0" * 16000)
|
||||
write_png(directory / f"{stem}.png", colour_card(320, 24))
|
||||
hit = {"frequency": 144.5e6, "category": "image",
|
||||
"classification": "SSTV (Martin M1)", "image_kind": "SSTV",
|
||||
"image_mode": "Martin M1", "image_width": 320, "image_height": 24,
|
||||
"image_complete": False,
|
||||
"image_path": str(directory / f"{stem}.png"), "confidence": 0.9}
|
||||
hit.update(over)
|
||||
(directory / f"{stem}.json").write_text(json.dumps({"hit": hit}))
|
||||
return directory / f"{stem}.wav"
|
||||
|
||||
|
||||
def _browser(directory, width=100, height=30):
|
||||
return Browser(directory, player=Player([]),
|
||||
console=Console(width=width, height=height,
|
||||
force_terminal=False, no_color=True))
|
||||
|
||||
|
||||
def _frame(browser):
|
||||
with browser.console.capture() as cap:
|
||||
browser.console.print(browser.render())
|
||||
return cap.get()
|
||||
|
||||
|
||||
def test_the_browser_says_a_recording_is_a_picture(tmp_path):
|
||||
_picture_capture(tmp_path)
|
||||
shown = _frame(_browser(tmp_path))
|
||||
assert "picture" in shown
|
||||
assert "SSTV Martin M1 320x24 partial" in shown
|
||||
|
||||
|
||||
def test_it_gives_the_path_in_full(tmp_path):
|
||||
"""In full, wrapped where it has to be: half a path opens nothing."""
|
||||
path = _picture_capture(tmp_path).with_suffix(".png")
|
||||
shown = _frame(_browser(tmp_path))
|
||||
flat = "".join(shown.split()).replace("│", "")
|
||||
assert str(path) in flat
|
||||
|
||||
|
||||
def test_a_short_path_is_on_one_line(tmp_path):
|
||||
path = _picture_capture(tmp_path).with_suffix(".png")
|
||||
assert str(path) in _frame(_browser(tmp_path, width=len(str(path)) + 12))
|
||||
|
||||
|
||||
def test_the_list_row_says_so_too(tmp_path):
|
||||
_picture_capture(tmp_path)
|
||||
browser = _browser(tmp_path)
|
||||
assert "SSTV" in _frame(browser).split("recordings in")[1]
|
||||
|
||||
|
||||
def test_a_picture_can_be_searched_for_by_kind(tmp_path):
|
||||
_picture_capture(tmp_path)
|
||||
browser = _browser(tmp_path)
|
||||
browser.query = "sstv"
|
||||
browser.apply()
|
||||
assert len(browser.view) == 1
|
||||
|
||||
|
||||
def test_o_prints_the_picture_rather_than_the_audio(tmp_path):
|
||||
path = _picture_capture(tmp_path).with_suffix(".png")
|
||||
browser = _browser(tmp_path)
|
||||
assert not browser.handle("o")
|
||||
assert browser.message == str(path)
|
||||
|
||||
|
||||
def test_the_picture_is_found_beside_the_recording_without_a_sidecar_path(
|
||||
tmp_path):
|
||||
"""A directory copied somewhere else keeps the sidecar, not the path in it."""
|
||||
_picture_capture(tmp_path, image_path="/gone/nowhere.png")
|
||||
cap = scan_directory(tmp_path)[0]
|
||||
assert cap.image_path.endswith("-nfm.png")
|
||||
|
||||
|
||||
def test_the_picture_moves_with_the_recording_when_it_is_filed(tmp_path):
|
||||
_picture_capture(tmp_path)
|
||||
browser = _browser(tmp_path)
|
||||
browser.handle("S")
|
||||
moved = {p.suffix for p in (tmp_path / "saved").iterdir()}
|
||||
assert moved == {".wav", ".json", ".png"}
|
||||
|
||||
|
||||
def test_every_channel_of_a_satellite_pass_belongs_to_the_recording(tmp_path):
|
||||
wav = _picture_capture(tmp_path)
|
||||
for name in ("A", "B"):
|
||||
write_png(wav.with_name(wav.stem + f"_{name}.png"),
|
||||
grey_card(64, 8))
|
||||
cap = scan_directory(tmp_path)[0]
|
||||
assert len([p for p in cap.files() if p.suffix == ".png"]) == 3
|
||||
assert len(cap.images) == 3
|
||||
|
||||
|
||||
def test_a_simulated_transmission_becomes_a_file_on_disk(tmp_path):
|
||||
"""The whole path, from a signal on the air to a PNG beside the recording.
|
||||
|
||||
The demo band carries a real Martin M1 transmission, so this exercises
|
||||
detection, the content check that used to throw pictures away, the
|
||||
decode, and the write.
|
||||
"""
|
||||
from bandsaunter.config import ScanConfig
|
||||
from bandsaunter.ranges import parse_range_list
|
||||
from bandsaunter.scanner import Scanner
|
||||
from bandsaunter.simulator import SimulatedDevice
|
||||
|
||||
cfg = ScanConfig(ranges=parse_range_list("144.45M-144.55M"),
|
||||
output_dir=str(tmp_path), transcribe=False,
|
||||
record_seconds=20, hang_seconds=2.0,
|
||||
min_record_seconds=1.0, max_runtime_seconds=60,
|
||||
dwell_seconds=0.05)
|
||||
scanner = Scanner(cfg, device=SimulatedDevice(realtime=False).open())
|
||||
scanner.prepare()
|
||||
scanner.run()
|
||||
|
||||
pictures = [h for h in scanner.hits if h.image_kind]
|
||||
assert pictures, "the SSTV transmission produced no picture"
|
||||
hit = pictures[0]
|
||||
assert hit.image_kind == "SSTV" and hit.image_mode == "Martin M1"
|
||||
assert hit.category == "image"
|
||||
assert hit.kept, "a picture was decoded and then discarded"
|
||||
written = list(tmp_path.glob("*.png"))
|
||||
assert written
|
||||
assert _read_png(written[0]).shape[1] == 320
|
||||
368
tests/test_signals_named.py
Normal file
368
tests/test_signals_named.py
Normal file
|
|
@ -0,0 +1,368 @@
|
|||
"""The formats that say what they are: ADS-B, utility meters, weather sensors.
|
||||
|
||||
What these three have in common is that none of them needs to be believed.
|
||||
Every ADS-B frame carries a 24-bit checksum, every meter message a 16-bit BCH,
|
||||
every AcuRite message a checksum and four parity bits -- so the only question
|
||||
a test can usefully ask is whether the fields come back holding what was put
|
||||
in, and whether anything that is not one of these is ever mistaken for one.
|
||||
|
||||
Each generator is written from the published format rather than from the
|
||||
decoder beside it, so a round trip is evidence about the format.
|
||||
"""
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
import adsb_gen as gen
|
||||
from bandsaunter.adsb import (AircraftRegistry, SAMPLE_RATE, crc24,
|
||||
decode_adsb, decode_frames, global_position)
|
||||
from bandsaunter.ism import (SCM_PREAMBLE, acurite_frame, decode_acurite,
|
||||
decode_ism, decode_scm, scm_frame)
|
||||
from bandsaunter.payload import (MIN_TEXT, hexdump, interpret, read_text,
|
||||
readable)
|
||||
|
||||
RATE = 2_000_000.0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ADS-B
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_the_parity_is_the_one_mode_s_uses():
|
||||
"""A good frame leaves nothing behind; a corrupted one does."""
|
||||
frame = gen.identification(0x4CA1FA, "RYR1234")
|
||||
assert crc24(frame) == 0
|
||||
broken = bytearray(frame)
|
||||
broken[5] ^= 0x01
|
||||
assert crc24(bytes(broken)) != 0
|
||||
|
||||
|
||||
def test_an_aircraft_saying_its_callsign_is_read_back():
|
||||
frames, registry = decode_adsb(
|
||||
gen.modulate([gen.identification(0x4CA1FA, "RYR1234")]), RATE)
|
||||
assert len(frames) == 1
|
||||
assert registry.aircraft["4CA1FA"].callsign == "RYR1234"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("lat,lon", [(51.5, -0.12), (40.7, -74.0),
|
||||
(-33.9, 151.2), (0.5, 0.5),
|
||||
(60.2, 24.9)])
|
||||
def test_a_position_needs_two_frames_and_comes_back_exactly(lat, lon):
|
||||
"""One frame is ambiguous by hundreds of miles; the pair is not."""
|
||||
frames = [gen.airborne_position(0xA0B1C2, lat, lon, 30000, odd=False),
|
||||
gen.airborne_position(0xA0B1C2, lat, lon, 30000, odd=True)]
|
||||
_, registry = decode_adsb(gen.modulate(frames), RATE)
|
||||
craft = registry.aircraft["A0B1C2"]
|
||||
assert craft.latitude == pytest.approx(lat, abs=0.01)
|
||||
assert craft.longitude == pytest.approx(lon, abs=0.01)
|
||||
|
||||
|
||||
def test_one_position_frame_alone_places_nothing():
|
||||
_, registry = decode_adsb(gen.modulate(
|
||||
[gen.airborne_position(0xA0B1C2, 51.5, -0.12, 30000, odd=False)]), RATE)
|
||||
assert not registry.aircraft["A0B1C2"].located
|
||||
|
||||
|
||||
@pytest.mark.parametrize("feet", [0, 1000, 12000, 35000, 43000])
|
||||
def test_altitude_comes_back_in_feet(feet):
|
||||
_, registry = decode_adsb(gen.modulate(
|
||||
[gen.airborne_position(0xABCDEF, 51.5, -0.12, feet, odd=False)]), RATE)
|
||||
assert registry.aircraft["ABCDEF"].altitude_ft == pytest.approx(feet,
|
||||
abs=25)
|
||||
|
||||
|
||||
def test_speed_and_climb_rate_come_back():
|
||||
_, registry = decode_adsb(gen.modulate(
|
||||
[gen.velocity(0x4CA1FA, 250, -180, 1216)]), RATE)
|
||||
craft = registry.aircraft["4CA1FA"]
|
||||
assert craft.ground_speed_kt == pytest.approx(308, abs=3)
|
||||
assert craft.track_deg == pytest.approx(126, abs=2)
|
||||
assert craft.vertical_rate_fpm == pytest.approx(1216, abs=64)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("noise", [0.0, 0.05, 0.15, 0.30])
|
||||
def test_every_frame_survives_a_noisy_receiver(noise):
|
||||
frames = [gen.identification(0x4CA1FA, "RYR1234"),
|
||||
gen.airborne_position(0x4CA1FA, 51.5, -0.12, 35000, odd=False),
|
||||
gen.airborne_position(0x4CA1FA, 51.5, -0.12, 35000, odd=True),
|
||||
gen.velocity(0x4CA1FA, 250, -180, 1216)]
|
||||
got, registry = decode_adsb(gen.modulate(frames, noise=noise, seed=3),
|
||||
RATE)
|
||||
assert len(got) == len(frames)
|
||||
assert registry.aircraft["4CA1FA"].located
|
||||
|
||||
|
||||
def test_many_aircraft_are_kept_apart():
|
||||
frames = []
|
||||
for i, icao in enumerate((0x4CA1FA, 0xA0B1C2, 0x3C6444, 0x780102)):
|
||||
frames.append(gen.identification(icao, f"FLT{i}"))
|
||||
frames.append(gen.airborne_position(icao, 50 + i, -1 - i, 30000, False))
|
||||
frames.append(gen.airborne_position(icao, 50 + i, -1 - i, 30000, True))
|
||||
_, registry = decode_adsb(gen.modulate(frames), RATE)
|
||||
assert len(registry) == 4
|
||||
assert all(craft.located for craft in registry.aircraft.values())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("seed", range(6))
|
||||
def test_silence_does_not_become_aircraft(seed):
|
||||
"""A frame of all zeros satisfies the checksum, and silence is one."""
|
||||
rng = np.random.default_rng(seed)
|
||||
for samples in (np.zeros(int(RATE // 4), dtype=np.complex64),
|
||||
(rng.standard_normal(int(RATE // 4))
|
||||
+ 1j * rng.standard_normal(int(RATE // 4))
|
||||
).astype(np.complex64) * 0.1):
|
||||
assert decode_frames(samples, RATE) == []
|
||||
|
||||
|
||||
def test_a_rate_too_low_to_see_a_bit_is_refused():
|
||||
"""One megabit a second cannot be read at one megasample a second."""
|
||||
iq = gen.modulate([gen.identification(0x4CA1FA, "TEST")])
|
||||
assert decode_frames(iq, SAMPLE_RATE / 2) == []
|
||||
|
||||
|
||||
def test_a_corrupted_frame_is_dropped_rather_than_reported():
|
||||
frame = bytearray(gen.identification(0x4CA1FA, "RYR1234"))
|
||||
frame[6] ^= 0xFF
|
||||
assert decode_frames(gen.modulate([bytes(frame)]), RATE) == []
|
||||
|
||||
|
||||
def test_a_position_straddling_a_zone_boundary_is_refused():
|
||||
"""Two frames from different latitude bands cannot be combined.
|
||||
|
||||
The longitude zones get wider towards the poles, so a pair that came from
|
||||
either side of a boundary would be resolved against two different grids
|
||||
and land somewhere neither of them was.
|
||||
"""
|
||||
from bandsaunter.adsb import Frame
|
||||
# These two resolve to latitudes with a different number of longitude
|
||||
# zones, which is exactly the case the standard says cannot be combined.
|
||||
even = Frame(cpr_lat=2048, cpr_lon=0, cpr_odd=False)
|
||||
odd = Frame(cpr_lat=12288, cpr_lon=0, cpr_odd=True)
|
||||
assert global_position(even, odd) is None
|
||||
# And a pair that does not straddle one still resolves.
|
||||
lat, lon = gen._cpr(51.5, -0.12, odd=False)
|
||||
olat, olon = gen._cpr(51.5, -0.12, odd=True)
|
||||
assert global_position(Frame(cpr_lat=lat, cpr_lon=lon),
|
||||
Frame(cpr_lat=olat, cpr_lon=olon,
|
||||
cpr_odd=True)) is not None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Utility meters
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("meter,reading,ert", [
|
||||
(12345678, 987654, 4), (0x3FFFFFF, 16777215, 5), (1, 0, 11),
|
||||
(555555, 42, 12), (67108863, 1, 13),
|
||||
])
|
||||
def test_a_meter_reading_comes_back_as_it_was_sent(meter, reading, ert):
|
||||
got = decode_scm(scm_frame(meter, reading, ert))
|
||||
assert got is not None
|
||||
assert got.identifier == str(meter)
|
||||
assert ("reading", str(reading)) in got.fields
|
||||
|
||||
|
||||
@pytest.mark.parametrize("ert,name", [(4, "electricity"), (5, "gas"),
|
||||
(11, "water"), (12, "gas")])
|
||||
def test_the_kind_of_meter_is_named(ert, name):
|
||||
got = decode_scm(scm_frame(7, 7, ert))
|
||||
assert got.device == f"{name} meter"
|
||||
|
||||
|
||||
def test_a_meter_type_that_is_not_known_is_numbered_not_guessed():
|
||||
got = decode_scm(scm_frame(7, 7, ert_type=2))
|
||||
assert "type 2" in got.device
|
||||
|
||||
|
||||
def test_the_tamper_flags_are_reported():
|
||||
got = decode_scm(scm_frame(7, 7, tamper_physical=2, tamper_encoder=1))
|
||||
names = dict(got.fields)
|
||||
assert names["physical tamper"] == "2"
|
||||
assert names["encoder tamper"] == "1"
|
||||
|
||||
|
||||
def test_a_message_is_found_after_whatever_came_before_it():
|
||||
bits = "0101101" + scm_frame(4242, 999) + "1101"
|
||||
assert decode_scm(bits).identifier == "4242"
|
||||
|
||||
|
||||
def test_a_meter_message_with_a_bit_wrong_is_refused():
|
||||
bits = list(scm_frame(12345678, 987654))
|
||||
bits[40] = "1" if bits[40] == "0" else "0"
|
||||
assert decode_scm("".join(bits)) is None
|
||||
|
||||
|
||||
def test_a_meter_number_too_large_is_an_error_not_a_wrong_reading():
|
||||
with pytest.raises(ValueError):
|
||||
scm_frame(1 << 27, 5)
|
||||
with pytest.raises(ValueError):
|
||||
scm_frame(5, 1 << 25)
|
||||
|
||||
|
||||
def test_random_bits_behind_a_real_preamble_are_refused():
|
||||
"""The preamble is 21 bits and turns up; the checksum is what matters."""
|
||||
rng = np.random.default_rng(0)
|
||||
accepted = 0
|
||||
for _ in range(3000):
|
||||
body = "".join(rng.integers(0, 2, 75).astype(str))
|
||||
if decode_scm(SCM_PREAMBLE + body) is not None:
|
||||
accepted += 1
|
||||
assert accepted == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AcuRite
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("sensor,celsius,humidity,channel", [
|
||||
(0x1234, 21.5, 48, "A"), (0x0001, -20.0, 5, "C"), (0x3FFF, 45.3, 100, "B"),
|
||||
(0x2AAA, 0.0, 50, "D"), (0x0555, -39.9, 1, "A"),
|
||||
])
|
||||
def test_a_sensor_reading_comes_back_as_it_was_sent(sensor, celsius,
|
||||
humidity, channel):
|
||||
got = decode_acurite(acurite_frame(sensor, celsius, humidity, channel))
|
||||
assert got is not None
|
||||
assert got.identifier == f"{sensor:04X}"
|
||||
fields = dict(got.fields)
|
||||
assert fields["temperature"] == f"{celsius:.1f} C"
|
||||
assert fields["humidity"] == f"{humidity}%"
|
||||
assert fields["channel"] == channel
|
||||
|
||||
|
||||
def test_a_flat_battery_is_reported_and_a_good_one_is_not():
|
||||
good = decode_acurite(acurite_frame(0x1234, 20.0, 50, "A", False))
|
||||
flat = decode_acurite(acurite_frame(0x1234, 20.0, 50, "A", True))
|
||||
assert "battery" not in dict(good.fields)
|
||||
assert dict(flat.fields)["battery"] == "low"
|
||||
|
||||
|
||||
def test_a_message_is_found_wherever_in_the_burst_it_starts():
|
||||
bits = "10110" + acurite_frame(0x0ABC, 12.3, 77, "B") + "0011"
|
||||
assert decode_acurite(bits).identifier == "0ABC"
|
||||
|
||||
|
||||
def test_a_sensor_message_with_a_bit_wrong_is_refused():
|
||||
bits = list(acurite_frame(0x1234, 21.5, 48, "A"))
|
||||
bits[20] = "1" if bits[20] == "0" else "0"
|
||||
assert decode_acurite("".join(bits)) is None
|
||||
|
||||
|
||||
def test_a_reading_outside_what_the_sensor_can_report_is_refused():
|
||||
"""The checksum can be satisfied by a message the hardware cannot send."""
|
||||
from bandsaunter.ism import _parity
|
||||
data = [0x00, 0x01, 0x04, 0x7F, 0x0F, 0x7F]
|
||||
for i in range(2, 6):
|
||||
if _parity(data[i]) != 1:
|
||||
data[i] |= 0x80
|
||||
data.append(sum(data[:6]) & 0xFF)
|
||||
bits = "".join(format(b, "08b") for b in data)
|
||||
assert decode_acurite(bits) is None
|
||||
|
||||
|
||||
def test_the_band_decides_which_is_tried_first_and_nothing_else():
|
||||
meter = scm_frame(4242, 999)
|
||||
assert decode_ism(meter, frequency=915e6).kind == "SCM"
|
||||
assert decode_ism(meter, frequency=433.92e6).kind == "SCM"
|
||||
|
||||
|
||||
def test_neither_reads_a_message_out_of_nothing():
|
||||
rng = np.random.default_rng(1)
|
||||
accepted = sum(1 for _ in range(5000)
|
||||
if decode_ism("".join(rng.integers(0, 2, 96).astype(str)))
|
||||
is not None)
|
||||
# The seven-byte sensor message is checked at every offset, so the bar is
|
||||
# a rate rather than zero. What reaches this in the scanner has already
|
||||
# had to arrive identically several times over.
|
||||
assert accepted / 5000 < 0.01
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Reading a payload
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _encode(text: str, width: int = 8, msb: bool = True, before: int = 0,
|
||||
after: int = 0, seed: int = 0) -> str:
|
||||
rng = np.random.default_rng(seed)
|
||||
bits = "".join(rng.integers(0, 2, before).astype(str))
|
||||
for ch in text:
|
||||
chunk = format(ord(ch), f"0{width}b")
|
||||
bits += chunk if msb else chunk[::-1]
|
||||
return bits + "".join(rng.integers(0, 2, after).astype(str))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("message", [
|
||||
"HELLO WORLD", "ENGINE 4 RESPOND", "the quick brown fox",
|
||||
"Meeting at seven", "BATTERY LOW", "unit twelve en route",
|
||||
])
|
||||
@pytest.mark.parametrize("width,msb", [(8, True), (8, False), (7, True)])
|
||||
def test_text_in_a_packet_is_read_out(message, width, msb):
|
||||
got = read_text(_encode(message, width, msb))
|
||||
assert got is not None, message
|
||||
assert got.text.lower() in message.lower()
|
||||
|
||||
|
||||
def test_text_is_found_behind_a_preamble_and_an_address():
|
||||
got = read_text(_encode("STATION OPEN", before=13, after=7, seed=4))
|
||||
assert got is not None and "STATION OPEN" in got.text
|
||||
|
||||
|
||||
def test_random_payloads_are_almost_never_read_as_text():
|
||||
"""Printability is not evidence, and this is the measurement that says so.
|
||||
|
||||
Every framing at every offset in both bit orders is about forty readings
|
||||
of each packet, and seven-bit values are printable three times in four, so
|
||||
a bar set on printability alone called 64% of random payloads text.
|
||||
"""
|
||||
rng = np.random.default_rng(5)
|
||||
hits = 0
|
||||
total = 0
|
||||
for n_bits in (24, 32, 48, 64, 96, 128, 192, 256, 512):
|
||||
for _ in range(250):
|
||||
total += 1
|
||||
if read_text("".join(rng.integers(0, 2, n_bits).astype(str))):
|
||||
hits += 1
|
||||
assert hits / total < 0.01, f"{100 * hits / total:.1f}% read as text"
|
||||
|
||||
|
||||
def test_a_run_of_one_case_and_no_vowels_is_not_text():
|
||||
assert read_text(_encode("XKCDZQRT")) is None
|
||||
|
||||
|
||||
def test_the_bytes_are_laid_out_with_their_characters_beside_them():
|
||||
lines = hexdump(_encode("ABCDEFGH"))
|
||||
assert lines[0].startswith("0000")
|
||||
assert "|ABCDEFGH|" in lines[0]
|
||||
|
||||
|
||||
def test_a_trailing_part_byte_is_shown_as_bits_not_padded():
|
||||
"""Padding four bits to a byte invents four zeroes nobody sent."""
|
||||
lines = hexdump("1" * 12)
|
||||
assert "4 bit(s): 1111" in lines[-1]
|
||||
|
||||
|
||||
def test_a_remote_control_gets_its_fields_named():
|
||||
readings = interpret("1" * 20 + "0100", encoding="PWM")
|
||||
fields = [r for r in readings if r.kind == "fields"]
|
||||
assert fields and fields[0].how.startswith("EV1527")
|
||||
assert dict(fields[0].fields)["button"] == "B"
|
||||
|
||||
|
||||
def test_every_payload_gets_at_least_its_bytes_back():
|
||||
lines = readable("10110010" * 4)
|
||||
assert lines and lines[0].startswith("0000")
|
||||
|
||||
|
||||
def test_a_payload_that_says_what_it_is_is_named():
|
||||
"""GRIB gets asked about as though it were a modulation.
|
||||
|
||||
It is not: it is the format weather models are published in, and it
|
||||
arrives by satellite data link and by e-mail. Where a byte stream begins
|
||||
with its magic number that is worth saying; nothing here renders one.
|
||||
"""
|
||||
bits = "".join(format(b, "08b") for b in b"GRIB\x00\x00\x00\x02payload!")
|
||||
named = [r for r in interpret(bits) if r.kind == "fields"]
|
||||
assert named and "GRIB" in named[0].line()
|
||||
|
||||
|
||||
def test_an_ordinary_payload_claims_no_format():
|
||||
bits = "".join(format(b, "08b") for b in b"\x01\x02\x03\x04\x05\x06\x07\x08")
|
||||
assert not [r for r in interpret(bits) if r.kind == "fields"]
|
||||
Loading…
Add table
Add a link
Reference in a new issue