8eb3bdbb86
A callsign is a flight number rather than a leg, and the free registers hold one route per number, so an aircraft over Arizona kept being handed a hop between two airports in Texas. Nothing on the air settles it: ADS-B carries no origin or destination. A commercial schedule service does know, because it holds the day's actual movements. Four are wired up and all four are optional: FlightAware AeroAPI, Flightradar24, OAG and Cirium. Each is asked before the free databases and each answers for the moment the aircraft was overhead rather than for the flight number in general, so the leg chosen is the one that was in the air. With no keys set nothing changes at all: a source with no key is skipped rather than asked and refused, and the free databases answer as before. Keys come from the environment and are never written to the settings file, because a settings file is meant to be copied between machines and pasted into a message asking for help, and an API key is not. There is a test that holds that line. None of the four has been run against its live service, since each wants a paid account. They were written from the published response shapes and are tested against those shapes, so each reader finds what it recognises and returns nothing otherwise: a service that has changed since costs a route rather than a scan. Cirium's plain departureTime is local and carries no offset, so the UTC field is preferred where it is there -- reading the local one as UTC is up to half a day out, which is exactly far enough to pick the wrong leg of the same number. Reading now happens inside the same guard as asking, as an answer shaped differently from the documented one is the failure most likely to actually happen. And the map. The zoom is now chosen from how wide the picture is rather than from the area alone, with half again over the width fetched and averaged down, since a downscaled tile is sharp and an upscaled one is not. The window fetches a little more world than it shows so panning does not leave the ground blank, and now fetches that bigger piece at the bigger piece's own size: rendering it into the window's own pixels and stretching it back was a fifth of an upscale over the whole map, which is what a sharp map looks like when it looks blurred. The comment in the fetcher said the opposite of what the code did, which is how it stayed hidden. At 1920 by 1080 over a hundred miles the tiles now hold about 1.6 times the pixels the window wants. At 3840 by 2160 the tile budget is reached, the zoom stops climbing and the map is enlarged after all; a smaller radius buys the detail back, and somebody else's tile server is not a thing to fetch a thousand tiles from for one picture. The README says so rather than implying otherwise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016PsWPTweCT6pwxKngvVxcg
796 lines
33 KiB
Python
796 lines
33 KiB
Python
"""Writing down what the aircraft said, and reading it back afterwards.
|
||
|
||
A receiver hears an aircraft for a few minutes as it crosses the sky and then
|
||
never again. What it heard is worth keeping: the position reports are a flight
|
||
path, and a flight path an hour old is the only way to see where anything went.
|
||
|
||
Two files come out of a listening session. The log is JSON Lines -- one object
|
||
per frame, in the order they arrived, with the raw hex of every frame kept
|
||
alongside what was read out of it, so nothing decoded here is the last word and
|
||
a better decoder can be run over the same evening later. The report is for
|
||
reading: one block per aircraft, saying what it is, who flies it, where it went
|
||
and how far.
|
||
|
||
The log is the input to the map. Everything the animation needs -- who, where,
|
||
when, how fast -- is in it, and nothing else is needed to draw the whole
|
||
evening again.
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import json
|
||
import math
|
||
import time
|
||
from statistics import median
|
||
from dataclasses import dataclass, field, replace
|
||
from datetime import datetime
|
||
from pathlib import Path
|
||
|
||
__all__ = ["Fix", "Track", "FlightLog", "read_logs", "report",
|
||
"write_kml", "LOG_VERSION", "EARTH_NM", "SPEED_UNITS",
|
||
"speed_label", "in_speed", "distance_label", "in_distance",
|
||
"DEFAULT_SPEED_UNIT", "centre_of", "read_position",
|
||
"box_around", "within", "recheck", "implied_speed_kt",
|
||
"MAX_GROUND_SPEED_KT"]
|
||
|
||
LOG_VERSION = 1
|
||
|
||
# One nautical mile is a minute of latitude, which is what makes it the unit
|
||
# every other number in this file is already in.
|
||
EARTH_NM = 3440.065
|
||
|
||
# What the aircraft says is knots and nautical miles, because that is what
|
||
# the standard sends and what the log therefore holds. Anything else is a
|
||
# conversion done at the moment of showing it to somebody, so the recorded
|
||
# data is never the one that had arithmetic applied to it.
|
||
#
|
||
# name: (speed label, knots -> this, distance label, nautical miles -> this)
|
||
SPEED_UNITS: dict[str, tuple[str, float, str, float]] = {
|
||
"knots": ("kt", 1.0, "nm", 1.0),
|
||
"mph": ("mph", 1.150779, "mi", 1.150779),
|
||
"kph": ("km/h", 1.852, "km", 1.852),
|
||
}
|
||
DEFAULT_SPEED_UNIT = "knots"
|
||
|
||
|
||
def speed_unit(name: str) -> tuple[str, float, str, float]:
|
||
"""The conversion for a unit name, falling back to what aircraft use."""
|
||
return SPEED_UNITS.get((name or "").strip().lower(),
|
||
SPEED_UNITS[DEFAULT_SPEED_UNIT])
|
||
|
||
|
||
def speed_label(name: str = DEFAULT_SPEED_UNIT) -> str:
|
||
"""What to write after a speed: kt, mph or km/h."""
|
||
return speed_unit(name)[0]
|
||
|
||
|
||
def in_speed(knots: float, name: str = DEFAULT_SPEED_UNIT) -> float:
|
||
"""A speed in knots, as the unit asked for."""
|
||
return float(knots) * speed_unit(name)[1]
|
||
|
||
|
||
def distance_label(name: str = DEFAULT_SPEED_UNIT) -> str:
|
||
"""The distance unit that goes with a speed unit: nm, mi or km.
|
||
|
||
Miles an hour with distances in nautical miles would be two different
|
||
miles on one picture, which is worse than either on its own.
|
||
"""
|
||
return speed_unit(name)[2]
|
||
|
||
|
||
def in_distance(nm: float, name: str = DEFAULT_SPEED_UNIT) -> float:
|
||
return float(nm) * speed_unit(name)[3]
|
||
|
||
|
||
@dataclass
|
||
class Fix:
|
||
"""One position report: where an aircraft was, and when."""
|
||
|
||
at: float = 0.0
|
||
latitude: float = 0.0
|
||
longitude: float = 0.0
|
||
altitude_ft: int = 0
|
||
ground_speed_kt: float = 0.0
|
||
track_deg: float = 0.0
|
||
vertical_rate_fpm: int = 0
|
||
|
||
def as_json(self) -> dict:
|
||
return {"t": round(self.at, 3),
|
||
"lat": round(self.latitude, 6),
|
||
"lon": round(self.longitude, 6),
|
||
"alt_ft": self.altitude_ft,
|
||
"gs_kt": round(self.ground_speed_kt, 1),
|
||
"track": round(self.track_deg, 1),
|
||
"vs_fpm": self.vertical_rate_fpm}
|
||
|
||
|
||
def distance_nm(lat1: float, lon1: float, lat2: float, lon2: float) -> float:
|
||
"""Great-circle distance in nautical miles."""
|
||
p1, p2 = math.radians(lat1), math.radians(lat2)
|
||
dp = p2 - p1
|
||
dl = math.radians(lon2 - lon1)
|
||
a = math.sin(dp / 2) ** 2 + math.cos(p1) * math.cos(p2) * math.sin(dl / 2) ** 2
|
||
return 2 * EARTH_NM * math.asin(min(1.0, math.sqrt(a)))
|
||
|
||
|
||
def bearing_deg(lat1: float, lon1: float, lat2: float, lon2: float) -> float:
|
||
"""Initial bearing from one point to another, in degrees true."""
|
||
p1, p2 = math.radians(lat1), math.radians(lat2)
|
||
dl = math.radians(lon2 - lon1)
|
||
y = math.sin(dl) * math.cos(p2)
|
||
x = math.cos(p1) * math.sin(p2) - math.sin(p1) * math.cos(p2) * math.cos(dl)
|
||
return math.degrees(math.atan2(y, x)) % 360.0
|
||
|
||
|
||
def move(lat: float, lon: float, bearing: float, nm: float) -> tuple[float, float]:
|
||
"""Where you get to going ``nm`` miles on a bearing: dead reckoning.
|
||
|
||
Used to fill the gaps. An aircraft heard once a minute is out of sight
|
||
for fifty-nine seconds of it, and it did not stop while nobody was
|
||
listening -- it carried on at the speed and heading it last reported.
|
||
"""
|
||
if nm == 0:
|
||
return lat, lon
|
||
p1 = math.radians(lat)
|
||
l1 = math.radians(lon)
|
||
d = nm / EARTH_NM
|
||
theta = math.radians(bearing)
|
||
p2 = math.asin(math.sin(p1) * math.cos(d)
|
||
+ math.cos(p1) * math.sin(d) * math.cos(theta))
|
||
l2 = l1 + math.atan2(math.sin(theta) * math.sin(d) * math.cos(p1),
|
||
math.cos(d) - math.sin(p1) * math.sin(p2))
|
||
return math.degrees(p2), (math.degrees(l2) + 540) % 360 - 180
|
||
|
||
|
||
def centre_of(tracks) -> tuple[float, float] | None:
|
||
"""Where the receiver most likely is, from everything it heard.
|
||
|
||
The median rather than the mean, because a handful of wrong positions
|
||
would drag an average halfway across a continent and cannot move a
|
||
median at all. A receiver hears aircraft all around it, so the middle
|
||
of what it heard is very close to where it is standing.
|
||
"""
|
||
lats = [f.latitude for t in tracks for f in t.fixes]
|
||
lons = [f.longitude for t in tracks for f in t.fixes]
|
||
if not lats:
|
||
return None
|
||
return median(lats), median(lons)
|
||
|
||
|
||
def read_position(text: str) -> tuple[float, float] | None:
|
||
"""A "lat,lon" pair as two numbers, or None if it is not one."""
|
||
try:
|
||
lat, lon = (float(x) for x in str(text).split(",", 1))
|
||
except (TypeError, ValueError):
|
||
return None
|
||
if not (-90.0 <= lat <= 90.0 and -180.0 <= lon <= 180.0):
|
||
return None
|
||
return lat, lon
|
||
|
||
|
||
def box_around(lat: float, lon: float, radius_nm: float) -> tuple:
|
||
"""The south, west, north, east of a circle of this radius.
|
||
|
||
A degree of latitude is sixty nautical miles everywhere; a degree of
|
||
longitude is sixty times the cosine of the latitude, which is why the
|
||
box is wider in degrees the further north it is drawn.
|
||
"""
|
||
span_lat = radius_nm / 60.0
|
||
span_lon = radius_nm / 60.0 / max(0.02, math.cos(math.radians(lat)))
|
||
return (max(-90.0, lat - span_lat), lon - span_lon,
|
||
min(90.0, lat + span_lat), lon + span_lon)
|
||
|
||
|
||
# Above anything with a transponder on it, so a fast aircraft is never called
|
||
# an error. Concorde cruised at 1150 kt.
|
||
MAX_GROUND_SPEED_KT = 2000.0
|
||
|
||
# Past this, two positions are not worth comparing: an aircraft out of range
|
||
# for five minutes may legitimately reappear anywhere it could have flown.
|
||
TRUST_SECONDS = 300.0
|
||
|
||
# A move shorter than this is never called an error, however little time it
|
||
# took. Positions arrive twice a second and are stamped to the millisecond,
|
||
# so two of them a thousandth of a second apart imply thousands of knots
|
||
# across a few yards -- and a compact-position error is never a few yards,
|
||
# it is a different longitude zone. Measured over one night's recording the
|
||
# two are cleanly separated: every real jump was over fifty miles and every
|
||
# false one under one.
|
||
MIN_JUMP_NM = 2.0
|
||
|
||
# How many recent positions each one is weighed against. Positions arrive
|
||
# about twice a second, so this is a few seconds of history -- enough to step
|
||
# over a run of bad decodes, and short enough that the work stays linear in
|
||
# the length of the log.
|
||
_CHAIN_WINDOW = 40
|
||
|
||
|
||
|
||
def implied_speed_kt(a: "Fix", b: "Fix") -> float:
|
||
"""How fast something would have to move to be in both places."""
|
||
gap = b.at - a.at
|
||
if gap <= 0:
|
||
return 0.0
|
||
return distance_nm(a.latitude, a.longitude,
|
||
b.latitude, b.longitude) / gap * 3600.0
|
||
|
||
|
||
def _reachable(a: "Fix", b: "Fix") -> bool:
|
||
gap = b.at - a.at
|
||
if gap <= 0 or gap > TRUST_SECONDS:
|
||
return True # too long ago to argue with
|
||
if distance_nm(a.latitude, a.longitude,
|
||
b.latitude, b.longitude) <= MIN_JUMP_NM:
|
||
return True # too small a move to be a bad decode
|
||
return implied_speed_kt(a, b) <= MAX_GROUND_SPEED_KT
|
||
|
||
|
||
def recheck(tracks) -> tuple[list, int]:
|
||
"""Drop the positions an aircraft could not have been in.
|
||
|
||
Returns the tracks and how many fixes went. For logs recorded before
|
||
the decoder checked the age of a compact-position pair: an even frame
|
||
kept from ten minutes ago, read against a fresh odd one, decodes to a
|
||
place on the wrong side of the world, and that is written down as
|
||
confidently as a real position.
|
||
|
||
An aircraft that goes out of range and comes back is not an error, so
|
||
two positions are only ever compared while they are close in time; past
|
||
five minutes the aircraft may legitimately be anywhere it could have
|
||
flown to, and nothing is rejected. Inside that window, though, a
|
||
position that cannot be reached from the one before it is wrong however
|
||
many equally wrong ones follow it -- three bad decodes in a row can land
|
||
in the same wrong place and agree with each other perfectly.
|
||
"""
|
||
out, dropped = [], 0
|
||
for track in tracks:
|
||
kept = _consistent(track.fixes)
|
||
dropped += len(track.fixes) - len(kept)
|
||
out.append(track if len(kept) == len(track.fixes)
|
||
else replace(track, fixes=kept))
|
||
return out, dropped
|
||
|
||
|
||
def _consistent(fixes: list) -> list:
|
||
"""The longest run of positions that tell one story.
|
||
|
||
Walking forward and keeping whatever is reachable from the last position
|
||
kept is the obvious way and the wrong one: it only takes one bad fix to
|
||
become the reference, and then every real position afterwards is five
|
||
hundred miles from where the aircraft is supposed to be and gets thrown
|
||
away instead. Measured on a real recording that discarded a fifth of
|
||
everything, most of it the truth.
|
||
|
||
So no position is the reference. Every chain of positions that could
|
||
describe one aeroplane is considered, and the longest is the answer --
|
||
the errors are outnumbered by definition, because they are errors.
|
||
"""
|
||
real = [f for f in fixes
|
||
if -90.0 <= f.latitude <= 90.0 and -180.0 <= f.longitude <= 180.0]
|
||
if len(real) < 2:
|
||
return real
|
||
# Two positions that contradict each other are not two positions: one of
|
||
# them is wrong and there is nothing to say which, so the later one goes.
|
||
# What comes out is at least a story, which is the whole promise here.
|
||
# Each position, against the recent ones rather than all of them: they
|
||
# arrive twice a second and nothing further back than this window can
|
||
# still be argued with anyway.
|
||
best = [1] * len(real)
|
||
came_from = [-1] * len(real)
|
||
for i, fix in enumerate(real):
|
||
for j in range(max(0, i - _CHAIN_WINDOW), i):
|
||
if best[j] + 1 > best[i] and _reachable(real[j], fix):
|
||
best[i] = best[j] + 1
|
||
came_from[i] = j
|
||
end = max(range(len(real)), key=lambda i: best[i])
|
||
chain = []
|
||
while end >= 0:
|
||
chain.append(real[end])
|
||
end = came_from[end]
|
||
chain.reverse()
|
||
return chain
|
||
|
||
|
||
def within(tracks, lat: float, lon: float, radius_nm: float) -> list:
|
||
"""The tracks with everything outside the radius dropped.
|
||
|
||
Per fix rather than per aircraft, because a track is rarely all good or
|
||
all bad: one wrong position in the middle of a real flight would
|
||
otherwise take the whole flight off the map with it, or keep the whole
|
||
map stretched to reach it.
|
||
"""
|
||
out = []
|
||
for track in tracks:
|
||
kept = [f for f in track.fixes
|
||
if distance_nm(lat, lon, f.latitude, f.longitude) <= radius_nm]
|
||
if len(kept) == len(track.fixes):
|
||
out.append(track)
|
||
continue
|
||
near = replace(track, fixes=kept)
|
||
out.append(near)
|
||
return out
|
||
|
||
|
||
@dataclass
|
||
class Track:
|
||
"""One aircraft's evening: who it was and everywhere it was seen."""
|
||
|
||
icao: str = ""
|
||
callsign: str = ""
|
||
fixes: list[Fix] = field(default_factory=list)
|
||
frames: int = 0
|
||
first_seen: float = 0.0
|
||
last_seen: float = 0.0
|
||
altitudes: list[tuple[float, int]] = field(default_factory=list)
|
||
|
||
@property
|
||
def located(self) -> bool:
|
||
return bool(self.fixes)
|
||
|
||
@property
|
||
def name(self) -> str:
|
||
return self.callsign or self.icao
|
||
|
||
@property
|
||
def seconds(self) -> float:
|
||
return max(0.0, self.last_seen - self.first_seen)
|
||
|
||
@property
|
||
def distance_nm(self) -> float:
|
||
"""How far it was watched flying, along the path it actually took."""
|
||
total = 0.0
|
||
for a, b in zip(self.fixes, self.fixes[1:]):
|
||
total += distance_nm(a.latitude, a.longitude, b.latitude, b.longitude)
|
||
return total
|
||
|
||
@property
|
||
def altitude_range(self) -> tuple[int, int]:
|
||
heights = [f.altitude_ft for f in self.fixes if f.altitude_ft] or \
|
||
[alt for _, alt in self.altitudes if alt]
|
||
return (min(heights), max(heights)) if heights else (0, 0)
|
||
|
||
@property
|
||
def top_speed_kt(self) -> float:
|
||
speeds = [f.ground_speed_kt for f in self.fixes if f.ground_speed_kt]
|
||
return max(speeds) if speeds else 0.0
|
||
|
||
def bounds(self) -> tuple[float, float, float, float]:
|
||
"""South, west, north, east: the box this track needs on a map."""
|
||
lats = [f.latitude for f in self.fixes]
|
||
lons = [f.longitude for f in self.fixes]
|
||
return (min(lats), min(lons), max(lats), max(lons))
|
||
|
||
def at(self, when: float, stale: float = 300.0) -> Fix | None:
|
||
"""Where the aircraft was at a given moment, or None if unknown.
|
||
|
||
Between two reports the position is interpolated along the time
|
||
between them, which is what makes a stream of fixes into movement.
|
||
After the last report it is dead-reckoned from the speed and heading
|
||
it was last flying, but only for a while: an aircraft that has not
|
||
been heard for five minutes has flown forty miles and is a guess, not
|
||
a position, and a map that keeps drawing it is inventing an aeroplane.
|
||
"""
|
||
if not self.fixes or when < self.fixes[0].at - 1e-9:
|
||
return None
|
||
last = self.fixes[-1]
|
||
if when > last.at:
|
||
gap = when - last.at
|
||
if gap > stale:
|
||
return None
|
||
if last.ground_speed_kt <= 0:
|
||
return last
|
||
lat, lon = move(last.latitude, last.longitude, last.track_deg,
|
||
last.ground_speed_kt * gap / 3600.0)
|
||
return Fix(at=when, latitude=lat, longitude=lon,
|
||
altitude_ft=last.altitude_ft + int(
|
||
last.vertical_rate_fpm * gap / 60.0),
|
||
ground_speed_kt=last.ground_speed_kt,
|
||
track_deg=last.track_deg,
|
||
vertical_rate_fpm=last.vertical_rate_fpm)
|
||
before = self.fixes[0]
|
||
for after in self.fixes[1:]:
|
||
if after.at >= when:
|
||
span = after.at - before.at
|
||
if span <= 0:
|
||
return after
|
||
part = (when - before.at) / span
|
||
return _between(before, after, part)
|
||
before = after
|
||
return last
|
||
|
||
def trail(self, until: float, seconds: float = 0.0) -> list[Fix]:
|
||
"""The path flown up to a moment, for drawing behind the aircraft."""
|
||
start = until - seconds if seconds else float("-inf")
|
||
out = [f for f in self.fixes if start <= f.at <= until]
|
||
now = self.at(until)
|
||
if now is not None and (not out or out[-1].at < now.at):
|
||
out.append(now)
|
||
return out
|
||
|
||
def describe(self, unit: str = DEFAULT_SPEED_UNIT) -> str:
|
||
"""One line: who, where from and to, how far and how high."""
|
||
bits = [self.icao]
|
||
if self.callsign:
|
||
bits.append(self.callsign)
|
||
if self.located:
|
||
first, last = self.fixes[0], self.fixes[-1]
|
||
bits.append(f"{first.latitude:.3f},{first.longitude:.3f} → "
|
||
f"{last.latitude:.3f},{last.longitude:.3f}")
|
||
bits.append(f"{in_distance(self.distance_nm, unit):.0f} "
|
||
f"{distance_label(unit)}")
|
||
low, high = self.altitude_range
|
||
if high:
|
||
bits.append(f"{low}–{high} ft" if low != high else f"{high} ft")
|
||
if self.top_speed_kt:
|
||
bits.append(f"{in_speed(self.top_speed_kt, unit):.0f} "
|
||
f"{speed_label(unit)}")
|
||
bits.append(f"{self.frames} frames")
|
||
return " ".join(bits)
|
||
|
||
|
||
def _between(before: Fix, after: Fix, part: float) -> Fix:
|
||
"""A position part of the way from one fix to the next.
|
||
|
||
Straight-line in latitude and longitude rather than along a great circle:
|
||
over the fifty miles between two reports from the same aircraft the
|
||
difference is a few metres, and pretending otherwise would be arithmetic
|
||
for its own sake.
|
||
"""
|
||
def mix(a: float, b: float) -> float:
|
||
return a + (b - a) * part
|
||
|
||
lon_a, lon_b = before.longitude, after.longitude
|
||
if abs(lon_b - lon_a) > 180.0: # across the date line
|
||
lon_b += 360.0 if lon_b < lon_a else -360.0
|
||
lon = (mix(lon_a, lon_b) + 540.0) % 360.0 - 180.0
|
||
heading = before.track_deg or bearing_deg(before.latitude, before.longitude,
|
||
after.latitude, after.longitude)
|
||
return Fix(at=mix(before.at, after.at),
|
||
latitude=mix(before.latitude, after.latitude), longitude=lon,
|
||
altitude_ft=int(round(mix(before.altitude_ft or after.altitude_ft,
|
||
after.altitude_ft or before.altitude_ft))),
|
||
ground_speed_kt=mix(before.ground_speed_kt, after.ground_speed_kt),
|
||
track_deg=heading,
|
||
vertical_rate_fpm=before.vertical_rate_fpm)
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# Writing, as the frames arrive
|
||
# ---------------------------------------------------------------------------
|
||
|
||
class FlightLog:
|
||
"""A JSON Lines record of everything heard, written frame by frame.
|
||
|
||
Flushed after every frame on purpose. A listening session ends when the
|
||
aeroplanes stop or the operator gets bored and presses control-C, and a
|
||
log that only reached the disk on a clean shutdown would be empty exactly
|
||
when it was most wanted.
|
||
"""
|
||
|
||
def __init__(self, path, receiver: str = "", frequency: float = 0.0,
|
||
sample_rate: float = 0.0, started: float = 0.0):
|
||
self.path = Path(path)
|
||
self.frames = 0
|
||
self.started = started or time.time()
|
||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||
self._file = self.path.open("a", encoding="utf8")
|
||
self._write({"log": "bandsaunter-adsb", "version": LOG_VERSION,
|
||
"started": round(self.started, 3),
|
||
"started_local": datetime.fromtimestamp(
|
||
self.started).strftime("%Y-%m-%d %H:%M:%S"),
|
||
"frequency": frequency, "sample_rate": sample_rate,
|
||
"receiver": receiver})
|
||
|
||
def _write(self, body: dict) -> None:
|
||
self._file.write(json.dumps(body, separators=(",", ":"),
|
||
ensure_ascii=False) + "\n")
|
||
self._file.flush()
|
||
|
||
def append(self, frame, craft=None, when: float = 0.0) -> None:
|
||
"""Record one frame: what arrived, and what was made of it.
|
||
|
||
The raw hex goes down whatever was understood, because the frame is
|
||
the evidence and everything else in the line is an opinion about it.
|
||
"""
|
||
body: dict = {"t": round(when or time.time(), 3),
|
||
"icao": frame.icao, "df": frame.df,
|
||
"tc": frame.type_code,
|
||
"hex": frame.data.hex().upper()}
|
||
if frame.callsign:
|
||
body["callsign"] = frame.callsign
|
||
if frame.altitude_ft:
|
||
body["alt_ft"] = frame.altitude_ft
|
||
if frame.ground_speed_kt:
|
||
body["gs_kt"] = round(frame.ground_speed_kt, 1)
|
||
body["track"] = round(frame.track_deg, 1)
|
||
if frame.vertical_rate_fpm:
|
||
body["vs_fpm"] = frame.vertical_rate_fpm
|
||
# The position comes from the aircraft rather than the frame: it takes
|
||
# an even frame and an odd one, and this is the moment the pair became
|
||
# a place.
|
||
if craft is not None and craft.located:
|
||
body["lat"] = round(craft.latitude, 6)
|
||
body["lon"] = round(craft.longitude, 6)
|
||
if craft.callsign and "callsign" not in body:
|
||
body["callsign"] = craft.callsign
|
||
self.frames += 1
|
||
self._write(body)
|
||
|
||
def close(self) -> None:
|
||
try:
|
||
self._file.close()
|
||
except OSError:
|
||
pass
|
||
|
||
def __enter__(self) -> "FlightLog":
|
||
return self
|
||
|
||
def __exit__(self, *exc) -> None:
|
||
self.close()
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# Reading, afterwards
|
||
# ---------------------------------------------------------------------------
|
||
|
||
def read_logs(paths) -> list[Track]:
|
||
"""Every aircraft in one or more logs, as tracks in time order.
|
||
|
||
A repeated position is dropped. An aircraft sitting on a stand reports
|
||
the same latitude and longitude twice a second for an hour, and a track
|
||
with seven thousand identical points in it is seven thousand times the
|
||
work to draw and no more informative than the one point.
|
||
"""
|
||
tracks: dict[str, Track] = {}
|
||
# What the aircraft last said about itself on a line that carried no
|
||
# position. Speed, heading and height arrive in their own frames, and
|
||
# they are facts about the aeroplane that the next position fix inherits.
|
||
said: dict[str, dict] = {}
|
||
for path in ([paths] if isinstance(paths, (str, Path)) else paths):
|
||
for line in _lines(Path(path)):
|
||
icao = str(line.get("icao") or "")
|
||
if not icao:
|
||
continue # the header, or a line about the run
|
||
track = tracks.get(icao)
|
||
if track is None:
|
||
track = Track(icao=icao, first_seen=float(line.get("t") or 0.0))
|
||
tracks[icao] = track
|
||
when = float(line.get("t") or 0.0)
|
||
track.frames += 1
|
||
track.first_seen = min(track.first_seen or when, when)
|
||
track.last_seen = max(track.last_seen, when)
|
||
if line.get("callsign"):
|
||
track.callsign = str(line["callsign"])
|
||
if line.get("alt_ft"):
|
||
track.altitudes.append((when, int(line["alt_ft"])))
|
||
# What it last said about itself, from whichever frame said it.
|
||
# Once an aircraft has been placed every line carries its
|
||
# position, so this cannot be gathered only from the lines that
|
||
# have none: it has to be updated from every line, or a velocity
|
||
# frame would be given the altitude the aircraft had when it was
|
||
# first heard and the track would flick between the two.
|
||
latest = said.setdefault(icao, {})
|
||
for key in ("alt_ft", "gs_kt", "track", "vs_fpm"):
|
||
if line.get(key):
|
||
latest[key] = line[key]
|
||
if "lat" not in line or "lon" not in line:
|
||
continue
|
||
fix = Fix(at=when, latitude=float(line["lat"]),
|
||
longitude=float(line["lon"]),
|
||
altitude_ft=int(line.get("alt_ft")
|
||
or latest.get("alt_ft") or 0),
|
||
ground_speed_kt=float(line.get("gs_kt")
|
||
or latest.get("gs_kt") or 0.0),
|
||
track_deg=float(line.get("track")
|
||
or latest.get("track") or 0.0),
|
||
vertical_rate_fpm=int(line.get("vs_fpm")
|
||
or latest.get("vs_fpm") or 0))
|
||
last = track.fixes[-1] if track.fixes else None
|
||
if last is not None and last.latitude == fix.latitude and \
|
||
last.longitude == fix.longitude:
|
||
_fill(last, fix)
|
||
continue
|
||
track.fixes.append(fix)
|
||
for track in tracks.values():
|
||
track.fixes.sort(key=lambda f: f.at)
|
||
_carry_forward(track)
|
||
return sorted(tracks.values(), key=lambda t: (t.first_seen, t.icao))
|
||
|
||
|
||
def _lines(path: Path):
|
||
try:
|
||
with path.open(encoding="utf8") as handle:
|
||
for line in handle:
|
||
line = line.strip()
|
||
if not line:
|
||
continue
|
||
try:
|
||
body = json.loads(line)
|
||
except ValueError:
|
||
continue # a half-written last line after a crash
|
||
if isinstance(body, dict):
|
||
yield body
|
||
except OSError:
|
||
return
|
||
|
||
|
||
def _fill(into: Fix, extra: Fix) -> None:
|
||
"""Fold a repeated position into the one already kept."""
|
||
into.altitude_ft = extra.altitude_ft or into.altitude_ft
|
||
into.ground_speed_kt = extra.ground_speed_kt or into.ground_speed_kt
|
||
into.track_deg = extra.track_deg or into.track_deg
|
||
into.vertical_rate_fpm = extra.vertical_rate_fpm or into.vertical_rate_fpm
|
||
|
||
|
||
def _carry_forward(track: Track) -> None:
|
||
"""Give every fix a speed, a heading and a height.
|
||
|
||
Position, velocity and altitude arrive in different frames, so a position
|
||
fix on its own often has no speed attached. What the aircraft last said
|
||
still applies -- it is the same aeroplane a second later -- and where it
|
||
never said anything at all the heading between one fix and the next is
|
||
what it was doing by definition.
|
||
"""
|
||
speed = heading = 0.0
|
||
altitude = climb = 0
|
||
for i, fix in enumerate(track.fixes):
|
||
if fix.ground_speed_kt:
|
||
speed, heading = fix.ground_speed_kt, fix.track_deg
|
||
else:
|
||
fix.ground_speed_kt, fix.track_deg = speed, heading
|
||
if fix.altitude_ft:
|
||
altitude = fix.altitude_ft
|
||
else:
|
||
fix.altitude_ft = altitude
|
||
if fix.vertical_rate_fpm:
|
||
climb = fix.vertical_rate_fpm
|
||
else:
|
||
fix.vertical_rate_fpm = climb
|
||
if not fix.track_deg and i + 1 < len(track.fixes):
|
||
nxt = track.fixes[i + 1]
|
||
fix.track_deg = bearing_deg(fix.latitude, fix.longitude,
|
||
nxt.latitude, nxt.longitude)
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# The readable report
|
||
# ---------------------------------------------------------------------------
|
||
|
||
def report(tracks: list[Track], book=None, title: str = "",
|
||
unit: str = DEFAULT_SPEED_UNIT) -> list[str]:
|
||
"""One block per aircraft, in the order they were first heard.
|
||
|
||
``book`` is a FlightBook, or None. Everything it can add is printed under
|
||
what was heard rather than mixed into it, so it is always clear which
|
||
lines came off the air and which came from a website.
|
||
"""
|
||
out: list[str] = []
|
||
if title:
|
||
out += [title, "=" * len(title), ""]
|
||
heard = [t for t in tracks if t.frames]
|
||
if not heard:
|
||
return out + ["nothing heard."]
|
||
span = (min(t.first_seen for t in heard), max(t.last_seen for t in heard))
|
||
out += [f"{len(heard)} aircraft, {sum(t.frames for t in heard)} frames, "
|
||
f"{_clock(span[0])} to {_clock(span[1])}",
|
||
f"{sum(1 for t in heard if t.located)} of them placed on the map",
|
||
""]
|
||
for track in heard:
|
||
out.append(f"{track.icao}"
|
||
+ (f" {track.callsign}" if track.callsign else ""))
|
||
entry = book.get(track.icao, track.callsign) if book is not None else None
|
||
if entry is not None and track.located and hasattr(book, "resolve"):
|
||
here = track.fixes[len(track.fixes) // 2]
|
||
entry = book.resolve(entry, here.latitude, here.longitude)
|
||
if entry is not None:
|
||
for line in entry.details():
|
||
# The route is printed as one line further down: "from" and
|
||
# "to" mean the airports here and the ends of the track a few
|
||
# lines below, and one page cannot hold both meanings.
|
||
if not line.startswith(("address:", "callsign:", "from:", "to:")):
|
||
out.append(f" {line}")
|
||
if entry.country and not entry.owner_country:
|
||
out.append(f" registered in: {entry.country}")
|
||
if entry.route:
|
||
out.append(f" route: {entry.route}{_doubtful(entry, track)}")
|
||
out.append(f" heard: {_clock(track.first_seen)} to "
|
||
f"{_clock(track.last_seen)} "
|
||
f"({_span(track.seconds)}, {track.frames} frames)")
|
||
if track.located:
|
||
first, last = track.fixes[0], track.fixes[-1]
|
||
out.append(f" from: {first.latitude:.4f}, {first.longitude:.4f}")
|
||
out.append(f" to: {last.latitude:.4f}, {last.longitude:.4f}")
|
||
out.append(f" flew: {in_distance(track.distance_nm, unit):.1f} "
|
||
f"{distance_label(unit)} over "
|
||
f"{len(track.fixes)} positions")
|
||
low, high = track.altitude_range
|
||
if high:
|
||
out.append(f" altitude: {low:,} to {high:,} ft"
|
||
if low != high else f" altitude: {high:,} ft")
|
||
if track.top_speed_kt:
|
||
out.append(f" speed: up to "
|
||
f"{in_speed(track.top_speed_kt, unit):.0f} "
|
||
f"{speed_label(unit)}")
|
||
out.append("")
|
||
return out
|
||
|
||
|
||
def _doubtful(entry, track: Track) -> str:
|
||
"""A note where an aircraft cannot have been flying the route given.
|
||
|
||
Said rather than hidden. The route is what the register holds for that
|
||
flight number and is worth writing down; what it is not is a statement
|
||
about where this aeroplane was going, and the difference matters enough
|
||
to spell out.
|
||
"""
|
||
if not track.located:
|
||
return ""
|
||
from .flights import route_fits
|
||
|
||
here = track.fixes[len(track.fixes) // 2]
|
||
if route_fits(entry, here.latitude, here.longitude):
|
||
return ""
|
||
return " (scheduled for this flight number; this aircraft was " \
|
||
"nowhere near it)"
|
||
|
||
|
||
def _clock(when: float) -> str:
|
||
return datetime.fromtimestamp(when).strftime("%Y-%m-%d %H:%M:%S") \
|
||
if when else "—"
|
||
|
||
|
||
def _span(seconds: float) -> str:
|
||
if seconds < 90:
|
||
return f"{seconds:.0f} s"
|
||
minutes, secs = divmod(int(seconds), 60)
|
||
if minutes < 90:
|
||
return f"{minutes} min {secs:02d} s"
|
||
hours, minutes = divmod(minutes, 60)
|
||
return f"{hours} h {minutes:02d} min"
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# The same thing for Google Earth
|
||
# ---------------------------------------------------------------------------
|
||
|
||
def write_kml(path, tracks: list[Track], book=None,
|
||
title: str = "bandsaunter — aircraft heard",
|
||
unit: str = DEFAULT_SPEED_UNIT) -> Path | None:
|
||
"""Every track as a line on the globe, with a pin where it was last seen."""
|
||
from xml.sax.saxutils import escape as xml_escape
|
||
|
||
located = [t for t in tracks if t.located]
|
||
if not located:
|
||
return None
|
||
out = ['<?xml version="1.0" encoding="UTF-8"?>',
|
||
'<kml xmlns="http://www.opengis.net/kml/2.2">', " <Document>",
|
||
f" <name>{xml_escape(title)}</name>",
|
||
' <Style id="path"><LineStyle><color>ff40c0ff</color>'
|
||
"<width>2</width></LineStyle></Style>"]
|
||
for track in located:
|
||
entry = book.get(track.icao, track.callsign) if book is not None else None
|
||
told = "\n".join([track.describe(unit)]
|
||
+ (entry.details() if entry is not None else []))
|
||
name = f"{track.icao} {track.callsign}".strip()
|
||
line = " ".join(f"{f.longitude:.6f},{f.latitude:.6f},"
|
||
f"{f.altitude_ft * 0.3048:.0f}" for f in track.fixes)
|
||
out += [" <Placemark>",
|
||
f" <name>{xml_escape(name)}</name>",
|
||
f" <description>{xml_escape(told)}</description>",
|
||
" <styleUrl>#path</styleUrl>",
|
||
" <LineString><altitudeMode>absolute</altitudeMode>",
|
||
" <extrude>0</extrude><tessellate>1</tessellate>",
|
||
f" <coordinates>{line}</coordinates>",
|
||
" </LineString>", " </Placemark>",
|
||
" <Placemark>",
|
||
f" <name>{xml_escape(name)}</name>",
|
||
" <Point><coordinates>"
|
||
f"{track.fixes[-1].longitude:.6f},{track.fixes[-1].latitude:.6f},"
|
||
f"{track.fixes[-1].altitude_ft * 0.3048:.0f}</coordinates></Point>",
|
||
" </Placemark>"]
|
||
out += [" </Document>", "</kml>", ""]
|
||
try:
|
||
Path(path).write_text("\n".join(out))
|
||
except OSError:
|
||
return None
|
||
return Path(path)
|