diff --git a/INSTALL.md b/INSTALL.md index 3204e70..fe53f8e 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -150,6 +150,42 @@ ask public registers about a callsign or a 24-bit address and cache the answers for a month; `--no-lookup` turns them off, and what the address and the callsign say on their own is worked out offline either way. +### Optional: a paid schedule service + +Nothing here needs installing either — these are accounts, not packages, and +all four are optional. They answer the one question the free registers cannot: +an airline runs the same flight number over several legs in a day, and a free +register holds one route per number, so it will often name somebody else's +leg. A schedule service holds the day's actual movements. + +| Service | Sign up at | Set | +|---|---|---| +| FlightAware AeroAPI | | `BANDSAUNTER_AEROAPI_KEY` | +| Flightradar24 | | `BANDSAUNTER_FR24_TOKEN` | +| OAG Flight Info | | `BANDSAUNTER_OAG_KEY` | +| Cirium (FlightStats) | | `BANDSAUNTER_CIRIUM_APP_ID` and `BANDSAUNTER_CIRIUM_APP_KEY` | + +Put the ones you have in your shell profile: + +```sh +echo 'export BANDSAUNTER_AEROAPI_KEY=your-key-here' >> ~/.bashrc +. ~/.bashrc +``` + +**Keys are read from the environment and never written to the settings file**, +on purpose: a settings file gets copied between machines and pasted into +messages asking for help, and an API key should not travel that way. + +Any service whose key is set is asked; one whose key is not set is skipped +silently, and the free registers answer exactly as they did before. +`--schedules flightaware,oag` picks which to ask and in what order. Only the +callsign and the time are ever sent. + +These readers were written from each service's published response format and +tested against it, but none has been run against a live service, because each +one needs a paid account. Each is written to return nothing rather than guess, +so a service that has changed its format costs you a route, not a scan. + --- ## Speech transcription, step by step diff --git a/README.md b/README.md index efe6a3c..ddb6d33 100644 --- a/README.md +++ b/README.md @@ -1116,6 +1116,29 @@ and `A835AF` is American with no network at all), and the first three letters of an airline callsign are its ICAO designator, so `RYR1234` is Ryanair. `--no-lookup` stops at that. +**A callsign is a flight number, not a leg.** An airline runs the same number +over several legs in a day — Southwest especially — and a register holds one +route for it, so an aircraft crossing Arizona is quite often handed a +half-hour hop between two airports in Texas. The two databases routinely +disagree with each other about the same flight number, and both are snapshots +years old. + +Nothing on the air settles it: **ADS-B carries no origin or destination.** An +aircraft broadcasts who and where it is, not where it is going. So what can be +done is checked rather than trusted: + +- A route the aircraft **cannot** be flying is left off the map and out of the + window. The two ends are known, and an aircraft on a route is never much + further along it than the route is long. It is still written in the report + with a note saying so, because it is what the register holds for that flight + number and worth having — it is just not a statement about where this + aeroplane was going. +- Where a source lists a **whole day's stops** rather than a leg — hexdb + answers `KORD-KEWR-KORD` for some flight numbers — the aircraft's own + position picks the leg out. Reading the ends off that string instead gives + Chicago to Chicago, which is not a flight. +- Where no leg fits, none is claimed. + ``` 4008F6 BAW49 registration: G-VROS @@ -1131,6 +1154,46 @@ of an airline callsign are its ICAO designator, so `RYR1234` is Ryanair. speed: up to 480 kt ``` +### Knowing the leg for certain + +That needs live schedule data, which none of the free sources carry. Four +commercial services are wired up, and all four are **optional**: + +| service | keys | +| --- | --- | +| [FlightAware AeroAPI](https://www.flightaware.com/commercial/aeroapi/) | `BANDSAUNTER_AEROAPI_KEY` | +| [Flightradar24](https://fr24api.flightradar24.com/) | `BANDSAUNTER_FR24_TOKEN` | +| [OAG Flight Info](https://developer.oag.com/) | `BANDSAUNTER_OAG_KEY` | +| [Cirium (FlightStats)](https://developer.cirium.com/) | `BANDSAUNTER_CIRIUM_APP_ID` and `BANDSAUNTER_CIRIUM_APP_KEY` | + +Each holds the timetable and the day's movements, so each can answer the +question the registers cannot: which leg of that flight number was in the air +at the moment the aircraft was overhead. A schedule service is asked first and +the free databases pick the question back up where it does not answer, so a +program with no keys set behaves exactly as it did before. + +**Keys are read from the environment, never the settings file** — a settings +file is meant to be copied between machines and pasted into a message asking +for help, and an API key is not. There is a test that enforces it. + +```sh +export BANDSAUNTER_AEROAPI_KEY=... +bandsaunter flights # ask every service with a key +bandsaunter flights --schedules flightaware # ask only that one +bandsaunter adsb --schedules oag,cirium # ask those two, in that order +``` + +A service with no key is skipped rather than asked and refused, since a +request is only a slow way of finding out there is no key. The callsign and +the moment are all that is sent. The same setting lives in the ADS-B options +menu under **Schedule services**. + +One caveat, stated plainly: **each reader was written from its service's +published response shape and tested against that shape; none has been run +against a live service**, because each wants a paid account. So each is +written to find what it recognises and return nothing at all otherwise — a +service that has changed since costs a route, not a scan. + ### The moving map ```bash @@ -1278,16 +1341,37 @@ are met rather than assumed: - **the attribution is drawn onto the picture**, because a GIF travels without the readme that would otherwise carry it. -A drawing is capped at a few dozen tiles — past that the zoom drops, since a -coarser map still says where the coastline is. `--no-basemap` draws the tracks -on their own, `--tiles URL` points at another server (your own, if you run -one), and when there is no network and nothing cached the picture falls back -to the plain grid it drew before. +`--no-basemap` draws the tracks on their own, `--tiles URL` points at another +server (your own, if you run one), and when there is no network and nothing +cached the picture falls back to the plain grid it drew before. -The map is drawn at the resolution it was fetched at, not stretched: tiles are -averaged down to the picture rather than point-sampled, so lettering and roads -stay whole instead of breaking up, and the window fetches enough pixels to -cover its margin at full detail rather than enlarging what it has. +**The zoom is chosen from how wide the picture is, not from the area alone.** +A map asked for at 1920 pixels fetches finer tiles than the same map asked for +at 960, and about 1.4× the width is fetched deliberately and then averaged +down — a downscaled tile is sharp and an upscaled one is not, so it is better +to fetch too much and shrink it than to fetch too little and stretch it. + +The window fetches a little more world than it shows, so that panning does not +leave the ground blank, and it fetches that bigger piece **at the bigger +piece's own size**: what the window then shows comes out pixel for pixel with +the screen. Rendering the wider piece into the window's own pixels and +stretching it back is an upscale of a fifth applied to the whole map, which is +what a sharp map looks like when it looks blurred. + +Measured, at a 100-mile radius: + +``` +animation at 960 px zoom 9 28 tiles 1792 px 1.9x oversampled +animation at 1920 px zoom 10 91 tiles 3328 px 1.7x oversampled +window at 1920x1080 zoom 10 135 tiles 3840 px 1.6x oversampled +window at 3840x2160 zoom 10 135 tiles 3840 px 0.8x — upscaled +``` + +A drawing is still capped, at a couple of hundred tiles, and the last line is +what that cap looks like: at 4K the zoom has already stopped climbing and the +map is enlarged after all. Somebody else's tile server is not a thing to fetch +a thousand tiles from for one picture. A smaller `--radius` buys the detail +back, since the same budget then covers less ground. `--map-brightness PERCENT` (70 by default) is how far up its range the map is drawn. The ground has to stay dark enough that the aircraft are the brightest diff --git a/bandsaunter/__init__.py b/bandsaunter/__init__.py index 35611de..beb6dea 100755 --- a/bandsaunter/__init__.py +++ b/bandsaunter/__init__.py @@ -9,7 +9,7 @@ and transcribing speech. # 2026-08-21_02 is the second build made on the 21st. The revision is padded # to two digits so versions sort as text. VERSION_DATE = "2026-09-04" -VERSION_REVISION = 6 +VERSION_REVISION = 9 __version__ = f"{VERSION_DATE}_{VERSION_REVISION:02d}" diff --git a/bandsaunter/aircraft.py b/bandsaunter/aircraft.py index 089adbb..61546b9 100644 --- a/bandsaunter/aircraft.py +++ b/bandsaunter/aircraft.py @@ -95,6 +95,7 @@ class AircraftOptions: frames: bool = False log: bool = True lookup: bool = True + schedules: str = "" kml: bool = False hold: float = 45.0 speed_unit: str = "knots" @@ -201,6 +202,22 @@ OPTIONS: tuple[Setting, ...] = ( "air are ever sent. What the address block and the callsign say on " "their own is worked out offline either way.", flags=("--lookup",), off_flags=("--no-lookup",)), + O("schedules", "Schedule services", "Listening", "text", + "which paid schedule services to ask, in order (blank = all with keys)", + "A callsign is a flight number and an airline runs the same number over " + "several legs in a day, so the free route databases -- which hold one " + "route per number -- often name somebody else's leg. A commercial " + "schedule service holds the timetable and the day's movements and can " + "say which leg is in the air now. Four are wired up: flightaware, " + "flightradar24, oag and cirium. Each wants a key, none is required, " + "and a service with no key is skipped in silence. Keys are read from " + "the environment rather than kept here, because a settings file gets " + "copied between machines and pasted into messages asking for help: " + "BANDSAUNTER_AEROAPI_KEY, BANDSAUNTER_FR24_TOKEN, BANDSAUNTER_OAG_KEY, " + "and BANDSAUNTER_CIRIUM_APP_ID with BANDSAUNTER_CIRIUM_APP_KEY.", + example="flightaware,cirium", + guidance="Leave it blank unless you want one service tried before " + "another. With no keys set, nothing changes."), O("kml", "Also write a KML", "Listening", "bool", "write the flight paths for Google Earth as well", "One line per aircraft on the globe, with a pin where it was last " @@ -608,7 +625,8 @@ def listen(console, options: AircraftOptions, output_dir: str, # The registers are asked while the listening runs rather than after it, # so a registration and a route appear on the line as they arrive. The # book answers immediately with what it knows and fills itself in later. - book = FlightBook(online=options.lookup) + book = FlightBook(online=options.lookup, + schedules=schedule_names(options)) display, live = _open_display(console, options, book, started) def on_block(total: int) -> None: @@ -676,7 +694,8 @@ def watch(console, options: AircraftOptions, output_dir: str, started = time.time() log = open_log(console, options, output_dir, started, log_path) registry = AircraftRegistry() - book = FlightBook(online=options.lookup) + book = FlightBook(online=options.lookup, + schedules=schedule_names(options)) sky = livemap.Sky(unit=options.speed_unit, hold=options.hold, home=read_position(options.location), radius_nm=radius_in_nm(options) or 100.0, @@ -687,10 +706,19 @@ def watch(console, options: AircraftOptions, output_dir: str, if options.simulate: sky.note = "simulated" + def told_about(craft): + """What a register says about one aircraft, for the window.""" + if not options.lookup: + return None + entry = book.get(craft.icao, craft.callsign) + if craft.located: + # A source that listed a whole day's stops has said which leg + # this is, once there is a position to read it with. + entry = book.resolve(entry, craft.latitude, craft.longitude) + return entry + def on_block(total: int) -> None: - sky.update([livemap.blip_for(craft, - book.get(craft.icao, craft.callsign) - if options.lookup else None) + sky.update([livemap.blip_for(craft, told_about(craft)) for craft in registry.aircraft.values()], total, len(registry)) @@ -838,6 +866,12 @@ def checked(console, options: AircraftOptions, tracks): return tracks +def schedule_names(options: AircraftOptions) -> list[str]: + """The schedule services to ask, in the order given.""" + return [name.strip() for name in str(options.schedules or "").split(",") + if name.strip()] + + def radius_in_nm(options: AircraftOptions) -> float: """The map radius as the drawing wants it, in nautical miles. @@ -897,7 +931,8 @@ def draw_log(console, options: AircraftOptions, paths, out_path=None): console.print(f"[yellow]{paths[0].name} holds no frames[/yellow]") return None tracks = checked(console, options, tracks) - book = FlightBook(online=options.lookup) + book = FlightBook(online=options.lookup, + schedules=schedule_names(options)) if options.lookup: for track in tracks: book.get(track.icao, track.callsign) diff --git a/bandsaunter/basemap.py b/bandsaunter/basemap.py index 1d7b87a..ede263e 100644 --- a/bandsaunter/basemap.py +++ b/bandsaunter/basemap.py @@ -38,7 +38,8 @@ from . import __version__ __all__ = ["decode_png", "tile_of", "choose_zoom", "fetch_tile", "mosaic", "ground_under", "TILE_URL", "ATTRIBUTION", "MAX_TILES", "MAX_ZOOM", - "cache_dir", "PNGError", "airports_in", "AIRPORTS_URL"] + "cache_dir", "PNGError", "airports_in", "AIRPORTS_URL", + "tile_span"] # The standard OpenStreetMap tiles. Any {z}/{x}/{y} server can be put here # instead; nothing below knows anything about this one in particular. @@ -48,10 +49,18 @@ TILE_URL = "https://tile.openstreetmap.org/{z}/{x}/{y}.png" # a GIF travels without its readme. ATTRIBUTION = "MAP DATA (C) OPENSTREETMAP CONTRIBUTORS" -# A drawing is worth a few dozen tiles and no more. Past that the zoom is -# reduced instead: a coarser map still says where the coastline is. -MAX_TILES = 90 +# The backstop on how many tiles one drawing may fetch. Reached only by a +# view so wide that no zoom covers it without fetching half a country; the +# usual limit is the size of the picture, which is asked for instead. A +# screenful at a hundred-mile radius comes to about a hundred and fifty, +# fetched once and kept. +MAX_TILES = 220 MAX_ZOOM = 13 + +# How much more detail to fetch than the picture holds. Averaging several +# source pixels into each output one is what makes lettering and coastlines +# come out smooth; taking exactly one leaves them as hard as they were. +OVERSAMPLE = 1.4 MIN_ZOOM = 2 TILE_PIXELS = 256 @@ -219,17 +228,39 @@ def tile_of(lat: float, lon: float, zoom: int) -> tuple[float, float]: return x, y +def tile_span(south: float, west: float, north: float, east: float, + zoom: int) -> tuple[int, int]: + """How many tiles wide and tall a box is at one zoom.""" + x0, y0 = tile_of(north, west, zoom) + x1, y1 = tile_of(south, east, zoom) + return (int(math.floor(x1)) - int(math.floor(x0)) + 1, + int(math.floor(y1)) - int(math.floor(y0)) + 1) + + def choose_zoom(south: float, west: float, north: float, east: float, - max_tiles: int = MAX_TILES, most: int = MAX_ZOOM) -> int: - """The most detail that fits inside the tile budget.""" - for zoom in range(min(most, MAX_ZOOM), MIN_ZOOM - 1, -1): - x0, y0 = tile_of(north, west, zoom) - x1, y1 = tile_of(south, east, zoom) - wide = int(math.floor(x1)) - int(math.floor(x0)) + 1 - tall = int(math.floor(y1)) - int(math.floor(y0)) + 1 - if wide * tall <= max_tiles: - return zoom - return MIN_ZOOM + width: int = 0, max_tiles: int = MAX_TILES, + most: int = MAX_ZOOM) -> int: + """The zoom to fetch at: enough for the picture, and no more. + + ``width`` is how many pixels across the picture will be. Without it + this fetched whatever the tile budget allowed, which is the wrong + question in both directions: on a small picture it fetched more than + could be shown, and on a large one it fetched less and the map was + blown up to fit -- which is what a low-resolution map looks like. + + So it climbs until the tiles hold at least as many pixels as the + picture wants, and stops there. The budget is the backstop, for a view + so wide that no zoom can cover it without fetching half a country. + """ + best = MIN_ZOOM + for zoom in range(MIN_ZOOM, min(most, MAX_ZOOM) + 1): + wide, tall = tile_span(south, west, north, east, zoom) + if wide * tall > max_tiles: + break + best = zoom + if width and wide * TILE_PIXELS >= width * OVERSAMPLE: + break # detail enough; more would only cost time + return best # --------------------------------------------------------------------------- @@ -468,7 +499,8 @@ def ground_under(south: float, west: float, north: float, east: float, """ if width < 1 or height < 1 or north <= south or east <= west: return None - zoom = choose_zoom(south, west, north, east) if zoom is None else zoom + if zoom is None: + zoom = choose_zoom(south, west, north, east, width=width) tiles, origin_x, origin_y = mosaic(south, west, north, east, zoom, fetch=fetch, **kw) if tiles is None: diff --git a/bandsaunter/cli.py b/bandsaunter/cli.py index b34fdab..f6a380f 100755 --- a/bandsaunter/cli.py +++ b/bandsaunter/cli.py @@ -174,6 +174,10 @@ examples: help="listen without writing anything down") ad.add_argument("--no-lookup", dest="lookup", action="store_false", help="do not ask the registers who the aircraft are") + ad.add_argument("--schedules", default=None, metavar="NAMES", + help="which schedule services to ask, comma separated: " + "flightaware, flightradar24, oag, cirium " + "(each needs a key in the environment)") ad.add_argument("--kml", nargs="?", const="", default=None, metavar="FILE", help="also write the flight paths for Google Earth") ad.add_argument("--map", nargs="?", const="", default=None, metavar="FILE", @@ -224,6 +228,10 @@ examples: help="report only, draw nothing") fl.add_argument("--no-lookup", dest="lookup", action="store_false", help="do not ask the registers who the aircraft are") + fl.add_argument("--schedules", default=None, metavar="NAMES", + help="which schedule services to ask, comma separated: " + "flightaware, flightradar24, oag, cirium " + "(each needs a key in the environment)") fl.add_argument("--kml", nargs="?", const="", default=None, metavar="FILE", help="also write the flight paths for Google Earth") fl.add_argument("--report", nargs="?", const="", default=None, metavar="FILE", @@ -1085,6 +1093,8 @@ def cmd_adsb(args) -> int: options.speed_unit = args.speed_unit if args.basemap is not None: options.basemap = args.basemap + if args.schedules is not None: + options.schedules = args.schedules if args.map: options.picture = Path(args.map).suffix.lstrip(".") or options.picture @@ -1135,11 +1145,18 @@ def cmd_flights(args) -> int: options.speed_unit = args.speed_unit if args.recheck: options.recheck = True + if args.schedules is not None: + options.schedules = args.schedules tracks = air.checked(console, options, tracks) - book = FlightBook(online=args.lookup) + book = FlightBook(online=args.lookup, + schedules=air.schedule_names(options)) if args.lookup: for track in tracks: - book.get(track.icao, track.callsign) + # The moment it was overhead, so a schedule service can say + # which leg was in the air then rather than which is now. + when = (track.fixes[len(track.fixes) // 2].at if track.located + else track.last_seen) + book.get(track.icao, track.callsign, when) book.wait(20.0) book.save() diff --git a/bandsaunter/flightlog.py b/bandsaunter/flightlog.py index 9330e20..be5d7d0 100644 --- a/bandsaunter/flightlog.py +++ b/bandsaunter/flightlog.py @@ -679,6 +679,9 @@ def report(tracks: list[Track], book=None, title: str = "", out.append(f"{track.icao}" + (f" {track.callsign}" if track.callsign else "")) entry = book.get(track.icao, track.callsign) if book is not None else None + if entry is not None and track.located and hasattr(book, "resolve"): + here = track.fixes[len(track.fixes) // 2] + entry = book.resolve(entry, here.latitude, here.longitude) if entry is not None: for line in entry.details(): # The route is printed as one line further down: "from" and @@ -689,7 +692,7 @@ def report(tracks: list[Track], book=None, title: str = "", if entry.country and not entry.owner_country: out.append(f" registered in: {entry.country}") if entry.route: - out.append(f" route: {entry.route}") + out.append(f" route: {entry.route}{_doubtful(entry, track)}") out.append(f" heard: {_clock(track.first_seen)} to " f"{_clock(track.last_seen)} " f"({_span(track.seconds)}, {track.frames} frames)") @@ -712,6 +715,25 @@ def report(tracks: list[Track], book=None, title: str = "", return out +def _doubtful(entry, track: Track) -> str: + """A note where an aircraft cannot have been flying the route given. + + Said rather than hidden. The route is what the register holds for that + flight number and is worth writing down; what it is not is a statement + about where this aeroplane was going, and the difference matters enough + to spell out. + """ + if not track.located: + return "" + from .flights import route_fits + + here = track.fixes[len(track.fixes) // 2] + if route_fits(entry, here.latitude, here.longitude): + return "" + return " (scheduled for this flight number; this aircraft was " \ + "nowhere near it)" + + def _clock(when: float) -> str: return datetime.fromtimestamp(when).strftime("%Y-%m-%d %H:%M:%S") \ if when else "—" diff --git a/bandsaunter/flightmap.py b/bandsaunter/flightmap.py index ec8a6b9..373116a 100644 --- a/bandsaunter/flightmap.py +++ b/bandsaunter/flightmap.py @@ -647,6 +647,15 @@ def label_lines(track: Track, now, unit: str = DEFAULT_SPEED_UNIT, rows.append((f"{kind} {entry.registration}", home)) elif kind or entry.registration: rows.append((kind or entry.registration, home)) + # Only when the aircraft could actually be flying it. A callsign is a + # flight number and an airline runs the same number over several legs in + # a day, so a register's one route for it is quite often somebody else's + # leg -- and a route drawn beside an aircraft reads as a statement about + # that aircraft. + from .flights import route_fits + + if not route_fits(entry, now.latitude, now.longitude): + return rows if entry.origin_code or entry.origin: rows.append((_short_place(entry.origin_code, entry.origin), entry.origin_country)) @@ -1000,8 +1009,16 @@ def _known_from(book, tracks: list[Track]) -> dict: """ if book is None: return {} - return {track.icao: book.get(track.icao, track.callsign) - for track in tracks} + out = {} + for track in tracks: + entry = book.get(track.icao, track.callsign) + if track.located and hasattr(book, "resolve"): + # Where a source listed a whole day's stops rather than a leg, + # the aircraft's own position says which leg it is on. + here = track.fixes[len(track.fixes) // 2] + entry = book.resolve(entry, here.latitude, here.longitude) + out[track.icao] = entry + return out def _airports_from(known: dict, book=None): diff --git a/bandsaunter/flights.py b/bandsaunter/flights.py index e81dcb7..3c04dcc 100644 --- a/bandsaunter/flights.py +++ b/bandsaunter/flights.py @@ -27,10 +27,11 @@ import threading import time import urllib.parse import urllib.request -from dataclasses import dataclass +from dataclasses import dataclass, replace from pathlib import Path __all__ = ["Flight", "FlightBook", "describe_address", "airline_of", + "route_fits", "ROUTE_SLACK_NM", "AIRCRAFT_URL", "ROUTE_URL", "CACHE_VERSION"] AIRCRAFT_URL = "https://api.adsbdb.com/v0/aircraft/{icao}" @@ -332,6 +333,14 @@ class Flight: airline: str = "" # who the callsign says, from the table country: str = "" # from the address block owner_country: str = "" # from the register, which can differ + # Every stop the flight number is recorded as making, in order. A + # source that says "KORD-KEWR-KORD" is describing a day's work rather + # than a leg, and reading the ends off it gives Chicago to Chicago. + stops: tuple = () + # Which service the route came from, where it was a schedule service + # rather than one of the free databases: those know the leg, and it is + # worth being able to say which answers are the trustworthy ones. + route_source: str = "" origin_code: str = "" origin: str = "" origin_lat: float = 0.0 @@ -395,6 +404,41 @@ class Flight: return out +# How far off a route an aircraft may be before it is decided that it cannot +# be flying it. Generous: an aircraft holds, diverts around weather and gets +# vectored, and none of that is a wrong route. +ROUTE_SLACK_NM = 150.0 +ROUTE_SLACK_PART = 0.5 + + +def route_fits(entry, lat: float, lon: float) -> bool: + """Could an aircraft here be flying the route this callsign is given? + + A callsign is a flight number, not a leg. An airline runs the same + number over several legs in a day -- Southwest especially -- and a + register holds one route for it, so an aircraft crossing Arizona is + quite often handed a thirty-minute hop between two airports in Texas. + Reported as fact that is simply wrong, and it is wrong in a way that is + easy to check: the two ends are known, and an aircraft on a route is + never much further along it than the route is long. + + True when there is nothing to check with, because not knowing is not the + same as knowing it is wrong. + """ + origin = (getattr(entry, "origin_lat", 0.0), + getattr(entry, "origin_lon", 0.0)) + destination = (getattr(entry, "destination_lat", 0.0), + getattr(entry, "destination_lon", 0.0)) + if not any(origin) or not any(destination): + return True + from .flightlog import distance_nm + + leg = distance_nm(origin[0], origin[1], destination[0], destination[1]) + by_way_of = (distance_nm(origin[0], origin[1], lat, lon) + + distance_nm(lat, lon, destination[0], destination[1])) + return by_way_of <= leg + max(ROUTE_SLACK_NM, ROUTE_SLACK_PART * leg) + + def _cache_path() -> Path: root = os.environ.get("XDG_CACHE_HOME") or "~/.cache" return Path(root).expanduser() / "bandsaunter" / "flights.json" @@ -420,7 +464,8 @@ class FlightBook: route_url: str = ROUTE_URL, backup_aircraft_url: str = BACKUP_AIRCRAFT_URL, backup_route_url: str = BACKUP_ROUTE_URL, - airport_url: str = AIRPORT_URL): + airport_url: str = AIRPORT_URL, + schedules=None): self.online = online self.timeout = timeout self.max_age = max_age @@ -429,6 +474,9 @@ class FlightBook: self.backup_aircraft_url = backup_aircraft_url self.backup_route_url = backup_route_url self.airport_url = airport_url + # Which commercial schedule services to ask, in order, before the + # free route databases. Empty means every one that has a key. + self.schedules = list(schedules or []) self.cache_path = Path(cache) if cache is not None else _cache_path() self._lock = threading.Lock() self._entries: dict[str, Flight] = {} @@ -495,7 +543,8 @@ class FlightBook: pass # -- lookup ----------------------------------------------------------- - def get(self, icao: str, callsign: str = "") -> Flight: + def get(self, icao: str, callsign: str = "", + when: float | None = None) -> Flight: """What is known about an aircraft now, starting a lookup if needed. Called again with a callsign once one has been heard -- identity and @@ -523,9 +572,11 @@ class FlightBook: if not self.online: return entry if fresh or wanted_route: - thread = threading.Thread(target=self._fetch, - args=(entry, fresh, wanted_route), - daemon=True) + thread = threading.Thread( + target=self._fetch, + args=(entry, fresh, wanted_route, + time.time() if when is None else when), + daemon=True) with self._lock: self._threads.append(thread) thread.start() @@ -587,7 +638,8 @@ class FlightBook: self._threads = [t for t in self._threads if t.is_alive()] # -- the network ------------------------------------------------------ - def _fetch(self, entry: Flight, airframe: bool, route: bool) -> None: + def _fetch(self, entry: Flight, airframe: bool, route: bool, + when: float | None = None) -> None: reached = False if airframe: for url, apply in ((self.aircraft_url, self._apply_aircraft), @@ -615,7 +667,12 @@ class FlightBook: self._dirty = True if route and entry.callsign: - found = None + # A schedule service first, where one has a key. It holds the + # timetable and the day's movements, so it can say which leg of + # a flight number is in the air now; the free databases hold one + # route per number and cannot. + found = self._scheduled(entry.callsign, + time.time() if when is None else when) for url, read in ((self.route_url, self._read_route), (self.backup_route_url, self._read_backup_route)): if not url or found: @@ -633,6 +690,16 @@ class FlightBook: self._apply_route(entry, found) self._dirty = True + def _scheduled(self, callsign: str, when: float) -> dict | None: + """What a commercial schedule service says, if one is set up.""" + try: + from . import schedules + + return schedules.route_for(callsign, when, self.schedules, + timeout=self.timeout) + except Exception: + return None + def _request(self, url: str) -> dict: """Ask one website one question. @@ -719,26 +786,86 @@ class FlightBook: @staticmethod def _read_backup_route(body: dict) -> dict | None: - """Read hexdb's route, which is a single "EIDW-EGSS" string.""" + """Read hexdb's route, which is a single "EIDW-EGSS" string. + + Sometimes it is a whole day: "KORD-KEWR-KORD". Taking the ends off + that gives Chicago to Chicago, which is not a flight; the stops are + kept instead, and which leg an aircraft is on is decided later, when + there is a position to decide it with. + """ raw = str((body or {}).get("route") or "").strip() parts = [p.strip().upper() for p in raw.split("-") if p.strip()] if len(parts) < 2: return None - # A multi-leg route lists every stop; the ends are what a map wants. # Nothing here says which country an airport is in, but its ICAO code # does: the first letter or two is a region. from .flags import country_of_icao - return {"origin_code": parts[0], "origin": parts[0], - "origin_country": country_of_icao(parts[0]), - "destination_code": parts[-1], "destination": parts[-1], - "destination_country": country_of_icao(parts[-1]), - "airline": ""} + route = {"stops": tuple(parts), "airline": ""} + if len(parts) == 2: + route.update({"origin_code": parts[0], "origin": parts[0], + "origin_country": country_of_icao(parts[0]), + "destination_code": parts[-1], + "destination": parts[-1], + "destination_country": country_of_icao(parts[-1])}) + return route + + def leg_for(self, entry: Flight, lat: float, lon: float): + """Which leg of a multi-stop day an aircraft here is flying. + + A source that lists every stop has said more than one that names two + airports, not less: with a position in hand the leg can be picked + out, because an aircraft on a leg is never much further along it + than the leg is long. Returns the two codes, or None when no leg + fits -- which is itself an answer, and a better one than naming a + leg the aircraft cannot be on. + """ + stops = tuple(getattr(entry, "stops", ()) or ()) + if len(stops) < 3: + return None + placed = {code: self.airport(code) for code in set(stops)} + for first, second in zip(stops, stops[1:]): + start, end = placed.get(first) or {}, placed.get(second) or {} + if not (start.get("latitude") or start.get("longitude")): + continue + if not (end.get("latitude") or end.get("longitude")): + continue + leg = Flight(icao=entry.icao, + origin_lat=start["latitude"], + origin_lon=start["longitude"], + destination_lat=end["latitude"], + destination_lon=end["longitude"]) + if route_fits(leg, lat, lon): + return first, second + return None + + def resolve(self, entry: Flight, lat: float, lon: float) -> Flight: + """The entry with the leg filled in, where one could be worked out.""" + picked = self.leg_for(entry, lat, lon) + if picked is None: + return entry + from .flags import country_of_icao + + first, second = picked + start = self.airport(first) or {} + end = self.airport(second) or {} + return replace( + entry, + origin_code=first, origin=start.get("name") or first, + origin_lat=start.get("latitude", 0.0), + origin_lon=start.get("longitude", 0.0), + origin_country=start.get("country") or country_of_icao(first), + destination_code=second, destination=end.get("name") or second, + destination_lat=end.get("latitude", 0.0), + destination_lon=end.get("longitude", 0.0), + destination_country=end.get("country") or country_of_icao(second)) @staticmethod def _apply_route(entry: Flight, route: dict | None) -> None: if not route: return + entry.stops = tuple(route.get("stops") or ()) + entry.route_source = str(route.get("source") or "") entry.origin_code = str(route.get("origin_code") or "") entry.origin = str(route.get("origin") or "") entry.origin_lat = float(route.get("origin_lat") or 0.0) diff --git a/bandsaunter/images.py b/bandsaunter/images.py index 3917066..6da81f7 100644 --- a/bandsaunter/images.py +++ b/bandsaunter/images.py @@ -167,7 +167,11 @@ GLYPHS = { "N": ("10001", "11001", "10101", "10011", "10001", "10001", "10001"), "O": ("01110", "10001", "10001", "10001", "10001", "10001", "01110"), "P": ("11110", "10001", "10001", "11110", "10000", "10000", "10000"), - "Q": ("01110", "10001", "10001", "10001", "10101", "10010", "01101"), + # A tail hanging below and right of a plain O. The earlier shape kept + # the tail inside the letter, where at this size it read as the nought's + # slash: a registration came back as N87650 when the aircraft was + # N8765Q. + "Q": ("01110", "10001", "10001", "10001", "10001", "01110", "00011"), "R": ("11110", "10001", "10001", "11110", "10100", "10010", "10001"), "S": ("01111", "10000", "10000", "01110", "00001", "00001", "11110"), "T": ("11111", "00100", "00100", "00100", "00100", "00100", "00100"), diff --git a/bandsaunter/livemap.py b/bandsaunter/livemap.py index 7c55f35..f0c86b1 100644 --- a/bandsaunter/livemap.py +++ b/bandsaunter/livemap.py @@ -64,9 +64,10 @@ RESETTLE_NM = 12.0 # How much map to fetch beyond what is being shown. A window that fetched # exactly what it needed would fetch again on every resize and every small -# drift; a third again in each direction covers both, at the cost of about -# twice as many tiles the first time. -GROUND_MARGIN = 0.33 +# drift. Modest, because the fetch is now at the picture's own resolution +# and every extra degree of margin is a ring of tiles: with the middle of +# the map settling rather than wandering, a little is enough. +GROUND_MARGIN = 0.12 # The widest a line in a box is allowed to get before it is folded. An # airport's full name and the town it is in run to forty characters on their @@ -162,6 +163,9 @@ class Blip: origin_country: str = "" # two letters, for the flag beside it destination: str = "" destination_country: str = "" + # Whether this aircraft could be flying the route its callsign is given. + # A callsign is a flight number rather than a leg, so often it could not. + route_fits: bool = True @property def located(self) -> bool: @@ -193,9 +197,9 @@ class Blip: out.append(("", self.operator, "")) # The two ends of the flight on their own lines rather than joined by # an arrow, so that each can carry the flag of the country it is in. - if self.origin: + if self.route_fits and self.origin: out.append(("from", self.origin, self.origin_country)) - if self.destination: + if self.route_fits and self.destination: out.append(("to", self.destination, self.destination_country)) if self.altitude_ft: climb = "" @@ -291,6 +295,10 @@ def blip_for(craft, entry=None) -> Blip: blip.origin_country = entry.origin_country blip.destination = entry.destination or entry.destination_code blip.destination_country = entry.destination_country + if blip.located: + from .flights import route_fits + + blip.route_fits = route_fits(entry, blip.latitude, blip.longitude) return blip @@ -1023,9 +1031,11 @@ def fetch_ground(sky: Sky, url: str = "", fetch=None) -> None: extra = {"fetch": fetch} if fetch is not None else {} if url: extra["url"] = url - # Fetched at the size of the window rather than the bigger box, - # so a wider view costs no more pixels; it is stretched back over - # the box when it is drawn. + # The size asked for is the box's own, not the window's, so the + # piece of it the window shows comes out pixel for pixel with + # the screen. Rendering the wider box into the window's pixels + # and stretching it back was a whole-map upscale of a fifth, + # which is what a sharp map looks like when it looks blurred. levels = basemap.ground_under(south, west, north, east, width, height, shades=_ground_shades(), **extra) diff --git a/bandsaunter/schedules.py b/bandsaunter/schedules.py new file mode 100644 index 0000000..c46c804 --- /dev/null +++ b/bandsaunter/schedules.py @@ -0,0 +1,454 @@ +"""Where a flight number is actually going, from a service that knows. + +Everything else in this program reads what an aircraft broadcasts, and an +aircraft does not broadcast where it is going: ADS-B carries an address, a +callsign, a position and a speed, and nothing about a destination. The free +route databases fill that gap with one route per flight number, which is a +guess dressed as a fact -- an airline runs the same number over several legs +in a day, and the databases disagree with each other about which one to hold. + +A commercial schedule service does know, because it holds the timetable and +the day's movements. Four are wired up here. All of them want a key, none +of them is required, and nothing about this program changes if no key is +ever set: a source with no key says so and is skipped, and the free +databases answer as they did before. + +Keys are read from the environment rather than the settings file, because a +settings file is meant to be copied between machines and pasted into a +message asking for help, and an API key is not. + + BANDSAUNTER_AEROAPI_KEY FlightAware AeroAPI + BANDSAUNTER_FR24_TOKEN Flightradar24 API + BANDSAUNTER_OAG_KEY OAG Flight Info + BANDSAUNTER_CIRIUM_APP_ID Cirium (FlightStats), with + BANDSAUNTER_CIRIUM_APP_KEY + +Every one of these was written from the published shape of its answers and +has been tested against those shapes; none has been run against the live +service, because that needs a paid key. Each reader is therefore written to +find what it recognises and return nothing at all otherwise, so that a +service which has changed since costs a route rather than a scan. +""" + +from __future__ import annotations + +import json +import os +import urllib.parse +import urllib.request +from datetime import datetime, timezone + +__all__ = ["SOURCES", "Schedule", "source_named", "available_sources", + "route_for", "SourceError"] + + +class SourceError(Exception): + """A schedule service could not answer.""" + + +class Schedule: + """One schedule service. + + Subclasses say where their key comes from, how to ask, and how to read + the answer. Nothing else in the program knows one from another. + """ + + name = "" + needs = () # the environment variables it wants + signup = "" # where a key comes from + + def __init__(self, timeout: float = 8.0): + self.timeout = timeout + + # -- what it needs ---------------------------------------------------- + def keys(self) -> dict: + return {name: os.environ.get(name, "").strip() for name in self.needs} + + def available(self) -> bool: + """Whether this source has everything it needs to be asked.""" + got = self.keys() + return bool(self.needs) and all(got.get(n) for n in self.needs) + + def missing(self) -> list[str]: + got = self.keys() + return [n for n in self.needs if not got.get(n)] + + # -- asking it -------------------------------------------------------- + def route(self, callsign: str, when: float) -> dict | None: + """The leg this callsign is flying at this moment, or None. + + Returns the same shape the free databases return, so that everything + downstream is unchanged: origin and destination codes, names and + countries, and the positions where they are given. + """ + if not self.available() or not callsign: + return None + try: + body = self.ask(callsign.strip().upper(), when) + return self.read(body, when) + except Exception as exc: + # Reading is inside this too: an answer shaped differently from + # the one documented is the failure most likely to happen, and + # it must come back as this service not knowing rather than as + # a traceback out of the middle of a scan. + raise SourceError(f"{self.name}: {exc}") from exc + + def ask(self, callsign: str, when: float): + raise NotImplementedError + + def read(self, body, when: float) -> dict | None: + raise NotImplementedError + + # -- the plumbing ----------------------------------------------------- + def fetch(self, url: str, headers: dict | None = None): + request = urllib.request.Request( + url, headers={"User-Agent": "bandsaunter", "Accept": "application/json", + **(headers or {})}) + with urllib.request.urlopen(request, timeout=self.timeout) as answer: + return json.loads(answer.read(400_000).decode("utf8", "replace")) + + +def _stamp(when: float) -> str: + """A moment, as the services want it written.""" + return datetime.fromtimestamp(when, timezone.utc).strftime( + "%Y-%m-%dT%H:%M:%SZ") + + +def _day(when: float) -> datetime: + return datetime.fromtimestamp(when, timezone.utc) + + +def _seconds(text: str) -> float: + """Read one of the several ways these services write a time.""" + text = (text or "").strip() + if not text: + return 0.0 + if text.isdigit(): + return float(text) + cleaned = text.replace("Z", "+00:00") + try: + moment = datetime.fromisoformat(cleaned) + except ValueError: + return 0.0 + if moment.tzinfo is None: + moment = moment.replace(tzinfo=timezone.utc) + return moment.timestamp() + + +def _leg(origin_code: str, destination_code: str, origin: str = "", + destination: str = "", origin_country: str = "", + destination_country: str = "", origin_lat: float = 0.0, + origin_lon: float = 0.0, destination_lat: float = 0.0, + destination_lon: float = 0.0, airline: str = "") -> dict | None: + """One leg, in the shape the rest of the program reads.""" + from .flags import country_of_icao + + origin_code = (origin_code or "").strip().upper() + destination_code = (destination_code or "").strip().upper() + if not origin_code or not destination_code: + return None + return {"stops": (origin_code, destination_code), + "origin_code": origin_code, "origin": origin or origin_code, + "origin_country": origin_country or country_of_icao(origin_code), + "origin_lat": origin_lat, "origin_lon": origin_lon, + "destination_code": destination_code, + "destination": destination or destination_code, + "destination_country": (destination_country + or country_of_icao(destination_code)), + "destination_lat": destination_lat, + "destination_lon": destination_lon, + "airline": airline} + + +def _closest(legs: list[tuple[float, float, dict]], when: float) -> dict | None: + """The leg whose window holds this moment, or the nearest one to it. + + A service hands back every movement of a flight number over a day or + two. The one being watched is the one in the air now; where the times + do not quite line up -- a delayed departure, a clock a few minutes out + -- the nearest is a better answer than none, and much better than the + first in the list, which is what taking element zero would give. + """ + if not legs: + return None + inside = [leg for start, end, leg in legs + if start and end and start <= when <= end] + if inside: + return inside[0] + + def distance(item): + start, end, _leg = item + if start and end: + return min(abs(when - start), abs(when - end)) + return abs(when - (start or end or when)) + + nearest = min(legs, key=distance) + # Nothing within half a day is not this flight; better to say nothing. + if distance(nearest) > 12 * 3600: + return None + return nearest[2] + + +# --------------------------------------------------------------------------- +# FlightAware AeroAPI +# --------------------------------------------------------------------------- + +class AeroAPI(Schedule): + """FlightAware's AeroAPI. + + Asked for the flights of one ident; answers with the recent and + scheduled movements, each with its own airports and times, which is + exactly what tells one leg of a flight number from another. + """ + + name = "flightaware" + needs = ("BANDSAUNTER_AEROAPI_KEY",) + signup = "https://www.flightaware.com/commercial/aeroapi/" + url = "https://aeroapi.flightaware.com/aeroapi/flights/{ident}" + + def ask(self, callsign: str, when: float): + return self.fetch( + self.url.format(ident=urllib.parse.quote(callsign)), + headers={"x-apikey": self.keys()["BANDSAUNTER_AEROAPI_KEY"]}) + + def read(self, body, when: float) -> dict | None: + legs = [] + for flight in (body or {}).get("flights") or []: + origin = flight.get("origin") or {} + destination = flight.get("destination") or {} + leg = _leg(origin.get("code_icao") or origin.get("code"), + destination.get("code_icao") or destination.get("code"), + origin=origin.get("name") or "", + destination=destination.get("name") or "", + airline=str(flight.get("operator") or "")) + if leg is None: + continue + start = _seconds(flight.get("actual_off") + or flight.get("estimated_off") + or flight.get("scheduled_off") + or flight.get("scheduled_out")) + end = _seconds(flight.get("actual_on") + or flight.get("estimated_on") + or flight.get("scheduled_on") + or flight.get("scheduled_in")) + legs.append((start, end, leg)) + return _closest(legs, when) + + +# --------------------------------------------------------------------------- +# Flightradar24 +# --------------------------------------------------------------------------- + +class Flightradar24(Schedule): + """The Flightradar24 API. + + Asked for the flight summary of a callsign over the day around the + observation, which comes back with the airports each movement used. + """ + + name = "flightradar24" + needs = ("BANDSAUNTER_FR24_TOKEN",) + signup = "https://fr24api.flightradar24.com/" + url = ("https://fr24api.flightradar24.com/api/flight-summary/light" + "?flights={ident}&flight_datetime_from={start}" + "&flight_datetime_to={end}") + + def ask(self, callsign: str, when: float): + return self.fetch( + self.url.format(ident=urllib.parse.quote(callsign), + start=urllib.parse.quote(_stamp(when - 12 * 3600)), + end=urllib.parse.quote(_stamp(when + 12 * 3600))), + headers={"Authorization": + f"Bearer {self.keys()['BANDSAUNTER_FR24_TOKEN']}", + "Accept-Version": "v1"}) + + def read(self, body, when: float) -> dict | None: + # Some of its endpoints wrap the rows in an object and some hand + # back the list itself. + rows = body if isinstance(body, list) else (body or {}).get("data") + legs = [] + for row in rows or []: + leg = _leg(row.get("orig_icao") or row.get("origin_icao"), + row.get("dest_icao") or row.get("destination_icao"), + airline=str(row.get("operating_as") + or row.get("painted_as") or "")) + if leg is None: + continue + legs.append((_seconds(row.get("datetime_takeoff")), + _seconds(row.get("datetime_landed")), leg)) + return _closest(legs, when) + + +# --------------------------------------------------------------------------- +# OAG +# --------------------------------------------------------------------------- + +class OAG(Schedule): + """OAG's flight information service, asked for one day's schedules.""" + + name = "oag" + needs = ("BANDSAUNTER_OAG_KEY",) + signup = "https://developer.oag.com/" + url = ("https://api.oag.com/flight-instances/?DepartureDateTime={day}" + "&CarrierCode={carrier}&FlightNumber={number}" + "&CodeType=ICAO&Content=All") + + def ask(self, callsign: str, when: float): + carrier, number = _split_callsign(callsign) + if not number: + raise SourceError("not an airline callsign") + return self.fetch( + self.url.format(day=_day(when).strftime("%Y-%m-%d"), + carrier=urllib.parse.quote(carrier), + number=urllib.parse.quote(number)), + headers={"Subscription-Key": self.keys()["BANDSAUNTER_OAG_KEY"]}) + + def read(self, body, when: float) -> dict | None: + legs = [] + for row in (body or {}).get("data") or []: + departure = row.get("departure") or {} + arrival = row.get("arrival") or {} + leg = _leg(_airport_code(departure), _airport_code(arrival), + origin=str((departure.get("airport") or {}).get("name") + or ""), + destination=str((arrival.get("airport") or {}).get("name") + or ""), + airline=str((row.get("carrier") or {}).get("icao") or "")) + if leg is None: + continue + legs.append((_seconds(_when_of(departure)), + _seconds(_when_of(arrival)), leg)) + return _closest(legs, when) + + +def _airport_code(end: dict) -> str: + airport = end.get("airport") or {} + return str(airport.get("icao") or airport.get("iata") or "") + + +def _when_of(end: dict) -> str: + times = end.get("date") or {} + clock = end.get("time") or {} + if isinstance(times, dict) and isinstance(clock, dict): + day = times.get("utc") or times.get("local") or "" + hour = clock.get("utc") or clock.get("local") or "" + if day and hour: + return f"{day}T{hour}Z" if "T" not in str(day) else str(day) + return str(end.get("dateTimeUtc") or end.get("dateTime") or "") + + +# --------------------------------------------------------------------------- +# Cirium (FlightStats) +# --------------------------------------------------------------------------- + +class Cirium(Schedule): + """Cirium's FlightStats schedules, asked for one flight on one day.""" + + name = "cirium" + needs = ("BANDSAUNTER_CIRIUM_APP_ID", "BANDSAUNTER_CIRIUM_APP_KEY") + signup = "https://developer.cirium.com/" + url = ("https://api.flightstats.com/flex/schedules/rest/v1/json/flight/" + "{carrier}/{number}/departing/{year}/{month}/{day}" + "?appId={app_id}&appKey={app_key}") + + def ask(self, callsign: str, when: float): + carrier, number = _split_callsign(callsign) + if not number: + raise SourceError("not an airline callsign") + keys = self.keys() + moment = _day(when) + return self.fetch(self.url.format( + carrier=urllib.parse.quote(carrier), + number=urllib.parse.quote(number), + year=moment.year, month=moment.month, day=moment.day, + app_id=urllib.parse.quote(keys["BANDSAUNTER_CIRIUM_APP_ID"]), + app_key=urllib.parse.quote(keys["BANDSAUNTER_CIRIUM_APP_KEY"]))) + + def read(self, body, when: float) -> dict | None: + # Keyed by the short code, because that is what a flight refers to + # its airports by; the four-letter one is inside the record. + places = {str(a.get("fs") or a.get("iata") or ""): a + for a in (body or {}).get("appendix", {}).get("airports", []) + if isinstance(a, dict)} + legs = [] + for row in (body or {}).get("scheduledFlights") or []: + origin = places.get(str(row.get("departureAirportFsCode") or ""), {}) + destination = places.get( + str(row.get("arrivalAirportFsCode") or ""), {}) + leg = _leg(origin.get("icao") or row.get("departureAirportFsCode"), + destination.get("icao") + or row.get("arrivalAirportFsCode"), + origin=str(origin.get("name") or ""), + destination=str(destination.get("name") or ""), + origin_country=str(origin.get("countryCode") or ""), + destination_country=str( + destination.get("countryCode") or ""), + origin_lat=float(origin.get("latitude") or 0.0), + origin_lon=float(origin.get("longitude") or 0.0), + destination_lat=float(destination.get("latitude") or 0.0), + destination_lon=float( + destination.get("longitude") or 0.0), + airline=str(row.get("carrierFsCode") or "")) + if leg is None: + continue + # The plain departureTime is local and carries no offset, so + # reading it as UTC puts a leg up to half a day from where it + # belongs and picks the wrong one. The UTC field is preferred + # wherever the answer carries it. + legs.append((_seconds(row.get("departureTimeUtc") + or row.get("departureTime")), + _seconds(row.get("arrivalTimeUtc") + or row.get("arrivalTime")), leg)) + return _closest(legs, when) + + +def _split_callsign(callsign: str) -> tuple[str, str]: + """An airline callsign as its designator and its flight number.""" + callsign = (callsign or "").strip().upper() + letters = "".join(c for c in callsign[:3] if c.isalpha()) + digits = callsign[len(letters):].lstrip() + if len(letters) < 2 or not digits.isdigit(): + return callsign, "" + return letters, digits + + +SOURCES: tuple = (AeroAPI, Flightradar24, OAG, Cirium) + + +def source_named(name: str, timeout: float = 8.0) -> Schedule | None: + for kind in SOURCES: + if kind.name == (name or "").strip().lower(): + return kind(timeout) + return None + + +def available_sources(names=None, timeout: float = 8.0) -> list[Schedule]: + """Every source that has a key, in the order asked for.""" + wanted = [n.strip().lower() for n in (names or []) if n.strip()] or \ + [kind.name for kind in SOURCES] + out = [] + for name in wanted: + source = source_named(name, timeout) + if source is not None and source.available(): + out.append(source) + return out + + +def route_for(callsign: str, when: float, names=None, + timeout: float = 8.0) -> dict | None: + """Ask each source in turn until one knows. + + A source that fails is passed over rather than allowed to stop the rest: + a key that has run out of quota should cost that service and not the + others. + """ + for source in available_sources(names, timeout): + try: + found = source.route(callsign, when) + except SourceError: + continue + if found: + found = dict(found) + found["source"] = source.name + return found + return None diff --git a/packaging/bandsaunter.1 b/packaging/bandsaunter.1 index 13c9a1f..dbc64d5 100644 --- a/packaging/bandsaunter.1 +++ b/packaging/bandsaunter.1 @@ -1,5 +1,5 @@ .\" Generated by packaging/make-man.py -- do not edit by hand. -.TH BANDSAUNTER 1 "2026-09-04" "bandsaunter 2026-09-04_06" "User Commands" +.TH BANDSAUNTER 1 "2026-09-04" "bandsaunter 2026-09-04_09" "User Commands" .SH NAME bandsaunter \- scan, record and identify radio signals with an RTL-SDR .SH SYNOPSIS @@ -1390,6 +1390,54 @@ address block says which country registered the aircraft, fixed by treaty, and the first three letters of an airline callsign are its ICAO designator. .B \-\-no\-lookup stops at that. +.PP +A callsign is a flight number rather than a leg. An airline runs the same +number over several legs in a day and a register holds one route for it, so an +aircraft crossing Arizona is quite often handed a half-hour hop between two +airports in Texas; the two registers routinely disagree about the same flight +number, and both are snapshots years old. Nothing on the air settles it, as +ADS-B carries no origin or destination: an aircraft broadcasts who and where +it is, not where it is going. +.PP +So a route the aircraft cannot be flying is left off the map and out of the +window \[em] the two ends are known, and an aircraft on a route is never much +further along it than the route is long \[em] and written in the report with a +note saying so, since it is what the register holds for that flight number and +worth having. Where a source lists a whole day's stops rather than a leg, the +aircraft's own position picks the leg out; where no leg fits, none is claimed. +.SS Schedule services +Knowing the leg for certain needs live schedule data, which none of the free +sources carry. Four commercial services are wired up and all four are +optional: FlightAware AeroAPI, Flightradar24, OAG and Cirium. Each holds the +timetable and the day's movements, so each can say which leg of a flight +number was in the air at the moment an aircraft was overhead. Where one +answers, its leg is used; where none does, the free databases answer as they +always did, and a program with no keys set behaves exactly as before. +.PP +Keys are read from the environment rather than the settings file, because a +settings file is meant to be copied between machines and pasted into a message +asking for help, and an API key is not. +.PP +.nf +BANDSAUNTER_AEROAPI_KEY FlightAware AeroAPI +BANDSAUNTER_FR24_TOKEN Flightradar24 +BANDSAUNTER_OAG_KEY OAG Flight Info +BANDSAUNTER_CIRIUM_APP_ID Cirium (FlightStats), with +BANDSAUNTER_CIRIUM_APP_KEY +.fi +.PP +.BI \-\-schedules " NAMES" +picks which to ask and in what order, comma separated, from +.BR flightaware ", " flightradar24 ", " oag " and " cirium ; +the default asks every one that has its key. A service with no key is skipped +rather than asked and refused. The callsign and the moment are all that is +sent. +.PP +Each reader was written from its service's published response shape and +tested against that shape; none has been run against a live service, since +each wants a paid account. So each is written to find what it recognises and +return nothing otherwise: a service that has changed since costs a route +rather than a scan, and the free databases pick the question back up. .SS The moving map .B bandsaunter flights reads a log back \[em] the newest one in the output directory unless told @@ -1476,8 +1524,20 @@ Tiles are cached in .I ~/.cache/bandsaunter/tiles and never fetched twice, every request identifies this program in its User-Agent, and the attribution the tiles require is written onto the picture -\[em] a GIF travels without the readme that would otherwise carry it. A drawing -is capped at a few dozen tiles; past that the zoom drops instead. +\[em] a GIF travels without the readme that would otherwise carry it. +.PP +The zoom is chosen from how wide the picture is, not from the area alone, so a +map asked for at 1920 pixels fetches finer tiles than the same map asked for +at 960. Half again over the width is fetched deliberately and averaged down, +since a downscaled tile is sharp and an upscaled one is not. +.PP +The window fetches a little more world than it shows so that panning does not +leave the ground blank, and fetches that bigger piece at the bigger piece's +own size, so what is shown comes out pixel for pixel with the screen. At 1920 +by 1080 and a hundred-mile radius the tiles hold about 1.6 times the pixels +the window wants. A drawing is capped at a couple of hundred tiles, which at +3840 by 2160 is reached: there the zoom has stopped climbing and the map is +enlarged after all, and a smaller radius buys the detail back. .PP .B \-\-no\-basemap draws the tracks on their own, diff --git a/packaging/make-man.py b/packaging/make-man.py index ccddfc1..866dc34 100755 --- a/packaging/make-man.py +++ b/packaging/make-man.py @@ -783,6 +783,54 @@ address block says which country registered the aircraft, fixed by treaty, and the first three letters of an airline callsign are its ICAO designator. .B \-\-no\-lookup stops at that. +.PP +A callsign is a flight number rather than a leg. An airline runs the same +number over several legs in a day and a register holds one route for it, so an +aircraft crossing Arizona is quite often handed a half-hour hop between two +airports in Texas; the two registers routinely disagree about the same flight +number, and both are snapshots years old. Nothing on the air settles it, as +ADS-B carries no origin or destination: an aircraft broadcasts who and where +it is, not where it is going. +.PP +So a route the aircraft cannot be flying is left off the map and out of the +window \[em] the two ends are known, and an aircraft on a route is never much +further along it than the route is long \[em] and written in the report with a +note saying so, since it is what the register holds for that flight number and +worth having. Where a source lists a whole day's stops rather than a leg, the +aircraft's own position picks the leg out; where no leg fits, none is claimed. +.SS Schedule services +Knowing the leg for certain needs live schedule data, which none of the free +sources carry. Four commercial services are wired up and all four are +optional: FlightAware AeroAPI, Flightradar24, OAG and Cirium. Each holds the +timetable and the day's movements, so each can say which leg of a flight +number was in the air at the moment an aircraft was overhead. Where one +answers, its leg is used; where none does, the free databases answer as they +always did, and a program with no keys set behaves exactly as before. +.PP +Keys are read from the environment rather than the settings file, because a +settings file is meant to be copied between machines and pasted into a message +asking for help, and an API key is not. +.PP +.nf +BANDSAUNTER_AEROAPI_KEY FlightAware AeroAPI +BANDSAUNTER_FR24_TOKEN Flightradar24 +BANDSAUNTER_OAG_KEY OAG Flight Info +BANDSAUNTER_CIRIUM_APP_ID Cirium (FlightStats), with +BANDSAUNTER_CIRIUM_APP_KEY +.fi +.PP +.BI \-\-schedules " NAMES" +picks which to ask and in what order, comma separated, from +.BR flightaware ", " flightradar24 ", " oag " and " cirium ; +the default asks every one that has its key. A service with no key is skipped +rather than asked and refused. The callsign and the moment are all that is +sent. +.PP +Each reader was written from its service's published response shape and +tested against that shape; none has been run against a live service, since +each wants a paid account. So each is written to find what it recognises and +return nothing otherwise: a service that has changed since costs a route +rather than a scan, and the free databases pick the question back up. .SS The moving map .B bandsaunter flights reads a log back \[em] the newest one in the output directory unless told @@ -869,8 +917,20 @@ Tiles are cached in .I ~/.cache/bandsaunter/tiles and never fetched twice, every request identifies this program in its User-Agent, and the attribution the tiles require is written onto the picture -\[em] a GIF travels without the readme that would otherwise carry it. A drawing -is capped at a few dozen tiles; past that the zoom drops instead. +\[em] a GIF travels without the readme that would otherwise carry it. +.PP +The zoom is chosen from how wide the picture is, not from the area alone, so a +map asked for at 1920 pixels fetches finer tiles than the same map asked for +at 960. Half again over the width is fetched deliberately and averaged down, +since a downscaled tile is sharp and an upscaled one is not. +.PP +The window fetches a little more world than it shows so that panning does not +leave the ground blank, and fetches that bigger piece at the bigger piece's +own size, so what is shown comes out pixel for pixel with the screen. At 1920 +by 1080 and a hundred-mile radius the tiles hold about 1.6 times the pixels +the window wants. A drawing is capped at a couple of hundred tiles, which at +3840 by 2160 is reached: there the zoom has stopped climbing and the map is +enlarged after all, and a smaller radius buys the detail back. .PP .B \-\-no\-basemap draws the tracks on their own, diff --git a/packaging/saunterbrowse.1 b/packaging/saunterbrowse.1 index 084a1e2..9226fc3 100644 --- a/packaging/saunterbrowse.1 +++ b/packaging/saunterbrowse.1 @@ -1,5 +1,5 @@ .\" Generated by packaging/make-browse-man.py -- do not edit by hand. -.TH SAUNTERBROWSE 1 "2026-09-04" "bandsaunter 2026-09-04_06" "User Commands" +.TH SAUNTERBROWSE 1 "2026-09-04" "bandsaunter 2026-09-04_08" "User Commands" .SH NAME saunterbrowse \- read and listen to what a bandsaunter scan collected .SH SYNOPSIS diff --git a/tests/test_flightmap.py b/tests/test_flightmap.py index 4efa7ae..478a4ab 100644 --- a/tests/test_flightmap.py +++ b/tests/test_flightmap.py @@ -1186,3 +1186,26 @@ def test_the_faint_colours_are_the_same_hues_only_dimmer(): def test_the_palette_still_has_room_after_the_fade_colours(): assert fm.FAINT + fm.RAMP_STEPS < fm.TRANSPARENT assert fm.PALETTE.shape == (256, 3) + + +def test_a_route_the_aircraft_cannot_be_flying_is_not_drawn_beside_it(): + """A route beside an aircraft reads as a statement about that aircraft, + and a callsign is a flight number rather than a leg.""" + track = straight(lat=32.7, lon=-110.4) + entry = _Entry(origin_code="KHOU", origin_lat=29.65, origin_lon=-95.28, + destination_code="KSAT", destination_lat=29.53, + destination_lon=-98.47) + said = [text for text, _flag in + fm.label_lines(track, track.fixes[0], "knots", entry)] + assert "KHOU" not in said and "KSAT" not in said + assert any("B739" in line for line in said), "the rest of it went too" + + +def test_a_route_it_could_be_flying_is_drawn(): + track = straight(lat=32.7, lon=-110.4) + entry = _Entry(origin_code="KLAX", origin_lat=33.94, origin_lon=-118.41, + destination_code="KDFW", destination_lat=32.90, + destination_lon=-97.04) + said = [text for text, _flag in + fm.label_lines(track, track.fixes[0], "knots", entry)] + assert "KLAX" in said and "KDFW" in said diff --git a/tests/test_flights.py b/tests/test_flights.py index 4354be7..e2750ea 100644 --- a/tests/test_flights.py +++ b/tests/test_flights.py @@ -598,3 +598,191 @@ def test_a_route_written_now_is_kept(tmp_path, register): again = FlightBook(cache=book.cache_path) entry = again.get("4CA1FA", "RYR1234") assert entry.origin_country == "GB" + + +# --------------------------------------------------------------------------- +# A callsign is a flight number, not a leg +# --------------------------------------------------------------------------- + +def _route(origin=(29.65, -95.28), destination=(29.53, -98.47)) -> Flight: + """Houston Hobby to San Antonio: a half-hour hop across Texas.""" + return Flight(icao="AC0FB6", callsign="SWA930", + origin_code="KHOU", origin="Houston", + origin_lat=origin[0], origin_lon=origin[1], + destination_code="KSAT", destination="San Antonio", + destination_lat=destination[0], destination_lon=destination[1]) + + +def test_an_aircraft_on_its_route_is_believed(): + from bandsaunter.flights import route_fits + + # Halfway between the two, and at either end. + assert route_fits(_route(), 29.6, -96.9) + assert route_fits(_route(), 29.65, -95.28) + assert route_fits(_route(), 29.53, -98.47) + + +def test_an_aircraft_nowhere_near_its_route_is_not(): + """The one that prompted this: a 737 over Arizona at cruise, given a + thirty-minute hop between two airports in Texas. An airline runs the + same flight number over several legs in a day and a register holds one + route for it.""" + from bandsaunter.flights import route_fits + + assert not route_fits(_route(), 32.7086, -110.4061) + + +def test_a_long_route_is_given_more_room_than_a_short_one(): + """An aircraft on a transcontinental leg wanders further from the great + circle than one on a hop, and neither is a wrong route.""" + from bandsaunter.flights import route_fits + + coast = _route(origin=(40.69, -74.17), destination=(33.43, -112.01)) + assert route_fits(coast, 32.7086, -110.4061) # Newark to Phoenix + assert route_fits(coast, 39.0, -95.0) + + +def test_a_route_with_no_positions_is_left_alone(): + """Not knowing is not the same as knowing it is wrong.""" + from bandsaunter.flights import route_fits + + bare = Flight(icao="A", origin_code="KHOU", destination_code="KSAT") + assert route_fits(bare, 32.7, -110.4) + assert route_fits(Flight(icao="A"), 0.0, 0.0) + + +def test_a_route_that_cannot_be_flown_is_still_written_down(tmp_path): + """Said rather than hidden: it is what the register holds for that + flight number, and worth having.""" + from bandsaunter.flightlog import report + + log = _log(tmp_path) + for i in range(3): + log.append(_Frame(icao="AC0FB6", callsign="SWA930"), + _Craft(32.7086, -110.4061 + i * 0.01, "SWA930"), + when=1_000_000.0 + i) + log.close() + + class _Book: + def get(self, icao, callsign=""): + return _route() + + text = "\n".join(report(read_logs(log.path), _Book())) + assert "Houston" in text and "San Antonio" in text + assert "nowhere near it" in text + + +def test_a_route_that_fits_is_not_second_guessed(tmp_path): + from bandsaunter.flightlog import report + + log = _log(tmp_path) + for i in range(3): + log.append(_Frame(icao="AC0FB6", callsign="SWA930"), + _Craft(29.6, -96.9 + i * 0.01, "SWA930"), + when=1_000_000.0 + i) + log.close() + + class _Book: + def get(self, icao, callsign=""): + return _route() + + text = "\n".join(report(read_logs(log.path), _Book())) + assert "Houston" in text + assert "nowhere near" not in text + + +# --------------------------------------------------------------------------- +# Picking the leg out of a day's work +# --------------------------------------------------------------------------- + +HEXDB_DAY = {"flight": "AAL2465", "route": "KORD-KEWR-KORD"} +HEXDB_LEG = {"flight": "BAW49", "route": "EGLL-KSEA"} + +AIRPORTS = { + "KORD": {"code": "KORD", "name": "Chicago O'Hare", "latitude": 41.978, + "longitude": -87.905, "country": "US"}, + "KEWR": {"code": "KEWR", "name": "Newark Liberty", "latitude": 40.692, + "longitude": -74.169, "country": "US"}, +} + + +@pytest.fixture +def with_airports(register, monkeypatch): + """A book that knows where a handful of airports are, and no network.""" + book, asked, answers = register + monkeypatch.setattr(type(book), "airport", + lambda self, code: AIRPORTS.get(code.upper(), {})) + return book, asked, answers + + +def test_two_stops_are_a_route(register): + book, asked, answers = register + answers["hexdb.io/api/v1/route"] = HEXDB_LEG + entry = book.get("400001", "BAW49") + book.wait(5.0) + assert (entry.origin_code, entry.destination_code) == ("EGLL", "KSEA") + assert entry.stops == ("EGLL", "KSEA") + + +def test_a_whole_day_of_stops_is_not_read_as_one_flight(register): + """"KORD-KEWR-KORD" read from its ends is Chicago to Chicago, which is + not a flight.""" + book, asked, answers = register + answers["hexdb.io/api/v1/route"] = HEXDB_DAY + entry = book.get("AD64CD", "AAL2465") + book.wait(5.0) + assert entry.stops == ("KORD", "KEWR", "KORD") + assert entry.origin_code == "" and entry.destination_code == "" + + +def test_the_leg_is_picked_out_by_where_the_aircraft_is(with_airports): + book, asked, answers = with_airports + answers["hexdb.io/api/v1/route"] = HEXDB_DAY + entry = book.get("AD64CD", "AAL2465") + book.wait(5.0) + # Over Pennsylvania, which is on the way from Chicago to Newark. + assert book.leg_for(entry, 40.8, -78.0) == ("KORD", "KEWR") + + +def test_resolving_fills_the_leg_in(with_airports): + book, asked, answers = with_airports + answers["hexdb.io/api/v1/route"] = HEXDB_DAY + entry = book.get("AD64CD", "AAL2465") + book.wait(5.0) + got = book.resolve(entry, 40.8, -78.0) + assert got.origin_code == "KORD" and got.destination_code == "KEWR" + assert "Chicago" in got.origin and got.origin_country == "US" + assert got.origin_lat == pytest.approx(41.978) + + +def test_an_aircraft_on_none_of_the_legs_is_given_none_of_them(with_airports): + """Which is an answer, and a better one than naming a leg it cannot + be on.""" + book, asked, answers = with_airports + answers["hexdb.io/api/v1/route"] = HEXDB_DAY + entry = book.get("AD64CD", "AAL2465") + book.wait(5.0) + assert book.leg_for(entry, 32.7, -110.4) is None # over Arizona + assert book.resolve(entry, 32.7, -110.4).origin_code == "" + + +def test_resolving_leaves_an_ordinary_route_alone(with_airports): + book, asked, answers = with_airports + answers["hexdb.io/api/v1/route"] = HEXDB_LEG + entry = book.get("400001", "BAW49") + book.wait(5.0) + assert book.resolve(entry, 51.0, -20.0) is entry + + +def test_an_airport_nobody_can_place_costs_only_its_own_leg(register, + monkeypatch): + book, asked, answers = register + monkeypatch.setattr(type(book), "airport", + lambda self, code: AIRPORTS.get(code.upper(), {})) + answers["hexdb.io/api/v1/route"] = {"flight": "X", "route": + "KORD-ZZZZ-KEWR"} + entry = book.get("AD64CD", "X") + book.wait(5.0) + # The middle stop cannot be placed, so neither leg touching it can be + # tested; nothing is claimed. + assert book.leg_for(entry, 40.8, -78.0) is None diff --git a/tests/test_images.py b/tests/test_images.py index aef4ac5..ae9ae2b 100644 --- a/tests/test_images.py +++ b/tests/test_images.py @@ -495,3 +495,24 @@ def test_a_simulated_transmission_becomes_a_file_on_disk(tmp_path): if not p.name.endswith("_waterfall.png")] assert written assert _read_png(written[0]).shape[1] == 320 + + +def test_a_q_cannot_be_read_as_a_nought(): + """A registration came back as N87650 when the aircraft was N8765Q.""" + from bandsaunter.images import GLYPHS + + assert GLYPHS["Q"] != GLYPHS["0"] + assert GLYPHS["Q"] != GLYPHS["O"] + # The tail hangs below and to the right of the letter, where a nought + # has nothing at all. + assert GLYPHS["Q"][-1].rstrip("0").endswith("1") + assert GLYPHS["Q"][-1][:3] == "000" + assert GLYPHS["0"][-1] != GLYPHS["Q"][-1] + + +def test_the_letters_most_easily_confused_are_all_different(): + from bandsaunter.images import GLYPHS + + for group in ("O0Q", "1I", "5S", "2Z", "8B"): + shapes = [GLYPHS[c] for c in group] + assert len(set(shapes)) == len(shapes), group diff --git a/tests/test_livemap.py b/tests/test_livemap.py index 8d35133..4de61a4 100644 --- a/tests/test_livemap.py +++ b/tests/test_livemap.py @@ -967,3 +967,38 @@ def test_one_that_has_gone_quiet_is_not_counted_as_overhead(app): assert _painted(picture) > 100 # both are drawn assert len(sky.flying()) == 2 assert sum(1 for b in sky.flying() if sky.strength(b) >= 1.0) == 1 + + +def test_a_route_the_aircraft_cannot_be_flying_is_left_out_of_the_box(): + from bandsaunter.adsb import Aircraft + from bandsaunter.flights import Flight + + craft = Aircraft(icao="AC0FB6", callsign="SWA930") + craft.latitude, craft.longitude = 32.7086, -110.4061 + entry = Flight(icao="AC0FB6", origin="Houston", origin_code="KHOU", + origin_lat=29.65, origin_lon=-95.28, + destination="San Antonio", destination_code="KSAT", + destination_lat=29.53, destination_lon=-98.47, + registration="N8765Q") + blip = blip_for(craft, entry) + assert blip.route_fits is False + labels = [label for label, _v, _f in blip.lines("knots")] + assert "from" not in labels and "to" not in labels + # And everything the aircraft itself said is still there. + assert any(v == "N8765Q" for _l, v, _f in blip.lines("knots")) + + +def test_a_route_it_could_be_flying_stays_in_the_box(): + from bandsaunter.adsb import Aircraft + from bandsaunter.flights import Flight + + craft = Aircraft(icao="AD64CD", callsign="AAL2465") + craft.latitude, craft.longitude = 33.1059, -110.5428 + entry = Flight(icao="AD64CD", origin="Santa Ana", origin_code="KSNA", + origin_lat=33.68, origin_lon=-117.87, + destination="Dallas", destination_code="KDFW", + destination_lat=32.90, destination_lon=-97.04) + blip = blip_for(craft, entry) + assert blip.route_fits is True + labels = [label for label, _v, _f in blip.lines("knots")] + assert "from" in labels and "to" in labels diff --git a/tests/test_schedules.py b/tests/test_schedules.py new file mode 100644 index 0000000..a4cf693 --- /dev/null +++ b/tests/test_schedules.py @@ -0,0 +1,467 @@ +"""The commercial schedule services. + +None of these has been run against its live service, because each wants a +paid key. What is tested is everything that can be: that a source with no +key is skipped rather than tried, that each reader takes the documented +shape of its answers and finds the leg in it, that the leg chosen is the one +in the air at the moment being asked about, and that a service which has +changed since costs a route rather than a scan. + +The recorded shapes below are written from each service's published +documentation. If one of them stops matching reality, this is where it will +show, and the failure will be a route that is not found rather than one that +is wrong. +""" +import json + +import pytest + +from bandsaunter import schedules + + +NOON = 1_788_600_000.0 # a moment to ask about + + +def at(offset_hours: float) -> str: + """A time, written the way these services write it.""" + from datetime import datetime, timezone + + return datetime.fromtimestamp(NOON + offset_hours * 3600, + timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") + + +# --------------------------------------------------------------------------- +# Without keys, which is how nearly everyone runs +# --------------------------------------------------------------------------- + +def test_every_source_says_what_it_needs(): + for kind in schedules.SOURCES: + source = kind() + assert source.name and source.needs and source.signup + assert all(n.startswith("BANDSAUNTER_") for n in source.needs) + + +def test_a_source_with_no_key_is_not_available(monkeypatch): + for kind in schedules.SOURCES: + for name in kind.needs: + monkeypatch.delenv(name, raising=False) + assert kind().available() is False + assert kind().missing() == list(kind.needs) + + +def test_a_source_with_no_key_is_never_asked(monkeypatch): + """Not asked, rather than asked and refused: there is nothing to ask + with, and a request would only be a way of finding that out slowly.""" + for kind in schedules.SOURCES: + for name in kind.needs: + monkeypatch.delenv(name, raising=False) + + def refuse(self, url, headers=None): + raise AssertionError("asked a service with no key") + + monkeypatch.setattr(schedules.Schedule, "fetch", refuse) + assert schedules.route_for("SWA930", NOON) is None + + +def test_half_a_key_is_no_key(monkeypatch): + """Cirium wants two; one of them is not enough.""" + monkeypatch.setenv("BANDSAUNTER_CIRIUM_APP_ID", "abc") + monkeypatch.delenv("BANDSAUNTER_CIRIUM_APP_KEY", raising=False) + source = schedules.source_named("cirium") + assert source.available() is False + assert source.missing() == ["BANDSAUNTER_CIRIUM_APP_KEY"] + + +def test_only_the_ones_with_keys_are_listed(monkeypatch): + for kind in schedules.SOURCES: + for name in kind.needs: + monkeypatch.delenv(name, raising=False) + assert schedules.available_sources() == [] + monkeypatch.setenv("BANDSAUNTER_AEROAPI_KEY", "k") + assert [s.name for s in schedules.available_sources()] == ["flightaware"] + + +def test_they_are_asked_in_the_order_given(monkeypatch): + monkeypatch.setenv("BANDSAUNTER_AEROAPI_KEY", "k") + monkeypatch.setenv("BANDSAUNTER_OAG_KEY", "k") + assert [s.name for s in schedules.available_sources(["oag", "flightaware"])] \ + == ["oag", "flightaware"] + + +def test_a_name_that_is_not_a_service_is_ignored(): + assert schedules.source_named("nonesuch") is None + assert schedules.available_sources(["nonesuch"]) == [] + + +# --------------------------------------------------------------------------- +# FlightAware AeroAPI +# --------------------------------------------------------------------------- + +AEROAPI = {"flights": [ + {"ident": "SWA930", "operator": "SWA", + "origin": {"code_icao": "KLAS", "name": "Harry Reid International"}, + "destination": {"code_icao": "KMDW", "name": "Chicago Midway"}, + "scheduled_off": at(-9), "scheduled_on": at(-6)}, + {"ident": "SWA930", "operator": "SWA", + "origin": {"code_icao": "KMDW", "name": "Chicago Midway"}, + "destination": {"code_icao": "KSAN", "name": "San Diego International"}, + "actual_off": at(-1), "estimated_on": at(2)}, + {"ident": "SWA930", "operator": "SWA", + "origin": {"code_icao": "KSAN", "name": "San Diego International"}, + "destination": {"code_icao": "KOAK", "name": "Oakland International"}, + "scheduled_off": at(4), "scheduled_on": at(6)}, +]} + + +def _read(source_name, body, when=NOON): + """Read one service's answer, without needing a key to do it.""" + return schedules.source_named(source_name).read(body, when) + + +def test_flightaware_finds_the_leg_that_is_in_the_air(): + """Three legs of one flight number in a day; the one being watched is + the one whose window holds the moment.""" + got = _read("flightaware", AEROAPI) + assert got["origin_code"] == "KMDW" + assert got["destination_code"] == "KSAN" + assert "Midway" in got["origin"] + assert got["stops"] == ("KMDW", "KSAN") + + +def test_flightaware_picks_a_different_leg_at_a_different_hour(): + early = _read("flightaware", AEROAPI, when=NOON - 8 * 3600) + assert (early["origin_code"], early["destination_code"]) == ("KLAS", "KMDW") + late = _read("flightaware", AEROAPI, when=NOON + 5 * 3600) + assert (late["origin_code"], late["destination_code"]) == ("KSAN", "KOAK") + + +def test_flightaware_says_nothing_about_a_day_it_has_no_flights_for(): + assert _read("flightaware", AEROAPI, when=NOON + 5 * 86_400) is None + + +def test_flightaware_survives_an_answer_it_does_not_recognise(): + for body in ({}, {"flights": []}, {"flights": [{}]}, + {"flights": [{"origin": None, "destination": None}]}, + {"flights": [{"origin": {"code_icao": "KLAS"}}]}): + assert _read("flightaware", body) is None + + +# --------------------------------------------------------------------------- +# Flightradar24 +# --------------------------------------------------------------------------- + +FR24 = {"data": [ + {"fr24_id": "1", "flight": "SWA930", "operating_as": "SWA", + "orig_icao": "KMDW", "dest_icao": "KSAN", + "datetime_takeoff": at(-1), "datetime_landed": at(2)}, + {"fr24_id": "2", "flight": "SWA930", "operating_as": "SWA", + "orig_icao": "KSAN", "dest_icao": "KOAK", + "datetime_takeoff": at(4), "datetime_landed": at(6)}, +]} + + +def test_flightradar24_finds_the_leg_in_the_air(): + got = _read("flightradar24", FR24) + assert (got["origin_code"], got["destination_code"]) == ("KMDW", "KSAN") + + +def test_flightradar24_reads_a_bare_list_too(): + """Some of its endpoints wrap the rows and some do not.""" + got = _read("flightradar24", FR24["data"]) + assert got is not None and got["origin_code"] == "KMDW" + + +def test_flightradar24_survives_nonsense(): + for body in ({}, {"data": []}, {"data": [{}]}, [], None): + assert _read("flightradar24", body) is None + + +# --------------------------------------------------------------------------- +# OAG +# --------------------------------------------------------------------------- + +OAG = {"data": [ + {"carrier": {"icao": "SWA"}, + "departure": {"airport": {"icao": "KMDW", "name": "Chicago Midway"}, + "date": {"utc": at(-1)[:10]}, + "time": {"utc": at(-1)[11:19]}}, + "arrival": {"airport": {"icao": "KSAN", "name": "San Diego"}, + "date": {"utc": at(2)[:10]}, "time": {"utc": at(2)[11:19]}}}, +]} + + +def test_oag_finds_the_leg(): + got = _read("oag", OAG) + assert (got["origin_code"], got["destination_code"]) == ("KMDW", "KSAN") + assert "Midway" in got["origin"] + + +def test_oag_survives_nonsense(): + for body in ({}, {"data": []}, {"data": [{}]}, + {"data": [{"departure": {}, "arrival": {}}]}): + assert _read("oag", body) is None + + +def test_oag_needs_an_airline_callsign(monkeypatch): + monkeypatch.setenv("BANDSAUNTER_OAG_KEY", "k") + source = schedules.source_named("oag") + with pytest.raises(schedules.SourceError): + source.route("N517HP", NOON) # a registration, not a flight + + +# --------------------------------------------------------------------------- +# Cirium +# --------------------------------------------------------------------------- + +CIRIUM = { + "scheduledFlights": [ + {"carrierFsCode": "WN", "flightNumber": "930", + "departureAirportFsCode": "MDW", "arrivalAirportFsCode": "SAN", + "departureTime": at(-1), "arrivalTime": at(2)}, + ], + "appendix": {"airports": [ + {"fs": "MDW", "icao": "KMDW", "name": "Chicago Midway", + "countryCode": "US", "latitude": 41.786, "longitude": -87.752}, + {"fs": "SAN", "icao": "KSAN", "name": "San Diego International", + "countryCode": "US", "latitude": 32.733, "longitude": -117.19}, + ]}, +} + + +def test_cirium_finds_the_leg_and_where_its_airports_are(): + got = _read("cirium", CIRIUM) + assert (got["origin_code"], got["destination_code"]) == ("KMDW", "KSAN") + assert got["origin_country"] == "US" + assert got["origin_lat"] == pytest.approx(41.786) + assert got["destination_lon"] == pytest.approx(-117.19) + + +def test_cirium_survives_nonsense(): + for body in ({}, {"scheduledFlights": []}, {"scheduledFlights": [{}]}, + {"scheduledFlights": [{"departureAirportFsCode": "MDW"}], + "appendix": {}}): + assert _read("cirium", body) is None + + +# --------------------------------------------------------------------------- +# Reading the answers +# --------------------------------------------------------------------------- + +def test_a_callsign_splits_into_an_airline_and_a_number(): + assert schedules._split_callsign("SWA930") == ("SWA", "930") + assert schedules._split_callsign("BAW49") == ("BAW", "49") + assert schedules._split_callsign("N517HP")[1] == "" # a registration + assert schedules._split_callsign("")[1] == "" + + +@pytest.mark.parametrize("text,ok", [ + ("2026-09-04T12:00:00Z", True), + ("2026-09-04T12:00:00+00:00", True), + ("1788600000", True), + ("", False), ("not a time", False), ("2026-13-45", False), +]) +def test_the_several_ways_a_time_is_written_are_all_read(text, ok): + got = schedules._seconds(text) + assert (got > 0) is ok + + +def test_the_leg_whose_window_holds_the_moment_wins(): + legs = [(NOON - 7200, NOON - 3600, {"n": "before"}), + (NOON - 600, NOON + 600, {"n": "now"}), + (NOON + 3600, NOON + 7200, {"n": "after"})] + assert schedules._closest(legs, NOON)["n"] == "now" + + +def test_the_nearest_leg_wins_when_none_quite_holds_it(): + """A departure runs late and the windows no longer line up; the nearest + is a better answer than none, and much better than the first in the + list.""" + legs = [(NOON - 7 * 3600, NOON - 6 * 3600, {"n": "long before"}), + (NOON + 600, NOON + 3600, {"n": "just after"})] + assert schedules._closest(legs, NOON)["n"] == "just after" + + +def test_nothing_within_half_a_day_is_not_this_flight(): + legs = [(NOON - 40 * 3600, NOON - 39 * 3600, {"n": "yesterday"})] + assert schedules._closest(legs, NOON) is None + assert schedules._closest([], NOON) is None + + +def test_a_leg_needs_both_ends(): + assert schedules._leg("KMDW", "") is None + assert schedules._leg("", "KSAN") is None + assert schedules._leg("kmdw", "ksan")["origin_code"] == "KMDW" + + +def test_a_leg_comes_back_in_the_shape_the_rest_of_the_program_reads(): + from bandsaunter.flights import Flight + + leg = schedules._leg("EGLL", "KSEA") + fields = set(Flight().__dict__) + for key in leg: + if key in ("stops", "airline"): + continue + assert key in fields, key + assert leg["origin_country"] == "GB" # worked out from the code + assert leg["destination_country"] == "US" + + +# --------------------------------------------------------------------------- +# And what happens when one falls over +# --------------------------------------------------------------------------- + +def test_a_service_that_fails_does_not_stop_the_next_one(monkeypatch): + """A key that has run out of quota should cost that service and not the + others.""" + monkeypatch.setenv("BANDSAUNTER_AEROAPI_KEY", "k") + monkeypatch.setenv("BANDSAUNTER_FR24_TOKEN", "k") + + def fetch(self, url, headers=None): + if self.name == "flightaware": + raise OSError("quota exceeded") + return FR24 + + monkeypatch.setattr(schedules.Schedule, "fetch", fetch) + got = schedules.route_for("SWA930", NOON, + ["flightaware", "flightradar24"]) + assert got is not None + assert got["origin_code"] == "KMDW" + assert got["source"] == "flightradar24" + + +def test_the_answer_says_which_service_gave_it(monkeypatch): + monkeypatch.setenv("BANDSAUNTER_AEROAPI_KEY", "k") + monkeypatch.setattr(schedules.Schedule, "fetch", + lambda self, url, headers=None: AEROAPI) + got = schedules.route_for("SWA930", NOON, ["flightaware"]) + assert got["source"] == "flightaware" + + +def test_the_key_is_sent_the_way_the_service_wants_it(monkeypatch): + """Each of them asks for its key somewhere different.""" + seen = {} + + def fetch(self, url, headers=None): + seen[self.name] = (url, headers or {}) + return {} + + monkeypatch.setattr(schedules.Schedule, "fetch", fetch) + monkeypatch.setenv("BANDSAUNTER_AEROAPI_KEY", "aero-key") + monkeypatch.setenv("BANDSAUNTER_FR24_TOKEN", "fr24-token") + monkeypatch.setenv("BANDSAUNTER_OAG_KEY", "oag-key") + monkeypatch.setenv("BANDSAUNTER_CIRIUM_APP_ID", "cid") + monkeypatch.setenv("BANDSAUNTER_CIRIUM_APP_KEY", "ckey") + for source in schedules.available_sources(): + source.route("SWA930", NOON) + + assert seen["flightaware"][1]["x-apikey"] == "aero-key" + assert "fr24-token" in seen["flightradar24"][1]["Authorization"] + assert seen["oag"][1]["Subscription-Key"] == "oag-key" + assert "appId=cid" in seen["cirium"][0] + assert "appKey=ckey" in seen["cirium"][0] + # And no key is ever put in a URL that did not ask for one there. + assert "aero-key" not in seen["flightaware"][0] + assert "fr24-token" not in seen["flightradar24"][0] + assert "oag-key" not in seen["oag"][0] + + +def test_a_key_is_never_written_into_the_settings_file(): + """A settings file gets copied between machines and pasted into messages + asking for help; an API key does not belong in one.""" + from bandsaunter.aircraft import AircraftOptions + + text = json.dumps(AircraftOptions().to_dict()) + for kind in schedules.SOURCES: + for name in kind.needs: + assert name not in text + + +def test_the_book_asks_the_services_before_the_free_databases(monkeypatch): + """They know the leg; the free databases hold one route per number.""" + import tempfile + from pathlib import Path + + from bandsaunter.flights import FlightBook + + monkeypatch.setenv("BANDSAUNTER_AEROAPI_KEY", "k") + monkeypatch.setattr(schedules.Schedule, "fetch", + lambda self, url, headers=None: AEROAPI) + asked = [] + + def free(self, url): + asked.append(url) + raise OSError("should not have been needed") + + monkeypatch.setattr(FlightBook, "_request", free) + book = FlightBook(cache=Path(tempfile.mkdtemp()) / "c.json") + entry = book.get("AC0FB6", "SWA930", when=NOON) + book.wait(5.0) + assert entry.origin_code == "KMDW" + assert entry.route_source == "flightaware" + assert not any("callsign" in url for url in asked), \ + "asked a free database when a schedule service had answered" + + +def test_with_no_keys_the_free_databases_answer_as_before(monkeypatch): + import tempfile + from pathlib import Path + + from bandsaunter.flights import FlightBook + + for kind in schedules.SOURCES: + for name in kind.needs: + monkeypatch.delenv(name, raising=False) + answers = {"adsbdb.com/v0/callsign": {"response": {"flightroute": { + "callsign": "SWA930", + "origin": {"icao_code": "KHOU", "name": "Hobby", + "country_iso_name": "US"}, + "destination": {"icao_code": "KSAT", "name": "San Antonio", + "country_iso_name": "US"}}}}} + + def request(self, url): + for fragment, body in answers.items(): + if fragment in url: + return body + raise OSError("not found") + + monkeypatch.setattr(FlightBook, "_request", request) + book = FlightBook(cache=Path(tempfile.mkdtemp()) / "c.json") + entry = book.get("AC0FB6", "SWA930", when=NOON) + book.wait(5.0) + assert entry.origin_code == "KHOU" + assert entry.route_source == "" + + +# --------------------------------------------------------------------------- +# Two things that would go wrong quietly +# --------------------------------------------------------------------------- + +def test_cirium_prefers_the_utc_time_over_the_local_one(): + """Its plain departureTime is local and carries no offset, so reading + that as UTC puts a leg up to half a day from where it belongs -- which + is exactly far enough to pick the wrong leg of the same number.""" + body = json.loads(json.dumps(CIRIUM)) + flight = body["scheduledFlights"][0] + flight["departureTimeUtc"] = at(-1) + flight["arrivalTimeUtc"] = at(2) + # The local times say the small hours, nine time zones away. + flight["departureTime"] = at(-10)[:-1] + flight["arrivalTime"] = at(-7)[:-1] + got = _read("cirium", body) + assert got is not None + assert (got["origin_code"], got["destination_code"]) == ("KMDW", "KSAN") + + +def test_a_reader_that_throws_is_a_source_that_does_not_know(monkeypatch): + """An answer shaped differently from the documented one is the failure + most likely to actually happen. It has to come back as this service + not knowing, not as a traceback out of the middle of a scan.""" + monkeypatch.setenv("BANDSAUNTER_AEROAPI_KEY", "k") + source = schedules.source_named("flightaware") + monkeypatch.setattr(type(source), "ask", lambda self, c, w: {"flights": 7}) + with pytest.raises(schedules.SourceError): + source.route("SWA930", NOON) + # and the caller above it turns that into silence, not a crash + monkeypatch.setattr(schedules.Schedule, "fetch", + lambda self, url, headers=None: {"flights": 7}) + assert schedules.route_for("SWA930", NOON) is None