Package the speech recogniser and its model for apt

No speech recogniser is in Debian, so installing bandsaunter from a .deb
left transcription to a manual pip step on every machine. A repository of
one's own is not bound by archive policy, so build-repo.sh now packages
faster-whisper and the base.en model alongside the application:

  bandsaunter                the application (Architecture: all)
  bandsaunter-transcribe     faster-whisper, vendored (amd64)
  bandsaunter-model-base-en  the model, so nothing reaches the network

The wheels land in /usr/lib/bandsaunter/vendor rather than dist-packages,
and transcribe.py appends that directory to sys.path -- appends, so an
apt-managed numpy or PyYAML still wins and the vendor copy only fills the
gap. Duplicates of what Debian already ships are stripped from the tree.
resolve_model() turns a bare "base.en" into the packaged copy when one is
installed, and leaves it alone to be downloaded when none is.

The app package recommends the other two, so "apt install bandsaunter"
brings the lot and --no-install-recommends still gets just the scanner.
Its postinst explains how to add a recogniser only when there genuinely
is not one -- including the case where apt has already unpacked the
recogniser package but not yet configured it.

Verified with the source tree hidden and no home directory: the packaged
CLI runs, and a real recording transcribes offline from the vendored
engine and packaged model while numpy still resolves to the system one.
apt itself resolves the repository over HTTP and plans all three.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

README.md

@@ -38,6 +38,52 @@ sudo apt install ./dist/bandsaunter_*.deb
 apt pulls in every dependency itself, and the package blacklists the DVB-T
 driver that would otherwise claim the receiver. Nothing else to do.
 
+Speech transcription is the one part that cannot come from Debian, because no
+speech recogniser is packaged there. Installed this way, everything else works
+and the install prints a short note saying how to add one. To have that arrive
+by apt as well, build the repository below instead.
+
+### From your own apt repository
+
+For installing on several machines, or on a fresh one, without hunting for the
+recogniser afterwards. `build-repo.sh` builds three packages and an apt index:
+
+| Package | Arch | Contents |
+|---|---|---|
+| `bandsaunter` | all | the application |
+| `bandsaunter-transcribe` | amd64 | faster-whisper and its dependencies, in `/usr/lib/bandsaunter/vendor` |
+| `bandsaunter-model-base-en` | all | the `base.en` model, in `/usr/share/bandsaunter/models` |
+
+```bash
+./packaging/build-repo.sh                 # writes dist/repo/
+rsync -a dist/repo/ server:/var/www/html/bandsaunter/
+```
+
+Serve that directory over HTTP from anywhere on the LAN, then on each machine:
+
+```bash
+echo 'deb [trusted=yes] http://server/bandsaunter ./' \
+    | sudo tee /etc/apt/sources.list.d/bandsaunter.list
+sudo apt update
+sudo apt install bandsaunter
+```
+
+That single command brings the recogniser and its model too — they are
+`Recommends`, which apt installs by default. `--no-install-recommends` gets
+just the application. Nothing reaches the network afterwards: the model is on
+disk, so the first transcription works offline.
+
+`[trusted=yes]` skips signing, which is the sensible trade on a private LAN.
+To sign it instead, run `gpg --clearsign` over `dist/repo/Release` to produce
+`InRelease` and drop the `[trusted=yes]`.
+
+The vendored packages are appended to `sys.path`, never prepended, so anything
+apt provides — numpy, PyYAML — still wins; the vendor directory only fills the
+gap Debian leaves. `BANDSAUNTER_VENDOR_DIR` and `BANDSAUNTER_MODEL_DIR`
+override both locations.
+
+Rebuilding for a new version is the same command; `apt upgrade` picks it up.
+
 ### From source
 
 ```bash
@@ -79,13 +125,18 @@ pip install vosk               # ~10 MB plus a 40 MB model, weaker on noise
 bandsaunter transcribe --engines
 ```
 
-That is why transcription is `Suggests:` rather than `Depends:` in the
+That is why the plain `.deb` cannot depend on one. Debian Policy forbids
-package. Debian Policy forbids anything in the archive from requiring
+anything in the archive from requiring software outside it, and a `postinst`
-software outside it, and a `postinst` that reaches out to PyPI would break
+that reaches out to PyPI would break offline and reproducible installs — so a
-offline and reproducible installs — so a package simply cannot pull these in.
+package in the archive simply cannot pull these in. Transcription is therefore
-Transcription is therefore off by default and reports plainly when no
+off by default and reports plainly when no recogniser is present, rather than
-recogniser is present, rather than the install failing or the feature
+the install failing or the feature appearing broken.
-appearing broken.
+
+A repository of your own is not bound by that rule, which is what
+[`build-repo.sh`](#from-your-own-apt-repository) exploits: it packages
+faster-whisper and its model itself, into a private directory rather than into
+`dist-packages`, and lets apt install them alongside. Nothing is downloaded at
+install time, and nothing collides with an apt-managed module.
 
 Mixing the two is nonetheless fine here. Modern Debian marks the system
 Python as externally managed (PEP 668), so a pip install lands in your user
@@ -100,11 +151,11 @@ from the `.deb` into `/usr/lib/python3/dist-packages` picks it up with no
 further configuration — verified, not assumed. A virtual environment works
 too, as long as bandsaunter runs inside it.
 
-If you would rather keep everything under apt, the only route is packaging a
+Getting these into Debian proper would be a different matter: it would mean
-recogniser for Debian yourself. That is a real undertaking for whisper: it
+packaging ctranslate2, tokenizers, onnxruntime and their dependencies, several
-would mean packaging ctranslate2, tokenizers, onnxruntime and their
+of which are large C++ or Rust projects, each to archive standards. That is
-dependencies, several of which are large C++ or Rust projects. It is why
+why none of them are there, and why the local repository vendors the wheels
-none of them are there.
+instead of trying to do it properly.
 
 ### Other distributions
 
@@ -682,7 +733,8 @@ depends on a trained model, so there is no built-in fallback.
 
 ```bash
 bandsaunter transcribe --engines      # what is installed
-pip install faster-whisper        # the recommended one
+sudo apt install bandsaunter-transcribe   # from your own repository
+pip install faster-whisper            # or straight from PyPI
 ```
 
 | Engine | Notes |

bandsaunter/transcribe.py

@@ -11,6 +11,7 @@ from __future__ import annotations
 import os
 import queue
 import shutil
+import sys
 import subprocess
 import tempfile
 import threading
@@ -24,12 +25,50 @@ import numpy as np
 from ._quiet import suppress_stderr
 
 __all__ = ["Transcript", "transcribe", "available_engine", "ENGINES",
-           "TranscriptionWorker", "describe_engines"]
+           "TranscriptionWorker", "describe_engines", "resolve_model",
+           "VENDOR_DIR", "MODEL_DIR"]
 
 # Best first.  Whisper handles the noise and clipping of radio audio far
 # better than the smaller recognisers, which were trained on clean speech.
 ENGINES = ("faster-whisper", "whisper", "whisper-cli", "vosk", "pocketsphinx")
 
+# No recogniser is packaged for Debian, so a package that wants to supply one
+# ships it here instead of in dist-packages.  Appended to the path rather than
+# prepended, so anything the system package manager provides still wins and a
+# vendored copy only ever fills a gap.
+VENDOR_DIR = Path(os.environ.get("BANDSAUNTER_VENDOR_DIR",
+                                 "/usr/lib/bandsaunter/vendor"))
+# Models shipped alongside, so a machine never has to reach the network.
+MODEL_DIR = Path(os.environ.get("BANDSAUNTER_MODEL_DIR",
+                                "/usr/share/bandsaunter/models"))
+
+
+def _add_vendor_path() -> None:
+    if VENDOR_DIR.is_dir():
+        path = str(VENDOR_DIR)
+        if path not in sys.path:
+            sys.path.append(path)
+
+
+_add_vendor_path()
+
+
+def resolve_model(name: str) -> str:
+    """A packaged model directory if there is one, otherwise the name itself.
+
+    Lets ``base.en`` mean the copy installed on this machine when one exists,
+    and fall back to the recogniser downloading it when one does not.
+    """
+    if not name:
+        return name
+    given = Path(name).expanduser()
+    if given.is_dir():
+        return str(given)
+    packaged = MODEL_DIR / name
+    if packaged.is_dir():
+        return str(packaged)
+    return name
+
 # Vosk's bundled models are named by region.
 _VOSK_LANGS = {"en": "en-us", "en-gb": "en-us", "": "en-us",
                "pt": "pt", "zh": "cn", "cn": "cn"}
@@ -108,7 +147,7 @@ _MODEL_LOCK = threading.Lock()
 
 def _faster_whisper(audio, rate, model, language):
     from faster_whisper import WhisperModel
-    key = ("faster-whisper", model or "base.en")
+    key = ("faster-whisper", resolve_model(model or "base.en"))
     with _MODEL_LOCK:
         engine = _MODELS.get(key)
         if engine is None:
@@ -129,7 +168,7 @@ def _faster_whisper(audio, rate, model, language):
 
 def _openai_whisper(audio, rate, model, language):
     import whisper
-    key = ("whisper", model or "base.en")
+    key = ("whisper", resolve_model(model or "base.en"))
     with _MODEL_LOCK:
         engine = _MODELS.get(key)
         if engine is None:
@@ -178,7 +217,7 @@ def _vosk(audio, rate, model, language):
     # Vosk names its bundled models by region, so the plain codes the rest of
     # the program uses have to be mapped or it reports "lang en does not exist".
     lang = _VOSK_LANGS.get((language or "en").lower(), language or "en-us")
-    path = Path(model).expanduser() if model else None
+    path = Path(resolve_model(model)).expanduser() if model else None
     use_path = bool(path and path.is_dir())
     key = ("vosk", str(path) if use_path else f"lang:{lang}")
     with _MODEL_LOCK:

packaging/build-deb.sh

@@ -30,10 +30,11 @@ sys.exit(main())
 EOF
 chmod 755 "$pkgdir/usr/bin/bandsaunter"
 
-# Every one of these is in Debian, so apt resolves the lot.  The speech
+# Every one of these is in Debian, so apt resolves the lot.  No speech
-# recognisers are not packaged for Debian and can only come from pip, so they
+# recogniser is packaged for Debian, so that part ships as its own package
-# are suggested rather than depended on -- transcription is off by default and
+# (built by build-repo.sh) and is recommended rather than depended on: apt
-# says so plainly when no recogniser is present.
+# pulls it in from the repository, and a bare "dpkg -i" of this file still
+# installs, without transcription.
 cat > "$pkgdir/DEBIAN/control" <<EOF
 Package: bandsaunter
 Version: ${version}-${revision}
@@ -42,7 +43,7 @@ Priority: optional
 Architecture: ${arch}
 Depends: python3 (>= 3.10), python3-numpy, python3-scipy, python3-rich,
  python3-yaml, librtlsdr0
-Recommends: espeak-ng
+Recommends: bandsaunter-transcribe, espeak-ng
 Suggests: rtl-sdr
 Maintainer: bandsaunter
 Installed-Size: $(du -ks "$pkgdir" | cut -f1)
@@ -66,6 +67,27 @@ if [ "$1" = configure ]; then
         echo 'bandsaunter: blacklisted dvb_usb_rtl28xxu; unplug and replug the'
         echo '             receiver, or run: sudo rmmod dvb_usb_rtl28xxu'
     fi
+
+    # Said once, at install time, only when there is genuinely nothing to
+    # transcribe with -- otherwise the recogniser package has already
+    # arrived and there is nothing to explain.
+    if ! python3 -c 'import importlib.util as u, sys
+sys.path.append("/usr/lib/bandsaunter/vendor")
+try:
+    found = any(u.find_spec(m) for m in
+                ("faster_whisper", "whisper", "vosk", "pocketsphinx"))
+except Exception:
+    found = True   # something is there, however unhappy; say nothing
+sys.exit(0 if found else 1)' \
+         2>/dev/null; then
+        echo 'bandsaunter: speech transcription is unavailable -- no recogniser'
+        echo '             is installed. Scanning, recording and CW decoding all'
+        echo '             work without one. To add transcription, either:'
+        echo '               apt install bandsaunter-transcribe'
+        echo '             (from the repository this package came from), or'
+        echo '               pip install --break-system-packages faster-whisper'
+        echo '             Then: bandsaunter transcribe --engines'
+    fi
 fi
 exit 0
 EOF

packaging/build-repo.sh

@@ -0,0 +1,121 @@
+#!/bin/sh
+# Build every package and an apt repository to serve them from.
+#
+# Produces three packages:
+#   bandsaunter                  the application itself
+#   bandsaunter-transcribe       the speech recogniser, which Debian does not
+#                                package, vendored into a private directory
+#   bandsaunter-model-<name>     the recogniser's model, so a machine never
+#                                has to reach the network
+#
+# The result is a flat apt repository: point a machine at it and
+# "apt install bandsaunter" brings the lot, offline from then on.
+set -eu
+
+here=$(cd "$(dirname "$0")/.." && pwd)
+out=${1:-$here/dist/repo}
+model=${MODEL:-base.en}
+hf_repo=${HF_REPO:-Systran/faster-whisper-$model}
+revision=${DEB_REVISION:-1}
+version=$(cd "$here" && python3 -c 'import bandsaunter; print(bandsaunter.debian_version())')
+
+# Compiled wheels tie this to one architecture.  amd64 only, by design.
+arch=amd64
+
+mkdir -p "$out"
+stage=$(mktemp -d)
+trap 'rm -rf "$stage"' EXIT
+
+say() { printf '%s\n' "$*" >&2; }
+
+# ---------------------------------------------------------------------------
+say "building bandsaunter ${version}-${revision}"
+"$here/packaging/build-deb.sh" "$out" >/dev/null
+
+# ---------------------------------------------------------------------------
+say "collecting the speech recogniser (this downloads from PyPI once)"
+wheels="$stage/wheels"
+mkdir -p "$wheels"
+pip download --quiet --only-binary=:all: --dest "$wheels" faster-whisper >&2
+
+pkg="$stage/bandsaunter-transcribe_${version}-${revision}_${arch}"
+vendor="$pkg/usr/lib/bandsaunter/vendor"
+mkdir -p "$vendor" "$pkg/DEBIAN"
+for w in "$wheels"/*.whl; do
+    python3 -m zipfile -e "$w" "$vendor/"
+done
+# Debian already provides these and they win on sys.path anyway, so shipping
+# them would be dead weight.
+rm -rf "$vendor"/numpy "$vendor"/numpy-* "$vendor"/yaml "$vendor"/PyYAML-* \
+       "$vendor"/setuptools "$vendor"/setuptools-* "$vendor"/pkg_resources \
+       "$vendor"/_yaml "$vendor"/_distutils_hack "$vendor"/numpy.libs
+# A .pth file only runs inside a real site directory; here it is dead weight.
+rm -f "$vendor"/*.pth
+find "$vendor" -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true
+
+cat > "$pkg/DEBIAN/control" <<EOF
+Package: bandsaunter-transcribe
+Version: ${version}-${revision}
+Section: hamradio
+Priority: optional
+Architecture: ${arch}
+Depends: bandsaunter (= ${version}-${revision}), python3-numpy, python3-yaml
+Recommends: bandsaunter-model-$(echo "$model" | tr '._' '--')
+Maintainer: bandsaunter
+Installed-Size: $(du -ks "$pkg" | cut -f1)
+Description: speech recogniser for bandsaunter
+ Transcribes recorded voice transmissions to text.
+ .
+ No speech recogniser is packaged for Debian, so faster-whisper and its
+ dependencies are installed here into a private directory rather than into
+ dist-packages. That directory is searched after the system one, so anything
+ apt provides still takes precedence and these copies only fill the gap.
+EOF
+fakeroot dpkg-deb --build -Zxz "$pkg" "$out" >/dev/null
+
+# ---------------------------------------------------------------------------
+say "collecting the $model model"
+modelpkg="$stage/bandsaunter-model-$(echo "$model" | tr '._' '--')_${version}-${revision}_all"
+modeldir="$modelpkg/usr/share/bandsaunter/models/$model"
+mkdir -p "$modeldir" "$modelpkg/DEBIAN"
+PYTHONPATH="$stage/wheels-unused" python3 - "$hf_repo" "$modeldir" <<'PY' >&2
+import shutil, sys
+from pathlib import Path
+from huggingface_hub import snapshot_download
+src = Path(snapshot_download(sys.argv[1]))
+dst = Path(sys.argv[2])
+for item in src.iterdir():
+    if item.name.startswith("."):
+        continue
+    target = dst / item.name
+    # Resolve the cache's symlinks so the package holds real files.
+    shutil.copy2(item.resolve(), target)
+PY
+
+cat > "$modelpkg/DEBIAN/control" <<EOF
+Package: bandsaunter-model-$(echo "$model" | tr '._' '--')
+Version: ${version}-${revision}
+Section: hamradio
+Priority: optional
+Architecture: all
+Depends: bandsaunter-transcribe
+Maintainer: bandsaunter
+Installed-Size: $(du -ks "$modelpkg" | cut -f1)
+Description: $model speech model for bandsaunter
+ The $model recognition model, installed locally so that transcription works
+ without reaching the network. Without this package the recogniser downloads
+ the model the first time it is used, on every machine.
+EOF
+fakeroot dpkg-deb --build -Zxz "$modelpkg" "$out" >/dev/null
+
+# ---------------------------------------------------------------------------
+say "indexing the repository"
+( cd "$out" && dpkg-scanpackages --multiversion . /dev/null > Packages 2>/dev/null
+  gzip -9kfn Packages
+  apt-ftparchive -o APT::FTPArchive::Release::Suite=stable \
+                 -o APT::FTPArchive::Release::Codename=bandsaunter \
+                 release . > Release )
+
+say ""
+say "repository ready: $out"
+ls -1sh "$out"/*.deb >&2

tests/test_transcribe.py

@@ -1,4 +1,5 @@
 """Speech to text: the engine plumbing, and how captures reach it."""
+import sys
 import time
 from datetime import datetime
 from pathlib import Path
@@ -376,3 +377,50 @@ def test_vosk_accepts_the_plain_language_code():
     result = tr.transcribe(say("one two three", 16000), 16000, engine="vosk",
                            language="en")
     assert result is not None and not result.note, result.note
+
+
+# --- packaged engine and models -------------------------------------------
+
+def test_a_packaged_model_is_used_in_place_of_its_name(tmp_path, monkeypatch):
+    """"base.en" should mean the copy on this machine when one is installed."""
+    monkeypatch.setattr(tr, "MODEL_DIR", tmp_path)
+    (tmp_path / "base.en").mkdir()
+    assert tr.resolve_model("base.en") == str(tmp_path / "base.en")
+
+
+def test_an_unpackaged_model_keeps_its_name_to_be_downloaded(tmp_path, monkeypatch):
+    monkeypatch.setattr(tr, "MODEL_DIR", tmp_path)
+    assert tr.resolve_model("small.en") == "small.en"
+    assert tr.resolve_model("") == ""
+
+
+def test_an_explicit_model_directory_wins_over_the_packaged_one(tmp_path, monkeypatch):
+    packaged = tmp_path / "packaged"
+    (packaged / "base.en").mkdir(parents=True)
+    monkeypatch.setattr(tr, "MODEL_DIR", packaged)
+    mine = tmp_path / "base.en"
+    mine.mkdir()
+    assert tr.resolve_model(str(mine)) == str(mine)
+
+
+def test_the_vendored_engine_is_searched_after_the_system_one(tmp_path, monkeypatch):
+    """Anything apt provides must still win; the vendor copy fills a gap."""
+    monkeypatch.setattr(sys, "path", list(sys.path))
+    monkeypatch.setattr(tr, "VENDOR_DIR", tmp_path)
+    tr._add_vendor_path()
+    assert sys.path[-1] == str(tmp_path)
+
+
+def test_the_vendor_path_is_added_once(tmp_path, monkeypatch):
+    monkeypatch.setattr(sys, "path", list(sys.path))
+    monkeypatch.setattr(tr, "VENDOR_DIR", tmp_path)
+    tr._add_vendor_path()
+    tr._add_vendor_path()
+    assert sys.path.count(str(tmp_path)) == 1
+
+
+def test_a_missing_vendor_directory_is_not_added(tmp_path, monkeypatch):
+    monkeypatch.setattr(sys, "path", list(sys.path))
+    monkeypatch.setattr(tr, "VENDOR_DIR", tmp_path / "absent")
+    tr._add_vendor_path()
+    assert str(tmp_path / "absent") not in sys.path