Package the speech recogniser and its model for apt
No speech recogniser is in Debian, so installing bandsaunter from a .deb left transcription to a manual pip step on every machine. A repository of one's own is not bound by archive policy, so build-repo.sh now packages faster-whisper and the base.en model alongside the application: bandsaunter the application (Architecture: all) bandsaunter-transcribe faster-whisper, vendored (amd64) bandsaunter-model-base-en the model, so nothing reaches the network The wheels land in /usr/lib/bandsaunter/vendor rather than dist-packages, and transcribe.py appends that directory to sys.path -- appends, so an apt-managed numpy or PyYAML still wins and the vendor copy only fills the gap. Duplicates of what Debian already ships are stripped from the tree. resolve_model() turns a bare "base.en" into the packaged copy when one is installed, and leaves it alone to be downloaded when none is. The app package recommends the other two, so "apt install bandsaunter" brings the lot and --no-install-recommends still gets just the scanner. Its postinst explains how to add a recogniser only when there genuinely is not one -- including the case where apt has already unpacked the recogniser package but not yet configured it. Verified with the source tree hidden and no home directory: the packaged CLI runs, and a real recording transcribes offline from the vendored engine and packaged model while numpy still resolves to the system one. apt itself resolves the repository over HTTP and plans all three. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
The Dust Council
parent
16f3128690
commit
8d94a52942
5 changed files with 304 additions and 22 deletions
|
|
@ -30,10 +30,11 @@ sys.exit(main())
|
|||
EOF
|
||||
chmod 755 "$pkgdir/usr/bin/bandsaunter"
|
||||
|
||||
# Every one of these is in Debian, so apt resolves the lot. The speech
|
||||
# recognisers are not packaged for Debian and can only come from pip, so they
|
||||
# are suggested rather than depended on -- transcription is off by default and
|
||||
# says so plainly when no recogniser is present.
|
||||
# Every one of these is in Debian, so apt resolves the lot. No speech
|
||||
# recogniser is packaged for Debian, so that part ships as its own package
|
||||
# (built by build-repo.sh) and is recommended rather than depended on: apt
|
||||
# pulls it in from the repository, and a bare "dpkg -i" of this file still
|
||||
# installs, without transcription.
|
||||
cat > "$pkgdir/DEBIAN/control" <<EOF
|
||||
Package: bandsaunter
|
||||
Version: ${version}-${revision}
|
||||
|
|
@ -42,7 +43,7 @@ Priority: optional
|
|||
Architecture: ${arch}
|
||||
Depends: python3 (>= 3.10), python3-numpy, python3-scipy, python3-rich,
|
||||
python3-yaml, librtlsdr0
|
||||
Recommends: espeak-ng
|
||||
Recommends: bandsaunter-transcribe, espeak-ng
|
||||
Suggests: rtl-sdr
|
||||
Maintainer: bandsaunter
|
||||
Installed-Size: $(du -ks "$pkgdir" | cut -f1)
|
||||
|
|
@ -66,6 +67,27 @@ if [ "$1" = configure ]; then
|
|||
echo 'bandsaunter: blacklisted dvb_usb_rtl28xxu; unplug and replug the'
|
||||
echo ' receiver, or run: sudo rmmod dvb_usb_rtl28xxu'
|
||||
fi
|
||||
|
||||
# Said once, at install time, only when there is genuinely nothing to
|
||||
# transcribe with -- otherwise the recogniser package has already
|
||||
# arrived and there is nothing to explain.
|
||||
if ! python3 -c 'import importlib.util as u, sys
|
||||
sys.path.append("/usr/lib/bandsaunter/vendor")
|
||||
try:
|
||||
found = any(u.find_spec(m) for m in
|
||||
("faster_whisper", "whisper", "vosk", "pocketsphinx"))
|
||||
except Exception:
|
||||
found = True # something is there, however unhappy; say nothing
|
||||
sys.exit(0 if found else 1)' \
|
||||
2>/dev/null; then
|
||||
echo 'bandsaunter: speech transcription is unavailable -- no recogniser'
|
||||
echo ' is installed. Scanning, recording and CW decoding all'
|
||||
echo ' work without one. To add transcription, either:'
|
||||
echo ' apt install bandsaunter-transcribe'
|
||||
echo ' (from the repository this package came from), or'
|
||||
echo ' pip install --break-system-packages faster-whisper'
|
||||
echo ' Then: bandsaunter transcribe --engines'
|
||||
fi
|
||||
fi
|
||||
exit 0
|
||||
EOF
|
||||
|
|
|
|||
121
packaging/build-repo.sh
Executable file
121
packaging/build-repo.sh
Executable file
|
|
@ -0,0 +1,121 @@
|
|||
#!/bin/sh
|
||||
# Build every package and an apt repository to serve them from.
|
||||
#
|
||||
# Produces three packages:
|
||||
# bandsaunter the application itself
|
||||
# bandsaunter-transcribe the speech recogniser, which Debian does not
|
||||
# package, vendored into a private directory
|
||||
# bandsaunter-model-<name> the recogniser's model, so a machine never
|
||||
# has to reach the network
|
||||
#
|
||||
# The result is a flat apt repository: point a machine at it and
|
||||
# "apt install bandsaunter" brings the lot, offline from then on.
|
||||
set -eu
|
||||
|
||||
here=$(cd "$(dirname "$0")/.." && pwd)
|
||||
out=${1:-$here/dist/repo}
|
||||
model=${MODEL:-base.en}
|
||||
hf_repo=${HF_REPO:-Systran/faster-whisper-$model}
|
||||
revision=${DEB_REVISION:-1}
|
||||
version=$(cd "$here" && python3 -c 'import bandsaunter; print(bandsaunter.debian_version())')
|
||||
|
||||
# Compiled wheels tie this to one architecture. amd64 only, by design.
|
||||
arch=amd64
|
||||
|
||||
mkdir -p "$out"
|
||||
stage=$(mktemp -d)
|
||||
trap 'rm -rf "$stage"' EXIT
|
||||
|
||||
say() { printf '%s\n' "$*" >&2; }
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
say "building bandsaunter ${version}-${revision}"
|
||||
"$here/packaging/build-deb.sh" "$out" >/dev/null
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
say "collecting the speech recogniser (this downloads from PyPI once)"
|
||||
wheels="$stage/wheels"
|
||||
mkdir -p "$wheels"
|
||||
pip download --quiet --only-binary=:all: --dest "$wheels" faster-whisper >&2
|
||||
|
||||
pkg="$stage/bandsaunter-transcribe_${version}-${revision}_${arch}"
|
||||
vendor="$pkg/usr/lib/bandsaunter/vendor"
|
||||
mkdir -p "$vendor" "$pkg/DEBIAN"
|
||||
for w in "$wheels"/*.whl; do
|
||||
python3 -m zipfile -e "$w" "$vendor/"
|
||||
done
|
||||
# Debian already provides these and they win on sys.path anyway, so shipping
|
||||
# them would be dead weight.
|
||||
rm -rf "$vendor"/numpy "$vendor"/numpy-* "$vendor"/yaml "$vendor"/PyYAML-* \
|
||||
"$vendor"/setuptools "$vendor"/setuptools-* "$vendor"/pkg_resources \
|
||||
"$vendor"/_yaml "$vendor"/_distutils_hack "$vendor"/numpy.libs
|
||||
# A .pth file only runs inside a real site directory; here it is dead weight.
|
||||
rm -f "$vendor"/*.pth
|
||||
find "$vendor" -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true
|
||||
|
||||
cat > "$pkg/DEBIAN/control" <<EOF
|
||||
Package: bandsaunter-transcribe
|
||||
Version: ${version}-${revision}
|
||||
Section: hamradio
|
||||
Priority: optional
|
||||
Architecture: ${arch}
|
||||
Depends: bandsaunter (= ${version}-${revision}), python3-numpy, python3-yaml
|
||||
Recommends: bandsaunter-model-$(echo "$model" | tr '._' '--')
|
||||
Maintainer: bandsaunter
|
||||
Installed-Size: $(du -ks "$pkg" | cut -f1)
|
||||
Description: speech recogniser for bandsaunter
|
||||
Transcribes recorded voice transmissions to text.
|
||||
.
|
||||
No speech recogniser is packaged for Debian, so faster-whisper and its
|
||||
dependencies are installed here into a private directory rather than into
|
||||
dist-packages. That directory is searched after the system one, so anything
|
||||
apt provides still takes precedence and these copies only fill the gap.
|
||||
EOF
|
||||
fakeroot dpkg-deb --build -Zxz "$pkg" "$out" >/dev/null
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
say "collecting the $model model"
|
||||
modelpkg="$stage/bandsaunter-model-$(echo "$model" | tr '._' '--')_${version}-${revision}_all"
|
||||
modeldir="$modelpkg/usr/share/bandsaunter/models/$model"
|
||||
mkdir -p "$modeldir" "$modelpkg/DEBIAN"
|
||||
PYTHONPATH="$stage/wheels-unused" python3 - "$hf_repo" "$modeldir" <<'PY' >&2
|
||||
import shutil, sys
|
||||
from pathlib import Path
|
||||
from huggingface_hub import snapshot_download
|
||||
src = Path(snapshot_download(sys.argv[1]))
|
||||
dst = Path(sys.argv[2])
|
||||
for item in src.iterdir():
|
||||
if item.name.startswith("."):
|
||||
continue
|
||||
target = dst / item.name
|
||||
# Resolve the cache's symlinks so the package holds real files.
|
||||
shutil.copy2(item.resolve(), target)
|
||||
PY
|
||||
|
||||
cat > "$modelpkg/DEBIAN/control" <<EOF
|
||||
Package: bandsaunter-model-$(echo "$model" | tr '._' '--')
|
||||
Version: ${version}-${revision}
|
||||
Section: hamradio
|
||||
Priority: optional
|
||||
Architecture: all
|
||||
Depends: bandsaunter-transcribe
|
||||
Maintainer: bandsaunter
|
||||
Installed-Size: $(du -ks "$modelpkg" | cut -f1)
|
||||
Description: $model speech model for bandsaunter
|
||||
The $model recognition model, installed locally so that transcription works
|
||||
without reaching the network. Without this package the recogniser downloads
|
||||
the model the first time it is used, on every machine.
|
||||
EOF
|
||||
fakeroot dpkg-deb --build -Zxz "$modelpkg" "$out" >/dev/null
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
say "indexing the repository"
|
||||
( cd "$out" && dpkg-scanpackages --multiversion . /dev/null > Packages 2>/dev/null
|
||||
gzip -9kfn Packages
|
||||
apt-ftparchive -o APT::FTPArchive::Release::Suite=stable \
|
||||
-o APT::FTPArchive::Release::Codename=bandsaunter \
|
||||
release . > Release )
|
||||
|
||||
say ""
|
||||
say "repository ready: $out"
|
||||
ls -1sh "$out"/*.deb >&2
|
||||
Loading…
Add table
Add a link
Reference in a new issue