serialize ASR across tracks onto one shared worker, cut 1.7B peak memory ~13GB->~7GB

qwen_asr has no quantized-weights option, so the only lever available to
reduce the 1.7B model's memory footprint is avoiding concurrent instances.
Previously each track (system, mic) ran its own transcribeTrack goroutine
with independent qwen_asr subprocesses, so simultaneous speech on both
tracks meant two ~6.9GiB model instances alive at once - over half the
24GB target machine's memory. Merge both tracks onto a single worker
(asrSegs channel, dispatch() routes both tracks onto it) so invocations
are strictly serialized; per-track order is preserved since VAD's output
is already chronological. Verified via pgrep -x qwen_asr polling during a
real dual-track capture: never more than 1 concurrent process.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
sttlab-tech
2026-08-10 11:30:23 +02:00
parent 3525e6b8fb
commit 936f1b6402
2 changed files with 61 additions and 47 deletions
+24 -2
View File
@@ -197,9 +197,10 @@ fast enough for one-subprocess-per-segment. Switching models is a runtime flag
Three concurrency/signal-handling bugs were found and fixed via real testing (not just code
review):
- `transcribeTrack` was using the SIGINT-canceled context for still-queued transcriptions
- `transcribeWorker` (then still per-track, named `transcribeTrack`) was using the
SIGINT-canceled context for still-queued transcriptions
(fixed with a separate background context for ASR calls).
- `main()` wasn't waiting for `transcribeTrack` goroutines to finish before exiting (fixed
- `main()` wasn't waiting for the ASR worker goroutine(s) to finish before exiting (fixed
with a `sync.WaitGroup`).
- `internal/capture.Stop()` called `cmd.Wait()` before the pipe-reading goroutines had
finished draining, which `os/exec` docs call out as incorrect and can truncate the last bit
@@ -242,6 +243,27 @@ review):
waiting for in-flight transcriptions..." message and a second Ctrl+C now force-exits
immediately instead of making the user wait through the full drain.
**ASR memory: single shared worker instead of one per track (2026-08-10).** The 1.7B model
("large" in `transcriptor-ui`) has no quantized-weights option — `qwen_asr` only supports
BF16 (see `third_party/qwen-asr/README.md`, "Memory Requirements": ~6.9GiB static footprint
for the 1.7B model). Before this fix, `cmd/transcriptor-ai/main.go` ran one `transcribeTrack`
goroutine per track (system, mic), each independently spawning `qwen_asr` subprocesses — so
whenever both tracks had speech at the same time, two 1.7B instances ran concurrently,
measured peaking at ~13GiB combined, over half of the 24GB target machine's unified memory
(see "Target hardware"). Fixed by merging both tracks onto one shared worker/channel
(`asrSegs`, `transcribeWorker` — renamed from `transcribeTrack` since it's no longer
per-track): `dispatch()` now sends every enabled track's segments onto the same channel
instead of two separate ones, so `qwen_asr` invocations are strictly serialized regardless of
which track they came from. Per-track chronological order is preserved (VAD's own output
channel is already chronological across both tracks; the worker just drains it in that
order). Trade-off: if both tracks talk at once, one track's transcription now waits for the
other's to finish instead of running in parallel — accepted, since the memory pressure was
the more pressing constraint on the 24GB target machine. Verified empirically, not just "it
compiles": ran the real pipeline with both tracks capturing simultaneous audio (a video
through system output, `say` for spoken input) and polled `pgrep -x qwen_asr` at 0.3s
intervals throughout — max concurrent `qwen_asr` processes observed was 1, confirmed never 2,
while both tracks still produced correctly-ordered transcript segments.
Not yet implemented: post-processing (deferred, see "Out of scope for now"), and anything
from `audiotee`'s CONTEXT.md §6.4-adjacent concerns beyond what's listed here.