initial commit: live meeting transcription pipeline
Go orchestrator: spawns audiotee for capture (system audio + mic, two tracks), segments with an energy-threshold VAD, transcribes each segment via qwen-asr (subprocess per segment, vendored as a pinned git submodule in third_party/qwen-asr), and serves the live transcript over SSE while also writing it durably to a text file. Includes a glossary/prompt-leakage guard (internal/asr/leak.go) that discards a segment if the model echoes the biasing prompt instead of transcribing. cmd/transcriptor-ai is the main orchestrator; cmd/transcript-tail is a minimal terminal SSE client. See CLAUDE.md for the full architecture and the reasoning behind each choice (Go over Python/Swift/Rust, SSE over WebSocket, why the VAD is a hand-rolled heuristic, etc). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
sttlab-tech
@@ -0,0 +1,6 @@
|
||||
.DS_Store
|
||||
/transcriptor-ai
|
||||
/transcript-tail
|
||||
/dist
|
||||
*.pcm
|
||||
*.wav
|
||||
@@ -0,0 +1,3 @@
|
||||
[submodule "third_party/qwen-asr"]
|
||||
path = third_party/qwen-asr
|
||||
url = https://github.com/antirez/qwen-asr
|
||||
@@ -0,0 +1,252 @@
|
||||
# transcriptor-ai
|
||||
|
||||
Local, no-cloud meeting transcription for macOS. See `README.md` for what this is, how to
|
||||
install dependencies, and how to run it — this file is for working on the code: decisions,
|
||||
reasoning, conventions, and current implementation status.
|
||||
|
||||
Design/decision history lives in a long chat transcript, not in this repo yet. This file is
|
||||
the distilled result. If something here seems arbitrary, assume there was a reason discussed
|
||||
elsewhere rather than re-litigating it from scratch — but flag it if it looks wrong given the
|
||||
current code.
|
||||
|
||||
## Goal
|
||||
|
||||
Primary: follow a meeting's transcript live while it happens, so the user can stay focused on
|
||||
listening instead of note-taking. Meetings mix French and English mid-sentence
|
||||
(code-switching), include non-native English accents, and technical/internal jargon that
|
||||
should be correctable via a small per-meeting glossary.
|
||||
|
||||
**Out of scope for now, do not pull forward without being asked:**
|
||||
- Real-time content analysis / suggested questions during the meeting (explicitly phase 2).
|
||||
- Post-processing (fuzzy-match glossary correction, LLM reread via llama.cpp). The pipeline
|
||||
currently publishes qwen-asr's raw output straight to the SSE broadcaster. Revisit only
|
||||
when asked — don't propose implementing this preemptively, it's been deferred twice.
|
||||
|
||||
Only live transcription (capture → VAD → ASR → SSE) is in scope today.
|
||||
|
||||
## Target hardware
|
||||
|
||||
**MacBook (base) M3, 24 GB unified memory.** Not a high-end Max/Pro chip — size and quantize
|
||||
models accordingly (prefer 4-bit; avoid anything like an 8B+ reread model). Do not assume
|
||||
generous headroom. A separate M4 Max/128GB machine exists for development but is not the
|
||||
deployment target; don't let it justify heavier defaults.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
audiotee (subprocess, 2 outputs)
|
||||
├─ stdout → system audio track (PCM)
|
||||
└─ --mic-output → mic track (PCM, via a FIFO for live streaming, not a plain file)
|
||||
│
|
||||
▼ (per track, independently)
|
||||
VAD (simple energy-threshold heuristic, hand-rolled, no ML dependency for v1)
|
||||
│ chunks with overlap
|
||||
▼
|
||||
ASR: qwen-asr subprocess (Qwen3-ASR-1.7B, --stream, --prompt for glossary biasing)
|
||||
│ timestamped segments: {track, text, is_final, timestamp}
|
||||
▼
|
||||
merge (chronological, across both tracks)
|
||||
│
|
||||
▼
|
||||
SSE endpoint (served by this Go binary) — terminal client is the first consumer
|
||||
```
|
||||
|
||||
Post-processing (fuzzy-match glossary correction, then LLM reread via llama.cpp subprocess)
|
||||
was in the original target architecture between merge and SSE, but is explicitly deferred —
|
||||
see "Out of scope for now" above. Raw qwen-asr output is published as-is today.
|
||||
|
||||
### Why these choices (don't re-decide without reason)
|
||||
|
||||
- **Two tracks, two independent VAD+ASR pipelines** (one per track), not one merged audio
|
||||
stream. This is a cheap "me vs them" diarization proxy — no real diarization model needed.
|
||||
Rejected merging into a single flow: it was considered and reverted, keep 2+2.
|
||||
- **Go**, not Python or Swift, for this orchestrator. All the actual ML work (ASR, LLM
|
||||
reread) is delegated to external subprocess binaries regardless of orchestrator language,
|
||||
so the orchestrator's job is just: spawn subprocesses, pipe bytes, run a small HTTP/SSE
|
||||
server, merge by timestamp. That's Go's home turf (`os/exec`, `net/http` stdlib, single
|
||||
static binary, no runtime dependency). Python was rejected for its dependency/environment
|
||||
management overhead; Swift was rejected because its ML ecosystem (Silero VAD, MLX model
|
||||
loading for this specific model) is immature enough to require significant custom porting
|
||||
work that buys nothing here. Rust was a close second to Go but adds ownership/borrowing
|
||||
ceremony with no corresponding benefit for I/O-bound glue code.
|
||||
- **VAD: hand-rolled energy-threshold heuristic**, not Silero, for v1. No Swift/Go ML
|
||||
ecosystem is mature enough here to justify the integration cost. Upgrade only if
|
||||
segmentation quality turns out to be the actual bottleneck — verify before adding
|
||||
complexity.
|
||||
- **ASR runtime: `qwen-asr`** (https://github.com/antirez/qwen-asr), a pure-C implementation
|
||||
with `--stream` and `--prompt` (glossary biasing) already built in. Consumed as a
|
||||
subprocess, one instance per track. Build is trivial on macOS: no dependency install
|
||||
needed beyond the OS's built-in Accelerate framework (`make blas`). No prebuilt binaries
|
||||
are published, so it's vendored as a **git submodule** (`third_party/qwen-asr`, pinned to a
|
||||
specific commit) and built by `scripts/build-dist.sh` — this exists specifically so
|
||||
`transcriptor-ui` (the planned native app, see "Related repos") can produce a fully
|
||||
self-contained, one-click-install `.app` with no separate manual clone/build step for the
|
||||
end user; models are the only thing still fetched at runtime, deliberately (multi-GB,
|
||||
doesn't belong baked into a build). Verified empirically: `--stream` does not print
|
||||
incremental partial text to stdout for a single bounded stdin stream — only the final
|
||||
transcription, once done. Our own VAD segment boundaries are what stand in for "live"
|
||||
granularity, not qwen-asr's internal streaming. One subprocess call per VAD segment
|
||||
(measured ~0.15-0.4x realtime including process startup and model load, on both the 0.6B
|
||||
and 1.7B models) is fast enough — no need for a persistent daemon process.
|
||||
- **LLM reread runtime: `llama.cpp`**, installed via `brew install llama.cpp` (prebuilt,
|
||||
official Homebrew formula — no source build needed). Model: small, 4-bit quantized,
|
||||
sized for the 24GB target machine, not the dev machine. Not wired in yet (deferred).
|
||||
- **Live display: SSE (Server-Sent Events)**, not WebSocket. The data flow is one-directional
|
||||
(server → clients); SSE is plain HTTP, trivially consumable by a terminal client and later
|
||||
by a browser (`EventSource`, no client library needed). WebSocket would be
|
||||
over-engineering for a need that has no bidirectional requirement today.
|
||||
- **No Docker.** Ruled out on technical grounds, not preference: Docker Desktop on macOS runs
|
||||
a Linux VM with no Metal GPU passthrough, which breaks any Metal-accelerated inference, and
|
||||
Core Audio access from inside a container is impractical.
|
||||
|
||||
### Message format (SSE)
|
||||
|
||||
One JSON object per transcript segment (defined in `internal/transcript`):
|
||||
```json
|
||||
{"track": "system", "text": "...", "is_final": false, "timestamp": "2026-08-07T14:32:01.123Z"}
|
||||
```
|
||||
`is_final: false` = a streaming ASR partial hypothesis that may still be revised.
|
||||
`is_final: true` = the segment is closed (VAD ended the utterance).
|
||||
|
||||
No timestamp-based reordering across tracks yet — messages are broadcast in the order
|
||||
`transcribeTrack` produces them. Measured ASR latency is small relative to segment spacing
|
||||
(3-5s), so this hasn't been observed to cause visible out-of-order delivery. Add a small
|
||||
reordering buffer in `internal/transcript` if it ever becomes a real problem — not before.
|
||||
|
||||
## Related repos
|
||||
|
||||
`../audiotee` — the Swift CLI this project spawns as a subprocess for audio capture (see
|
||||
`internal/capture`). Consumed as a subprocess deliberately, not imported as a Swift library
|
||||
(`AudioTeeCore` is exposed as one, but that only matters for a Swift consumer, which this
|
||||
isn't) — see its own `CONTEXT.md` §6.2 for why that repo's TCC-identity signing work matters
|
||||
specifically because of this. Two tracks (system audio + mic) go to two separate outputs
|
||||
(stdout + a FIFO), never multiplexed onto one stream — two concurrent Core Audio IO threads
|
||||
writing raw PCM chunks to a shared fd risked interleaving/corruption.
|
||||
|
||||
`../transcriptor-ui` — a native macOS SwiftUI app (separate Xcode project), the "one-click"
|
||||
way to actually use this — **built and working as of 2026-08-08**, not just planned anymore.
|
||||
It spawns `transcriptor-ai` as a subprocess (same pattern as `transcriptor-ai` → `audiotee` —
|
||||
each layer manages the one below it via a subprocess + a clean protocol, never merged into
|
||||
one project/toolchain) and is an SSE client of it, same as `transcript-tail`, just with a
|
||||
native UI instead of a terminal. Its Xcode build phase builds this repo's `scripts/build-dist.sh`
|
||||
and `../audiotee`'s `scripts/build-signed.sh` and embeds all three binaries in the `.app`
|
||||
bundle — verified fully self-contained (tested with `~/bin/audiotee` moved aside, capture
|
||||
still worked from the embedded copy). Deliberately **not** a monorepo with this one — see
|
||||
"Why these choices": mixing Go and Swift tooling in one Xcode project would undermine the
|
||||
reason Go was chosen for this repo in the first place. transcriptor-ui will eventually let
|
||||
the user pick ASR model/options and configure the glossary and post-processing
|
||||
(question-detection, meeting-summary generation via a local LLM) — model selection is still a
|
||||
dev-time hardcoded path there, nothing else is wired up yet; check transcriptor-ui's own
|
||||
CLAUDE.md for its current state rather than assuming.
|
||||
|
||||
**`-audiotee-binary` flag** exists specifically for transcriptor-ui's benefit: without it,
|
||||
`internal/capture` only finds `audiotee` via `PATH` then `~/bin/audiotee`, which is fine for
|
||||
CLI use but wrong for a bundled app that must not depend on anything pre-installed outside
|
||||
its own `.app` — transcriptor-ui passes its embedded copy's path explicitly.
|
||||
|
||||
## Conventions
|
||||
|
||||
- Go idioms: `gofmt`/`go vet` clean, standard project layout (`cmd/`, `internal/`).
|
||||
- Prefer the standard library over third-party Go modules where it's a reasonably close fit
|
||||
(`os/exec`, `net/http`, `encoding/json`) — the "self-contained binary, no dependency bazar"
|
||||
property is a deliberate, stated goal of this project, not an accident.
|
||||
- Code and comments in English. Conversation with the user in French (matches the sibling
|
||||
`audiotee` repo's convention).
|
||||
- Don't reintroduce Python, Swift, Docker, or WebSocket for this project without a new,
|
||||
explicit reason — see "Why these choices" above.
|
||||
- User-facing install/usage docs belong in `README.md`, not here — keep this file about
|
||||
reasoning and internal state, not instructions for running the tool.
|
||||
|
||||
## Status
|
||||
|
||||
Capture → VAD → ASR → SSE is implemented and verified end to end:
|
||||
- `internal/capture` — audiotee subprocess wrapper (stdout + mic FIFO).
|
||||
- `internal/vad` — energy-threshold segmentation.
|
||||
- `internal/asr` — one qwen_asr subprocess call per segment.
|
||||
- `internal/transcript` — `Broadcaster` (SSE), `FileWriter` (durable text file, one line per
|
||||
message, fsync'd on every write), `Message` format shared by both.
|
||||
- `cmd/transcriptor-ai` — wires all of the above together; currently also the test harness.
|
||||
- `cmd/transcript-tail` — minimal SSE terminal client. Exits calmly (code 0, "stream ended
|
||||
(server stopped?)") when the server shuts down, instead of an alarming raw connection error.
|
||||
- `scripts/build-dist.sh` — builds `qwen_asr` (from the `third_party/qwen-asr` submodule) and
|
||||
`transcriptor-ai` into `dist/`, ready for `transcriptor-ui` to embed. Verified: builds
|
||||
cleanly from the submodule, and the resulting `dist/qwen_asr` actually transcribes
|
||||
(smoke-tested), not just "it compiled."
|
||||
- `-capture-system`/`-capture-mic` flags (transcriptor-ui's benefit: a UI toggle for what to
|
||||
capture). **Asymmetric by necessity, document this if it ever confuses someone**:
|
||||
`-capture-mic=false` skips mic capture at the `audiotee` level entirely (no
|
||||
`--capture-mic` passed, no mic permission requested) — `internal/capture.Config.CaptureMic`
|
||||
controls this. `-capture-system=false` **cannot** do the equivalent — audiotee has no flag
|
||||
to skip its system tap, it's always running — so system-only-disabled just means
|
||||
`transcriptor-ai` receives system chunks (VAD still runs on them, negligible cost) but never
|
||||
dispatches them to an ASR worker (`main.go`'s `dispatch` helper drops them). Verified all
|
||||
three modes (system-only, mic-only, both) via CLI: mic-only correctly showed zero
|
||||
`track: system` messages even with system audio playing, confirming the drop works, not
|
||||
just the flag parsing.
|
||||
|
||||
Decision (2026-08-08): the future native macOS UI (`transcriptor-ui`, a separate repo/Xcode
|
||||
project — see below) will *not* write the transcript file itself; `transcriptor-ai` does, via
|
||||
`-transcript-file`, specifically because it's the more crash-resistant of the two (simpler
|
||||
control flow than a UI process) and the whole point of this file is surviving a crash.
|
||||
|
||||
Verified: real human speech (English + French code-switching) through an actual microphone,
|
||||
with both qwen3-asr-0.6b (fast iteration) and qwen3-asr-1.7b (the target model). 1.7B
|
||||
measured at ~0.4x realtime (2.3s to process a 5.9s clip, model load included) — comfortably
|
||||
fast enough for one-subprocess-per-segment. Switching models is a runtime flag
|
||||
(`-asr-model-dir`), not a rebuild.
|
||||
|
||||
Three concurrency/signal-handling bugs were found and fixed via real testing (not just code
|
||||
review):
|
||||
- `transcribeTrack` was using the SIGINT-canceled context for still-queued transcriptions
|
||||
(fixed with a separate background context for ASR calls).
|
||||
- `main()` wasn't waiting for `transcribeTrack` goroutines to finish before exiting (fixed
|
||||
with a `sync.WaitGroup`).
|
||||
- `internal/capture.Stop()` called `cmd.Wait()` before the pipe-reading goroutines had
|
||||
finished draining, which `os/exec` docs call out as incorrect and can truncate the last bit
|
||||
of audio.
|
||||
- **`qwen_asr` subprocesses were killed by the terminal's own Ctrl+C**, independent of any of
|
||||
the above: child processes inherit the parent's process group by default, and a real
|
||||
terminal's Ctrl+C sends SIGINT to the whole foreground process group — not just to
|
||||
`transcriptor-ai`. So an in-flight `qwen_asr` call died with "signal: interrupt" regardless
|
||||
of the context-handling fix above, which only guards against *our own code* cancelling it.
|
||||
Fixed in `internal/asr/runner.go` with `SysProcAttr.Setpgid: true`, putting each `qwen_asr`
|
||||
in its own process group. **This bug was invisible to `kill -INT <transcriptor-ai-pid>`**,
|
||||
the way earlier manual tests in this session were done — that only signals one PID, not a
|
||||
process group, so it never reproduced the real terminal behavior. Verified instead by
|
||||
sending SIGINT to the process group (`os.killpg` from a small Python harness, or
|
||||
`kill -INT -<pgid>` — careful with the latter run from an interactive shell, it'll also hit
|
||||
the shell itself if unrelated jobs share its group).
|
||||
|
||||
- **Duplicate text at force-cut segment boundaries**, found from real usage on a real
|
||||
meeting-like recording (documentary audio) — visibly bad enough that the transcript was
|
||||
unusable, e.g. "à vivre" transcribed once at the end of a segment and again at the start of
|
||||
the next. Root cause: the 300ms `Overlap` audio carried into the next force-cut segment (for
|
||||
ASR context continuity) was being fully re-transcribed *and* re-published, so both copies
|
||||
showed up in the live transcript. Fixed by defaulting `vad.Config.Overlap` to `0` — accept
|
||||
an occasional word cut in half at a hard boundary rather than systematic duplication, which
|
||||
is far worse for actually reading the transcript. Found and fixed alongside it: `tail()` in
|
||||
`internal/vad/vad.go` treated `n <= 0` as "return the whole buffer" instead of empty, which
|
||||
would have made `Overlap: 0` carry over the *entire* previous segment instead of nothing —
|
||||
fixed to return `nil` for `n <= 0`. If overlap is ever reintroduced, it needs text-level
|
||||
dedup on the published output, not just raw audio carry-over.
|
||||
- **System and mic tracks transcribing the same content** — not a code bug, a structural
|
||||
limit of tapping raw Core Audio input: if audio plays through speakers (not headphones),
|
||||
the mic picks up the acoustic leakage and transcribes it too. Conferencing apps do their own
|
||||
acoustic echo cancellation on the signal they send to other participants, but that doesn't
|
||||
touch what our raw mic tap sees. Real-world implication: headphones are likely a hard
|
||||
requirement for the two-track "me vs them" diarization to actually hold during real
|
||||
meetings, not just a nice-to-have — unresolved, needs a decision (require headphones, or
|
||||
build real AEC — a much bigger feature) rather than a quick fix.
|
||||
- **Shutdown could feel unresponsive** during a large in-flight transcription backlog (no
|
||||
feedback while waiting, especially on the slower 1.7B model). Fixed with a "shutting down,
|
||||
waiting for in-flight transcriptions..." message and a second Ctrl+C now force-exits
|
||||
immediately instead of making the user wait through the full drain.
|
||||
|
||||
Not yet implemented: post-processing (deferred, see "Out of scope for now"), and anything
|
||||
from `audiotee`'s CONTEXT.md §6.4-adjacent concerns beyond what's listed here.
|
||||
|
||||
**Resume point (2026-08-07):** the live pipeline is done and validated on real speech with
|
||||
both models, and the force-cut duplication bug is fixed and verified. The system/mic acoustic
|
||||
leakage issue (headphones vs AEC) is still an open decision. No specific next task was chosen
|
||||
before pausing beyond that — ask the user
|
||||
rather than assuming which.
|
||||
@@ -0,0 +1,120 @@
|
||||
# transcriptor-ai
|
||||
|
||||
Local, no-cloud meeting transcription for macOS. Captures system audio and microphone as two
|
||||
separate tracks, transcribes both live, and serves a live-updating transcript over HTTP (SSE).
|
||||
Everything runs on-device — no external API calls at runtime.
|
||||
|
||||
## How it works
|
||||
|
||||
```
|
||||
audiotee (system audio + mic, 2 tracks)
|
||||
→ VAD (speech segmentation, per track)
|
||||
→ qwen-asr (speech-to-text, per track)
|
||||
→ live transcript, served over SSE
|
||||
```
|
||||
|
||||
Two tracks (system audio vs microphone) act as a cheap "me vs them" diarization: whatever you
|
||||
hear vs whatever you say, without needing a real speaker-diarization model.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- macOS (Apple Silicon)
|
||||
- Go 1.26+ (`brew install go`)
|
||||
- **[audiotee](../audiotee)** — built and signed with a stable identity so its audio
|
||||
permissions survive rebuilds. Follow that repo's own README/CONTEXT.md first; you should
|
||||
end up with a working binary, e.g. at `~/bin/audiotee`.
|
||||
- **qwen-asr** — vendored as a git submodule (`third_party/qwen-asr`, pinned to a specific
|
||||
commit), built automatically by `scripts/build-dist.sh` below. You still need to download a
|
||||
model yourself (not part of the build — see "Models"), since that's a multi-GB download,
|
||||
not something to fetch on every build:
|
||||
```bash
|
||||
git submodule update --init third_party/qwen-asr # if you haven't run build-dist.sh yet
|
||||
cd third_party/qwen-asr && ./download_model.sh --model large # or --model small
|
||||
```
|
||||
|
||||
`llama.cpp` (`brew install llama.cpp`) will be needed once post-processing (glossary
|
||||
correction, LLM reread) is implemented, but isn't used yet — no need to install it today.
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
git clone --recurse-submodules <this repo> # or: git submodule update --init
|
||||
scripts/build-dist.sh
|
||||
```
|
||||
Builds `qwen_asr` (from the submodule) and `transcriptor-ai` into `dist/`. Also build
|
||||
`transcript-tail` directly, it's not part of `dist/` (that's for what `transcriptor-ui`, the
|
||||
future native app, embeds — see `CLAUDE.md`):
|
||||
```bash
|
||||
go build -o transcript-tail ./cmd/transcript-tail
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Start the transcriber:
|
||||
|
||||
```bash
|
||||
./dist/transcriptor-ai \
|
||||
-asr-binary ./dist/qwen_asr \
|
||||
-asr-model-dir ./third_party/qwen-asr/qwen3-asr-1.7b \
|
||||
-transcript-file ~/Desktop/meeting-transcript.txt
|
||||
```
|
||||
|
||||
In another terminal, follow the live transcript:
|
||||
|
||||
```bash
|
||||
./transcript-tail
|
||||
```
|
||||
|
||||
Output looks like:
|
||||
```
|
||||
[mic ] Bonjour, ceci est un test.
|
||||
[system…] Hello, this is a partial hypothesis that may still be revised
|
||||
```
|
||||
(no marker = a closed/final segment; `…` = a partial hypothesis, ASR-in-progress)
|
||||
|
||||
Stop with Ctrl+C — shutdown is graceful (in-flight transcriptions finish before exiting).
|
||||
|
||||
### Flags
|
||||
|
||||
`transcriptor-ai`:
|
||||
|
||||
| Flag | Default | Description |
|
||||
|---|---|---|
|
||||
| `-audiotee-binary` | PATH, then `~/bin/audiotee` | Path to the `audiotee` binary |
|
||||
| `-asr-binary` | *(required)* | Path to the `qwen_asr` binary |
|
||||
| `-asr-model-dir` | *(required)* | Path to a qwen-asr model directory |
|
||||
| `-prompt` | *(empty)* | Glossary biasing text passed to qwen-asr (e.g. `"Preserve spelling: Kubernetes, PostgreSQL"`) |
|
||||
| `-http-addr` | `:8420` | Address to serve the live transcript SSE endpoint on |
|
||||
| `-transcript-file` | *(empty)* | Path to append the live transcript to as plain text, flushed to disk after every line. Not written if empty. |
|
||||
| `-capture-system` | `true` | Transcribe the system audio track. Note: audiotee's system tap always runs regardless of this flag — disabling it only stops *transcribing* that track, it doesn't skip the underlying capture (audiotee has no flag for that). |
|
||||
| `-capture-mic` | `true` | Capture and transcribe the microphone track. Unlike `-capture-system`, disabling this actually skips mic capture entirely (no `--capture-mic` passed to audiotee, no microphone permission requested). |
|
||||
|
||||
At least one of `-capture-system`/`-capture-mic` must stay enabled.
|
||||
|
||||
`transcript-tail`:
|
||||
|
||||
| Flag | Default | Description |
|
||||
|---|---|---|
|
||||
| `-url` | `http://localhost:8420/events` | transcriptor-ai SSE endpoint to follow |
|
||||
|
||||
### Consuming the transcript programmatically
|
||||
|
||||
The SSE endpoint (`http://localhost:8420/events` by default) streams one JSON object per
|
||||
transcript segment:
|
||||
```json
|
||||
{"track": "system", "text": "...", "is_final": false, "timestamp": "2026-08-07T14:32:01.123Z"}
|
||||
```
|
||||
Any SSE client works, e.g. `curl -N http://localhost:8420/events`, or a browser's
|
||||
`EventSource`.
|
||||
|
||||
## Status
|
||||
|
||||
Live transcription (capture → VAD → ASR → SSE) works end to end, tested with real speech
|
||||
(English + French, code-switching) through an actual microphone. Not yet implemented:
|
||||
post-processing (glossary fuzzy-matching, LLM reread) and real-time content analysis — see
|
||||
`CLAUDE.md` for what's deliberately deferred and why.
|
||||
|
||||
## Related
|
||||
|
||||
- [audiotee](../audiotee) — the system/mic audio capture CLI this project depends on.
|
||||
- [antirez/qwen-asr](https://github.com/antirez/qwen-asr) — the speech-to-text engine used.
|
||||
@@ -0,0 +1,66 @@
|
||||
// Command transcript-tail is a minimal terminal client for transcriptor-ai's
|
||||
// live SSE transcript endpoint — connects, parses each `data: {...}` event,
|
||||
// and prints just the text, instead of raw JSON (`curl -N` works too, but
|
||||
// isn't pleasant to read live). See CLAUDE.md for the SSE message format.
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"flag"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"github.com/stephanetailland/transcriptor-ai/internal/transcript"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if err := run(); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "transcript-tail:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func run() error {
|
||||
url := flag.String("url", "http://localhost:8420/events", "transcriptor-ai SSE endpoint to follow")
|
||||
flag.Parse()
|
||||
|
||||
resp, err := http.Get(*url)
|
||||
if err != nil {
|
||||
return fmt.Errorf("connect: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("unexpected status: %s", resp.Status)
|
||||
}
|
||||
|
||||
scanner := bufio.NewScanner(resp.Body)
|
||||
for scanner.Scan() {
|
||||
data, ok := strings.CutPrefix(scanner.Text(), "data: ")
|
||||
if !ok {
|
||||
continue // SSE keep-alive/blank lines, comments, etc.
|
||||
}
|
||||
var msg transcript.Message
|
||||
if err := json.Unmarshal([]byte(data), &msg); err != nil {
|
||||
continue
|
||||
}
|
||||
printMessage(msg)
|
||||
}
|
||||
// The connection ending here — whether cleanly or as a network-level
|
||||
// error — almost always just means the transcriptor-ai server stopped.
|
||||
// We can't reliably tell that apart from a real connection problem, so
|
||||
// don't be alarmist about it: this isn't a failure of transcript-tail
|
||||
// itself, the way a startup connection error (above) is.
|
||||
fmt.Fprintln(os.Stderr, "transcript-tail: stream ended (server stopped?)")
|
||||
return nil
|
||||
}
|
||||
|
||||
func printMessage(msg transcript.Message) {
|
||||
marker := " "
|
||||
if !msg.IsFinal {
|
||||
marker = "…" // still a partial hypothesis, may be revised
|
||||
}
|
||||
fmt.Printf("[%s%s] %s\n", msg.Track, marker, msg.Text)
|
||||
}
|
||||
@@ -0,0 +1,278 @@
|
||||
// Command transcriptor-ai orchestrates local, real-time meeting transcription:
|
||||
// it drives audiotee for audio capture, qwen-asr for speech recognition, and
|
||||
// serves the resulting transcript live over SSE. See CLAUDE.md for the full
|
||||
// architecture and the reasoning behind it.
|
||||
//
|
||||
// This is a capture+VAD+ASR+SSE test harness: it runs audiotee, feeds
|
||||
// detected speech segments to qwen_asr (one subprocess call per segment, one
|
||||
// worker per track), and publishes the results both to stdout and to an SSE
|
||||
// endpoint (`curl -N http://<http-addr>/events` is the terminal client for
|
||||
// now — see CLAUDE.md on why SSE, not WebSocket). Post-processing (fuzzy
|
||||
// glossary correction, LLM reread) isn't wired in yet.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/stephanetailland/transcriptor-ai/internal/asr"
|
||||
"github.com/stephanetailland/transcriptor-ai/internal/capture"
|
||||
"github.com/stephanetailland/transcriptor-ai/internal/transcript"
|
||||
"github.com/stephanetailland/transcriptor-ai/internal/vad"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if err := run(); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "transcriptor-ai:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func run() error {
|
||||
audioteeBinary := flag.String("audiotee-binary", "", "path to the audiotee binary (optional; defaults to PATH then ~/bin/audiotee — see internal/capture)")
|
||||
asrBinary := flag.String("asr-binary", "", "path to the qwen_asr binary (required)")
|
||||
asrModelDir := flag.String("asr-model-dir", "", "path to the qwen_asr model directory (required)")
|
||||
prompt := flag.String("prompt", "", "glossary biasing prompt passed to qwen_asr")
|
||||
httpAddr := flag.String("http-addr", ":8420", "address to serve the live transcript SSE endpoint on")
|
||||
transcriptFile := flag.String("transcript-file", "", "path to append the live transcript to as plain text (optional; not written if empty)")
|
||||
captureSystem := flag.Bool("capture-system", true, "transcribe the system audio track (audiotee's system tap always runs regardless of this flag — disabling it only stops transcribing that track, see internal/capture.Config.CaptureMic doc)")
|
||||
captureMic := flag.Bool("capture-mic", true, "capture and transcribe the microphone track")
|
||||
flag.Parse()
|
||||
|
||||
if *asrBinary == "" || *asrModelDir == "" {
|
||||
return fmt.Errorf("both -asr-binary and -asr-model-dir are required")
|
||||
}
|
||||
if !*captureSystem && !*captureMic {
|
||||
return fmt.Errorf("at least one of -capture-system or -capture-mic must be enabled")
|
||||
}
|
||||
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
systemPath := filepath.Join(os.TempDir(), "transcriptor-ai-system.pcm")
|
||||
micPath := filepath.Join(os.TempDir(), "transcriptor-ai-mic.pcm")
|
||||
|
||||
systemOut, err := os.Create(systemPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create system output file: %w", err)
|
||||
}
|
||||
defer systemOut.Close()
|
||||
|
||||
micOut, err := os.Create(micPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create mic output file: %w", err)
|
||||
}
|
||||
defer micOut.Close()
|
||||
|
||||
var fileWriter *transcript.FileWriter
|
||||
if *transcriptFile != "" {
|
||||
fileWriter, err = transcript.NewFileWriter(*transcriptFile)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer fileWriter.Close()
|
||||
fmt.Fprintln(os.Stderr, "transcript file ->", *transcriptFile)
|
||||
}
|
||||
|
||||
broadcaster := transcript.NewBroadcaster()
|
||||
httpServer := &http.Server{Addr: *httpAddr, Handler: sseMux(broadcaster)}
|
||||
httpErrs := make(chan error, 1)
|
||||
go func() {
|
||||
if err := httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||
httpErrs <- err
|
||||
}
|
||||
}()
|
||||
|
||||
const sampleRate = 16000
|
||||
c := capture.New(capture.Config{SampleRate: sampleRate, AudioteePath: *audioteeBinary, CaptureMic: *captureMic})
|
||||
if err := c.Start(ctx); err != nil {
|
||||
return fmt.Errorf("start capture: %w", err)
|
||||
}
|
||||
|
||||
fmt.Fprintln(os.Stderr, "capturing... Ctrl+C to stop")
|
||||
fmt.Fprintln(os.Stderr, "system track ->", systemPath)
|
||||
fmt.Fprintln(os.Stderr, "mic track ->", micPath)
|
||||
fmt.Fprintln(os.Stderr, "live transcript (SSE) -> curl -N http://"+httpAddrForDisplay(*httpAddr)+"/events")
|
||||
|
||||
// Tee raw chunks to disk (for listening back later) while also feeding
|
||||
// them to VAD. Buffered so the tee doesn't block segmentation.
|
||||
tee := make(chan capture.Chunk, 64)
|
||||
go func() {
|
||||
defer close(tee)
|
||||
for chunk := range c.Chunks() {
|
||||
switch chunk.Track {
|
||||
case capture.TrackSystem:
|
||||
systemOut.Write(chunk.Data)
|
||||
case capture.TrackMic:
|
||||
micOut.Write(chunk.Data)
|
||||
}
|
||||
tee <- chunk
|
||||
}
|
||||
}()
|
||||
|
||||
detector := vad.New(vad.DefaultConfig(sampleRate))
|
||||
segments := detector.Run(tee)
|
||||
|
||||
// Two independent transcribers (2 tracks, 2 ASR engines — see
|
||||
// CLAUDE.md), each processing its own track's segments sequentially. A
|
||||
// fresh qwen_asr subprocess per segment is fast enough (~0.15-0.2x
|
||||
// realtime measured) that this doesn't need to be more concurrent than
|
||||
// that.
|
||||
transcriber := asr.New(asr.Config{BinaryPath: *asrBinary, ModelDir: *asrModelDir, Prompt: *prompt})
|
||||
// transcribeTrack uses its own background context, not the
|
||||
// shutdown-signal ctx above: segments still queued when Ctrl+C arrives
|
||||
// (including the ones flushed by c.Stop() below) must still get
|
||||
// transcribed during the graceful drain, not fail with "context
|
||||
// canceled" because the shutdown signal already canceled ctx.
|
||||
asrCtx := context.Background()
|
||||
var systemSegs, micSegs chan vad.Segment
|
||||
var wg sync.WaitGroup
|
||||
if *captureSystem {
|
||||
systemSegs = make(chan vad.Segment, 8)
|
||||
wg.Add(1)
|
||||
go func() { defer wg.Done(); transcribeTrack(asrCtx, transcriber, systemSegs, broadcaster, fileWriter) }()
|
||||
}
|
||||
if *captureMic {
|
||||
micSegs = make(chan vad.Segment, 8)
|
||||
wg.Add(1)
|
||||
go func() { defer wg.Done(); transcribeTrack(asrCtx, transcriber, micSegs, broadcaster, fileWriter) }()
|
||||
}
|
||||
|
||||
loop:
|
||||
for {
|
||||
select {
|
||||
case seg, ok := <-segments:
|
||||
if !ok {
|
||||
break loop
|
||||
}
|
||||
logSegment(seg)
|
||||
dispatch(seg, systemSegs, micSegs)
|
||||
case err := <-c.Errs():
|
||||
fmt.Fprintln(os.Stderr, "capture error:", err)
|
||||
case err := <-httpErrs:
|
||||
fmt.Fprintln(os.Stderr, "http server error:", err)
|
||||
case <-ctx.Done():
|
||||
break loop
|
||||
}
|
||||
}
|
||||
|
||||
// A graceful shutdown can take a few seconds (draining queued segments
|
||||
// through qwen_asr, up to ~2s each on the 1.7B model) with no visible
|
||||
// progress otherwise, which reads as "hung" — say so, and let a second
|
||||
// Ctrl+C skip the wait and exit immediately instead of forcing the user
|
||||
// to keep hitting it.
|
||||
fmt.Fprintln(os.Stderr, "shutting down, waiting for in-flight transcriptions to finish (Ctrl+C again to force quit)...")
|
||||
forceQuit := make(chan os.Signal, 1)
|
||||
signal.Notify(forceQuit, syscall.SIGINT, syscall.SIGTERM)
|
||||
go func() {
|
||||
<-forceQuit
|
||||
fmt.Fprintln(os.Stderr, "\nforcing immediate exit")
|
||||
os.Exit(1)
|
||||
}()
|
||||
|
||||
c.Stop()
|
||||
// Drain any segments flushed by Stop() closing the chunk stream.
|
||||
for seg := range segments {
|
||||
logSegment(seg)
|
||||
dispatch(seg, systemSegs, micSegs)
|
||||
}
|
||||
|
||||
// Closing a nil channel panics — only close the ones actually created
|
||||
// (i.e. for tracks that were enabled).
|
||||
if systemSegs != nil {
|
||||
close(systemSegs)
|
||||
}
|
||||
if micSegs != nil {
|
||||
close(micSegs)
|
||||
}
|
||||
wg.Wait() // let in-flight/queued transcriptions finish before exiting, and
|
||||
// before their results are published — otherwise a connected SSE client
|
||||
// would miss the tail end of the transcript.
|
||||
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
_ = httpServer.Shutdown(shutdownCtx)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// dispatch routes seg to the channel for its track. Sending to a nil
|
||||
// channel (a disabled track — see -capture-system/-capture-mic) blocks
|
||||
// forever in Go, so it must be checked, not just sent to unconditionally;
|
||||
// a disabled track's segments are silently dropped (VAD still runs on
|
||||
// every track's chunks regardless — negligible cost — but a disabled
|
||||
// track never reaches ASR).
|
||||
func dispatch(seg vad.Segment, systemSegs, micSegs chan<- vad.Segment) {
|
||||
switch seg.Track {
|
||||
case capture.TrackSystem:
|
||||
if systemSegs != nil {
|
||||
systemSegs <- seg
|
||||
}
|
||||
case capture.TrackMic:
|
||||
if micSegs != nil {
|
||||
micSegs <- seg
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func sseMux(b *transcript.Broadcaster) http.Handler {
|
||||
mux := http.NewServeMux()
|
||||
mux.Handle("/events", b)
|
||||
return mux
|
||||
}
|
||||
|
||||
// httpAddrForDisplay turns a ":8420"-style listen address into something
|
||||
// pasteable ("localhost:8420") for the printed curl hint.
|
||||
func httpAddrForDisplay(addr string) string {
|
||||
if len(addr) > 0 && addr[0] == ':' {
|
||||
return "localhost" + addr
|
||||
}
|
||||
return addr
|
||||
}
|
||||
|
||||
func transcribeTrack(ctx context.Context, t *asr.Transcriber, segs <-chan vad.Segment, b *transcript.Broadcaster, fw *transcript.FileWriter) {
|
||||
for seg := range segs {
|
||||
text, err := t.Transcribe(ctx, seg.Data)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "[%s] asr error: %v\n", seg.Track, err)
|
||||
continue
|
||||
}
|
||||
if text == "" {
|
||||
continue
|
||||
}
|
||||
msg := transcript.Message{Track: seg.Track, Text: text, IsFinal: seg.IsFinal, Timestamp: seg.End}
|
||||
b.Publish(msg)
|
||||
if fw != nil {
|
||||
// Loud, not fatal: losing one line to a transient write error
|
||||
// shouldn't take down a live transcription session — but stay
|
||||
// visible, since silent data loss would defeat the point of
|
||||
// this file existing at all.
|
||||
if err := fw.Write(msg); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "[%s] transcript file write error: %v\n", seg.Track, err)
|
||||
}
|
||||
}
|
||||
|
||||
finality := "final"
|
||||
if !seg.IsFinal {
|
||||
finality = "partial"
|
||||
}
|
||||
fmt.Printf("[%s/%s] %s\n", seg.Track, finality, text)
|
||||
}
|
||||
}
|
||||
|
||||
func logSegment(seg vad.Segment) {
|
||||
kind := "final"
|
||||
if !seg.IsFinal {
|
||||
kind = "force-cut"
|
||||
}
|
||||
fmt.Fprintf(os.Stderr, "[%s] %-6s dur=%-6s bytes=%d (%s)\n",
|
||||
seg.Start.Format("15:04:05.000"), seg.Track, seg.End.Sub(seg.Start).Round(10*time.Millisecond), len(seg.Data), kind)
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
// Package asr transcribes bounded PCM segments via qwen-asr
|
||||
// (https://github.com/antirez/qwen-asr), invoked as a one-shot subprocess
|
||||
// per segment rather than a long-lived daemon.
|
||||
//
|
||||
// This was verified empirically, not assumed: `qwen_asr --stdin --stream
|
||||
// --silent` does not print incremental partial text as it processes a
|
||||
// single stdin stream — it prints only the final transcription once done,
|
||||
// even in --stream mode (that flag changes its internal chunked-encoding
|
||||
// strategy, not what gets written to stdout). Measured latency including
|
||||
// process startup and model load, on the 0.6B model, was ~0.15-0.2x
|
||||
// realtime for a ~6s clip — comfortably fast enough to spawn fresh per VAD
|
||||
// segment instead of keeping a persistent process fed over a pipe.
|
||||
// Our own VAD segment boundaries (internal/vad) are therefore what stands
|
||||
// in for "live" here: each vad.Segment.IsFinal maps directly to the
|
||||
// SSE message's is_final field described in CLAUDE.md.
|
||||
package asr
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Config points at a built qwen_asr binary and a downloaded model
|
||||
// directory. Neither has an established default location yet (unlike
|
||||
// audiotee's ~/bin convention), so both are required.
|
||||
type Config struct {
|
||||
BinaryPath string
|
||||
ModelDir string
|
||||
// Prompt biases transcription toward glossary terms, e.g.
|
||||
// "Preserve spelling: Kubernetes, PostgreSQL". Optional.
|
||||
Prompt string
|
||||
// Threads sets qwen_asr's -t flag. Zero means "let it pick" (all CPUs).
|
||||
Threads int
|
||||
}
|
||||
|
||||
type Transcriber struct {
|
||||
cfg Config
|
||||
run runFunc // seam for testing without a real binary
|
||||
}
|
||||
|
||||
func New(cfg Config) *Transcriber {
|
||||
return &Transcriber{cfg: cfg, run: runCommand}
|
||||
}
|
||||
|
||||
// Transcribe runs qwen_asr once against pcm (raw s16le, 16kHz, mono — the
|
||||
// format both audiotee and internal/vad produce) and returns the
|
||||
// transcribed text, trimmed.
|
||||
func (t *Transcriber) Transcribe(ctx context.Context, pcm []byte) (string, error) {
|
||||
args := []string{"-d", t.cfg.ModelDir, "--stdin", "--stream", "--silent"}
|
||||
if t.cfg.Prompt != "" {
|
||||
args = append(args, "--prompt", t.cfg.Prompt)
|
||||
}
|
||||
if t.cfg.Threads > 0 {
|
||||
args = append(args, "-t", strconv.Itoa(t.cfg.Threads))
|
||||
}
|
||||
|
||||
stdout, stderr, err := t.run(ctx, t.cfg.BinaryPath, args, pcm)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("qwen_asr: %w (stderr: %s)", err, bytes.TrimSpace(stderr))
|
||||
}
|
||||
text := strings.TrimSpace(string(stdout))
|
||||
|
||||
if looksLikePromptLeak(text, t.cfg.Prompt) {
|
||||
// Discard rather than return it as a genuine transcription — see
|
||||
// leak.go. Logged here (not just silently dropped) since this
|
||||
// package has no structured logger of its own; matches the plain
|
||||
// stderr diagnostics the CLI already prints elsewhere.
|
||||
fmt.Fprintf(os.Stderr, "asr: discarding segment, looks like a --prompt leak, not a transcription: %q\n", text)
|
||||
return "", nil
|
||||
}
|
||||
|
||||
return text, nil
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
package asr
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
// looksLikePromptLeak reports whether text appears to be the model echoing
|
||||
// the --prompt biasing content back verbatim instead of genuinely
|
||||
// transcribing the segment's audio — a known failure mode of
|
||||
// prompt-conditioned generation, confirmed via real testing against
|
||||
// qwen_asr (see transcriptor-ui/CLAUDE.md, "Glossary prompt-leakage bug"):
|
||||
// a whole segment came back as literally the glossary terms, twice, on two
|
||||
// separate 5-minute test runs, even after switching to the tool's
|
||||
// recommended "Preserve spelling: ..." prompt framing (which reduces but
|
||||
// does not eliminate the risk).
|
||||
//
|
||||
// Heuristic: if most of text's words also appear among prompt's words, and
|
||||
// there are enough of them to rule out a legitimate short mention of one
|
||||
// or two glossary terms in real speech, treat it as a leak. Both observed
|
||||
// real leaks were near-total reproductions of the prompt's term list, so a
|
||||
// strict threshold (80% word overlap, at least 4 words) catches them
|
||||
// without plausibly false-flagging a normal sentence that happens to
|
||||
// mention "Claude" or "Mistral" once — normal speech has enough other
|
||||
// words to keep the overlap ratio well below that.
|
||||
func looksLikePromptLeak(text, prompt string) bool {
|
||||
if prompt == "" || text == "" {
|
||||
return false
|
||||
}
|
||||
textWords := normalizeWords(text)
|
||||
if len(textWords) < 4 {
|
||||
return false
|
||||
}
|
||||
promptWords := wordSet(normalizeWords(prompt))
|
||||
|
||||
matched := 0
|
||||
for _, w := range textWords {
|
||||
if promptWords[w] {
|
||||
matched++
|
||||
}
|
||||
}
|
||||
return float64(matched)/float64(len(textWords)) >= 0.8
|
||||
}
|
||||
|
||||
// normalizeWords lowercases and splits on anything that isn't a letter or
|
||||
// digit — unicode.IsLetter (not an ASCII-only check) so accented
|
||||
// characters (French: é, è, ï, ç, ...) stay part of their word instead of
|
||||
// being split into fragments.
|
||||
func normalizeWords(s string) []string {
|
||||
return strings.FieldsFunc(strings.ToLower(s), func(r rune) bool {
|
||||
return !unicode.IsLetter(r) && !unicode.IsDigit(r)
|
||||
})
|
||||
}
|
||||
|
||||
func wordSet(words []string) map[string]bool {
|
||||
set := make(map[string]bool, len(words))
|
||||
for _, w := range words {
|
||||
set[w] = true
|
||||
}
|
||||
return set
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
package asr
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"os/exec"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
// runFunc executes binary with args, feeding stdin, and returns
|
||||
// stdout/stderr. It's a seam so tests can substitute a fake process instead
|
||||
// of requiring the real qwen_asr binary and a multi-GB model on disk.
|
||||
type runFunc func(ctx context.Context, binary string, args []string, stdin []byte) (stdout, stderr []byte, err error)
|
||||
|
||||
func runCommand(ctx context.Context, binary string, args []string, stdin []byte) ([]byte, []byte, error) {
|
||||
cmd := exec.CommandContext(ctx, binary, args...)
|
||||
cmd.Stdin = bytes.NewReader(stdin)
|
||||
var stdout, stderr bytes.Buffer
|
||||
cmd.Stdout = &stdout
|
||||
cmd.Stderr = &stderr
|
||||
// Run qwen_asr in its own process group so the terminal's Ctrl+C
|
||||
// (SIGINT to the whole foreground process group) doesn't kill an
|
||||
// in-flight transcription directly. It's still cancelable through ctx,
|
||||
// same as before — this only stops the shell from also signaling it.
|
||||
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
|
||||
err := cmd.Run()
|
||||
return stdout.Bytes(), stderr.Bytes(), err
|
||||
}
|
||||
@@ -0,0 +1,238 @@
|
||||
// Package capture wraps the audiotee subprocess and turns its two audio
|
||||
// outputs (system audio on stdout, microphone on a FIFO) into a single
|
||||
// stream of tagged chunks that downstream VAD/ASR stages can consume.
|
||||
package capture
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Track identifies which audio source a Chunk came from.
|
||||
type Track string
|
||||
|
||||
const (
|
||||
TrackSystem Track = "system"
|
||||
TrackMic Track = "mic"
|
||||
)
|
||||
|
||||
// Chunk is a slice of raw PCM audio (16-bit signed, little-endian, mono, at
|
||||
// Config.SampleRate) read off one of audiotee's two outputs. Timestamp is
|
||||
// when this process read the chunk, not when audiotee captured it — audiotee
|
||||
// does not emit per-chunk timestamps (see transcriptor-ai/CLAUDE.md), so this
|
||||
// is an approximation good enough for live display, not sample-accurate
|
||||
// realignment.
|
||||
type Chunk struct {
|
||||
Track Track
|
||||
Data []byte
|
||||
Timestamp time.Time
|
||||
}
|
||||
|
||||
// Config controls how the audiotee subprocess is launched.
|
||||
type Config struct {
|
||||
// AudioteePath is the path to the audiotee binary. If empty, it's
|
||||
// resolved via PATH and then $HOME/bin/audiotee.
|
||||
AudioteePath string
|
||||
// SampleRate is passed to audiotee's --sample-rate flag.
|
||||
SampleRate int
|
||||
// ChunkBytes is the read buffer size used per Chunk. Must be even
|
||||
// (16-bit samples). Defaults to 3200 bytes (~100ms at 16kHz mono).
|
||||
ChunkBytes int
|
||||
// CaptureMic controls whether audiotee is asked to also capture the
|
||||
// mic track (--capture-mic --mic-output). audiotee has no equivalent
|
||||
// flag to skip the system tap — it's always captured — so there is no
|
||||
// "mic only" at this layer; a caller that only wants mic output must
|
||||
// still receive (and can simply ignore) TrackSystem chunks. See
|
||||
// CLAUDE.md for why this is a known, accepted limitation rather than
|
||||
// something fixed here.
|
||||
CaptureMic bool
|
||||
}
|
||||
|
||||
// Capturer runs audiotee and streams both its tracks as Chunks.
|
||||
type Capturer struct {
|
||||
cfg Config
|
||||
cmd *exec.Cmd
|
||||
fifoPath string
|
||||
micFile *os.File
|
||||
|
||||
chunks chan Chunk
|
||||
errs chan error
|
||||
|
||||
wg sync.WaitGroup
|
||||
done chan struct{} // closed once both track readers have hit EOF
|
||||
}
|
||||
|
||||
// New creates a Capturer. Call Start to launch audiotee.
|
||||
func New(cfg Config) *Capturer {
|
||||
if cfg.ChunkBytes <= 0 {
|
||||
cfg.ChunkBytes = 3200
|
||||
}
|
||||
if cfg.ChunkBytes%2 != 0 {
|
||||
cfg.ChunkBytes++
|
||||
}
|
||||
return &Capturer{
|
||||
cfg: cfg,
|
||||
chunks: make(chan Chunk, 64),
|
||||
errs: make(chan error, 4),
|
||||
}
|
||||
}
|
||||
|
||||
// Chunks returns the channel both tracks' audio is delivered on. Closed once
|
||||
// both tracks have stopped delivering data (audiotee exited or Stop was
|
||||
// called).
|
||||
func (c *Capturer) Chunks() <-chan Chunk { return c.chunks }
|
||||
|
||||
// Errs returns non-fatal errors encountered while reading either track.
|
||||
func (c *Capturer) Errs() <-chan error { return c.errs }
|
||||
|
||||
// Start resolves the audiotee binary, launches it with system+mic capture
|
||||
// enabled, and begins streaming both tracks. It returns once audiotee has
|
||||
// been launched; reading happens in background goroutines.
|
||||
func (c *Capturer) Start(ctx context.Context) error {
|
||||
audioteePath, err := resolveAudioteePath(c.cfg.AudioteePath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
args := []string{"--sample-rate", fmt.Sprintf("%d", c.cfg.SampleRate)}
|
||||
|
||||
var fifoPath string
|
||||
if c.cfg.CaptureMic {
|
||||
fifoPath = filepath.Join(os.TempDir(), fmt.Sprintf("transcriptor-ai-mic-%d.fifo", os.Getpid()))
|
||||
if err := syscall.Mkfifo(fifoPath, 0o600); err != nil {
|
||||
return fmt.Errorf("create mic fifo: %w", err)
|
||||
}
|
||||
c.fifoPath = fifoPath
|
||||
args = append(args, "--capture-mic", "--mic-output", fifoPath)
|
||||
}
|
||||
|
||||
cmd := exec.CommandContext(ctx, audioteePath, args...)
|
||||
cmd.Stderr = os.Stderr // audiotee's JSON logs, useful as-is while iterating
|
||||
|
||||
stdout, err := cmd.StdoutPipe()
|
||||
if err != nil {
|
||||
c.cleanupFifo()
|
||||
return fmt.Errorf("attach stdout pipe: %w", err)
|
||||
}
|
||||
|
||||
// Open the FIFO for reading in the background before starting audiotee:
|
||||
// opening a FIFO blocks until the other end is opened too, so this must
|
||||
// not block Start() itself, and audiotee (the writer) hasn't been
|
||||
// spawned yet at this point.
|
||||
var micOpened chan struct{}
|
||||
var micFile *os.File
|
||||
var micOpenErr error
|
||||
if c.cfg.CaptureMic {
|
||||
micOpened = make(chan struct{})
|
||||
go func() {
|
||||
defer close(micOpened)
|
||||
micFile, micOpenErr = os.OpenFile(fifoPath, os.O_RDONLY, 0)
|
||||
}()
|
||||
}
|
||||
|
||||
if err := cmd.Start(); err != nil {
|
||||
c.cleanupFifo()
|
||||
return fmt.Errorf("start audiotee: %w", err)
|
||||
}
|
||||
c.cmd = cmd
|
||||
|
||||
if c.cfg.CaptureMic {
|
||||
<-micOpened
|
||||
if micOpenErr != nil {
|
||||
return fmt.Errorf("open mic fifo: %w", micOpenErr)
|
||||
}
|
||||
c.micFile = micFile
|
||||
}
|
||||
|
||||
c.done = make(chan struct{})
|
||||
c.wg.Add(1)
|
||||
go c.readTrack(TrackSystem, stdout)
|
||||
|
||||
if c.cfg.CaptureMic {
|
||||
c.wg.Add(1)
|
||||
go c.readTrack(TrackMic, micFile)
|
||||
}
|
||||
|
||||
go func() {
|
||||
c.wg.Wait()
|
||||
close(c.chunks)
|
||||
close(c.done)
|
||||
}()
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Capturer) readTrack(track Track, r io.Reader) {
|
||||
defer c.wg.Done()
|
||||
buf := make([]byte, c.cfg.ChunkBytes)
|
||||
for {
|
||||
n, err := io.ReadFull(r, buf)
|
||||
if n > 0 {
|
||||
data := make([]byte, n)
|
||||
copy(data, buf[:n])
|
||||
c.chunks <- Chunk{Track: track, Data: data, Timestamp: time.Now()}
|
||||
}
|
||||
if err != nil {
|
||||
if err != io.EOF && err != io.ErrUnexpectedEOF {
|
||||
select {
|
||||
case c.errs <- fmt.Errorf("%s track: %w", track, err):
|
||||
default:
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Stop terminates audiotee gracefully (SIGINT, matching the fix in audiotee
|
||||
// itself so shutdown doesn't hang) and cleans up the FIFO.
|
||||
func (c *Capturer) Stop() error {
|
||||
if c.cmd != nil && c.cmd.Process != nil {
|
||||
_ = c.cmd.Process.Signal(syscall.SIGINT)
|
||||
}
|
||||
if c.done != nil {
|
||||
// audiotee exiting closes its stdout pipe and the mic FIFO write
|
||||
// end, which is what makes readTrack see EOF — wait for that before
|
||||
// reaping the process. Calling cmd.Wait() first can close the pipe
|
||||
// out from under an in-progress read (see os/exec's StdoutPipe doc).
|
||||
<-c.done
|
||||
}
|
||||
if c.cmd != nil && c.cmd.Process != nil {
|
||||
_ = c.cmd.Wait()
|
||||
}
|
||||
if c.micFile != nil {
|
||||
_ = c.micFile.Close()
|
||||
}
|
||||
c.cleanupFifo()
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Capturer) cleanupFifo() {
|
||||
if c.fifoPath != "" {
|
||||
_ = os.Remove(c.fifoPath)
|
||||
}
|
||||
}
|
||||
|
||||
func resolveAudioteePath(configured string) (string, error) {
|
||||
if configured != "" {
|
||||
return configured, nil
|
||||
}
|
||||
if p, err := exec.LookPath("audiotee"); err == nil {
|
||||
return p, nil
|
||||
}
|
||||
home, err := os.UserHomeDir()
|
||||
if err == nil {
|
||||
candidate := filepath.Join(home, "bin", "audiotee")
|
||||
if info, statErr := os.Stat(candidate); statErr == nil && !info.IsDir() {
|
||||
return candidate, nil
|
||||
}
|
||||
}
|
||||
return "", fmt.Errorf("audiotee binary not found on PATH or in ~/bin — build it via audiotee/scripts/build-signed.sh")
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
// Package transcript defines the live transcript message format (see
|
||||
// CLAUDE.md) and a Broadcaster that fans a single stream of messages out to
|
||||
// any number of SSE clients — the merge stage's output contract.
|
||||
//
|
||||
// Ordering note: messages are broadcast in the order transcribeTrack
|
||||
// produces them, not re-sorted by Timestamp. With two tracks processed
|
||||
// independently, a slower track could in principle publish slightly out of
|
||||
// chronological order relative to the other. Measured ASR latency
|
||||
// (~0.15-0.2x realtime) is small relative to segment spacing (3-5s), so
|
||||
// this hasn't been observed in practice — add a small reordering buffer
|
||||
// here if it ever becomes visible, not before.
|
||||
package transcript
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/stephanetailland/transcriptor-ai/internal/capture"
|
||||
)
|
||||
|
||||
// Message is one transcript segment, matching the SSE JSON format
|
||||
// documented in CLAUDE.md.
|
||||
type Message struct {
|
||||
Track capture.Track `json:"track"`
|
||||
Text string `json:"text"`
|
||||
IsFinal bool `json:"is_final"`
|
||||
Timestamp time.Time `json:"timestamp"`
|
||||
}
|
||||
|
||||
// clientBuffer bounds how many undelivered messages a slow SSE client can
|
||||
// accumulate before new messages are dropped for it, so one stalled client
|
||||
// can't back-pressure the whole pipeline.
|
||||
const clientBuffer = 32
|
||||
|
||||
// Broadcaster fans Published messages out to every currently-connected SSE
|
||||
// client and doubles as the http.Handler for the SSE endpoint.
|
||||
type Broadcaster struct {
|
||||
mu sync.Mutex
|
||||
clients map[chan Message]struct{}
|
||||
}
|
||||
|
||||
func NewBroadcaster() *Broadcaster {
|
||||
return &Broadcaster{clients: make(map[chan Message]struct{})}
|
||||
}
|
||||
|
||||
// Publish sends msg to every connected client. Non-blocking: a client whose
|
||||
// buffer is full misses the message rather than stalling the sender.
|
||||
func (b *Broadcaster) Publish(msg Message) {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
for ch := range b.clients {
|
||||
select {
|
||||
case ch <- msg:
|
||||
default:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (b *Broadcaster) subscribe() chan Message {
|
||||
ch := make(chan Message, clientBuffer)
|
||||
b.mu.Lock()
|
||||
b.clients[ch] = struct{}{}
|
||||
b.mu.Unlock()
|
||||
return ch
|
||||
}
|
||||
|
||||
func (b *Broadcaster) unsubscribe(ch chan Message) {
|
||||
b.mu.Lock()
|
||||
delete(b.clients, ch)
|
||||
b.mu.Unlock()
|
||||
}
|
||||
|
||||
// ServeHTTP streams messages to the client as Server-Sent Events until the
|
||||
// client disconnects. See CLAUDE.md for why SSE over WebSocket: the flow is
|
||||
// one-directional, and this needs no client library beyond EventSource in a
|
||||
// browser or a plain HTTP client in a terminal (e.g. `curl -N`).
|
||||
func (b *Broadcaster) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
flusher, ok := w.(http.Flusher)
|
||||
if !ok {
|
||||
http.Error(w, "streaming unsupported", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
w.Header().Set("Cache-Control", "no-cache")
|
||||
w.Header().Set("Connection", "keep-alive")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
flusher.Flush()
|
||||
|
||||
ch := b.subscribe()
|
||||
defer b.unsubscribe(ch)
|
||||
|
||||
for {
|
||||
select {
|
||||
case msg, ok := <-ch:
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
data, err := json.Marshal(msg)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
fmt.Fprintf(w, "data: %s\n\n", data)
|
||||
flusher.Flush()
|
||||
case <-r.Context().Done():
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
package transcript
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// FileWriter appends each Message to a text file as a human-readable line,
|
||||
// flushed to disk immediately. Unlike Broadcaster.Publish (which drops
|
||||
// messages for a slow/stalled SSE client on purpose), FileWriter never
|
||||
// drops — this is the durable record a crash shouldn't lose data from, so
|
||||
// every Write is synchronous and fsync'd before returning.
|
||||
type FileWriter struct {
|
||||
mu sync.Mutex
|
||||
f *os.File
|
||||
}
|
||||
|
||||
// NewFileWriter opens path for appending, creating it if needed. Appending
|
||||
// (not truncating) means restarting transcriptor-ai against the same path
|
||||
// continues the same transcript file rather than discarding it.
|
||||
func NewFileWriter(path string) (*FileWriter, error) {
|
||||
f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o644)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open transcript file: %w", err)
|
||||
}
|
||||
return &FileWriter{f: f}, nil
|
||||
}
|
||||
|
||||
// Write appends one line for msg and fsyncs before returning.
|
||||
func (fw *FileWriter) Write(msg Message) error {
|
||||
fw.mu.Lock()
|
||||
defer fw.mu.Unlock()
|
||||
line := fmt.Sprintf("[%s] [%s] %s\n", msg.Timestamp.Format("15:04:05"), msg.Track, msg.Text)
|
||||
if _, err := io.WriteString(fw.f, line); err != nil {
|
||||
return fmt.Errorf("write transcript line: %w", err)
|
||||
}
|
||||
if err := fw.f.Sync(); err != nil {
|
||||
return fmt.Errorf("sync transcript file: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (fw *FileWriter) Close() error {
|
||||
fw.mu.Lock()
|
||||
defer fw.mu.Unlock()
|
||||
return fw.f.Close()
|
||||
}
|
||||
@@ -0,0 +1,217 @@
|
||||
// Package vad turns a stream of raw audio chunks into bounded speech
|
||||
// segments using a simple energy-threshold heuristic — no ML model. See
|
||||
// transcriptor-ai/CLAUDE.md for why: no Go/Swift ML ecosystem was mature
|
||||
// enough to justify the integration cost for this. Revisit only if
|
||||
// segmentation quality proves to be the actual bottleneck.
|
||||
package vad
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"math"
|
||||
"time"
|
||||
|
||||
"github.com/stephanetailland/transcriptor-ai/internal/capture"
|
||||
)
|
||||
|
||||
// Segment is a bounded span of speech audio ready for ASR.
|
||||
type Segment struct {
|
||||
Track capture.Track
|
||||
Data []byte // raw PCM, concatenation of the contributing chunks
|
||||
Start time.Time
|
||||
End time.Time
|
||||
// IsFinal is true when silence closed the utterance, false when the
|
||||
// segment was force-cut at MaxSegmentDuration and more audio for the
|
||||
// same utterance follows in the next segment (which starts with
|
||||
// Config.Overlap worth of audio repeated for ASR context continuity).
|
||||
IsFinal bool
|
||||
}
|
||||
|
||||
// Config controls the energy-threshold heuristic. All durations are
|
||||
// expressed in wall-clock time; SampleRate converts them to byte counts.
|
||||
type Config struct {
|
||||
SampleRate int
|
||||
|
||||
// SilenceThresholdDB is the RMS level (dBFS, 0 = full scale) below which
|
||||
// a chunk is considered silence. Real speech in testing sits around -40
|
||||
// to -20 dBFS; captured silence sits around -90 dBFS. Needs empirical
|
||||
// tuning against real meeting audio — this default is a starting point,
|
||||
// not a validated value.
|
||||
SilenceThresholdDB float64
|
||||
|
||||
// SilenceHangover is how long silence must persist after speech before
|
||||
// the segment is closed as final (IsFinal: true).
|
||||
SilenceHangover time.Duration
|
||||
|
||||
// MinSpeechDuration is the minimum contiguous voiced audio required
|
||||
// before a segment starts accumulating, to filter transient noise
|
||||
// (clicks, pops) rather than opening a segment on every blip.
|
||||
MinSpeechDuration time.Duration
|
||||
|
||||
// MaxSegmentDuration force-cuts a segment even mid-speech, so a long
|
||||
// monologue still produces incremental, timely output instead of one
|
||||
// unbounded segment.
|
||||
MaxSegmentDuration time.Duration
|
||||
|
||||
// Overlap is how much trailing audio is carried into the next segment
|
||||
// when force-cut, for ASR context continuity across the cut. Tradeoff:
|
||||
// the carried-over audio gets transcribed again, and both copies are
|
||||
// published — a nonzero Overlap produces visible duplicate text at
|
||||
// every force-cut boundary (confirmed in real testing: this is worse
|
||||
// than the word occasionally getting cut in half at a hard boundary,
|
||||
// which is why DefaultConfig uses 0). Only raise this if paired with
|
||||
// dedup logic on the published text — never on its own.
|
||||
Overlap time.Duration
|
||||
}
|
||||
|
||||
// DefaultConfig returns starting-point thresholds; tune against real audio.
|
||||
func DefaultConfig(sampleRate int) Config {
|
||||
return Config{
|
||||
SampleRate: sampleRate,
|
||||
SilenceThresholdDB: -40,
|
||||
SilenceHangover: 600 * time.Millisecond,
|
||||
MinSpeechDuration: 200 * time.Millisecond,
|
||||
MaxSegmentDuration: 4 * time.Second,
|
||||
Overlap: 0,
|
||||
}
|
||||
}
|
||||
|
||||
func (c Config) bytesForDuration(d time.Duration) int {
|
||||
// 16-bit mono samples: 2 bytes/sample.
|
||||
samples := int(float64(c.SampleRate) * d.Seconds())
|
||||
n := samples * 2
|
||||
if n%2 != 0 {
|
||||
n++
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// Detector runs the heuristic independently per track (system and mic have
|
||||
// separate state) over a shared, tagged chunk stream.
|
||||
type Detector struct {
|
||||
cfg Config
|
||||
states map[capture.Track]*trackState
|
||||
}
|
||||
|
||||
func New(cfg Config) *Detector {
|
||||
return &Detector{cfg: cfg, states: make(map[capture.Track]*trackState)}
|
||||
}
|
||||
|
||||
type trackState struct {
|
||||
inSpeech bool
|
||||
buffer []byte
|
||||
segStart time.Time
|
||||
|
||||
// candidate holds voiced audio not yet promoted to a real segment,
|
||||
// while we wait to confirm MinSpeechDuration.
|
||||
candidate []byte
|
||||
candidateStart time.Time
|
||||
|
||||
silenceSince time.Time // zero value means "not currently in a silence run"
|
||||
lastChunkAt time.Time // timestamp of the most recently processed chunk
|
||||
}
|
||||
|
||||
// Run consumes chunks until the channel closes, emitting Segments on the
|
||||
// returned channel. Any in-progress segment is flushed (as final, since
|
||||
// capture ended) before the output channel closes.
|
||||
func (d *Detector) Run(chunks <-chan capture.Chunk) <-chan Segment {
|
||||
out := make(chan Segment, 16)
|
||||
go func() {
|
||||
defer close(out)
|
||||
for chunk := range chunks {
|
||||
d.process(chunk, out)
|
||||
}
|
||||
for track, st := range d.states {
|
||||
if st.inSpeech && len(st.buffer) > 0 {
|
||||
out <- Segment{Track: track, Data: st.buffer, Start: st.segStart, End: st.lastChunkAt, IsFinal: true}
|
||||
}
|
||||
}
|
||||
}()
|
||||
return out
|
||||
}
|
||||
|
||||
func (d *Detector) process(chunk capture.Chunk, out chan<- Segment) {
|
||||
st, ok := d.states[chunk.Track]
|
||||
if !ok {
|
||||
st = &trackState{}
|
||||
d.states[chunk.Track] = st
|
||||
}
|
||||
|
||||
speech := dbFS(chunk.Data) >= d.cfg.SilenceThresholdDB
|
||||
st.lastChunkAt = chunk.Timestamp
|
||||
|
||||
switch {
|
||||
case speech && !st.inSpeech:
|
||||
// Possible start of an utterance — buffer as a candidate until
|
||||
// MinSpeechDuration is confirmed, to filter transient noise.
|
||||
if len(st.candidate) == 0 {
|
||||
st.candidateStart = chunk.Timestamp
|
||||
}
|
||||
st.candidate = append(st.candidate, chunk.Data...)
|
||||
if chunk.Timestamp.Sub(st.candidateStart) >= d.cfg.MinSpeechDuration {
|
||||
st.inSpeech = true
|
||||
st.segStart = st.candidateStart
|
||||
st.buffer = st.candidate
|
||||
st.candidate = nil
|
||||
st.silenceSince = time.Time{}
|
||||
}
|
||||
|
||||
case speech && st.inSpeech:
|
||||
st.buffer = append(st.buffer, chunk.Data...)
|
||||
st.silenceSince = time.Time{} // speech resumed, cancel any pending hangover
|
||||
if chunk.Timestamp.Sub(st.segStart) >= d.cfg.MaxSegmentDuration {
|
||||
out <- Segment{Track: chunk.Track, Data: st.buffer, Start: st.segStart, End: chunk.Timestamp, IsFinal: false}
|
||||
st.buffer = tail(st.buffer, d.cfg.bytesForDuration(d.cfg.Overlap))
|
||||
st.segStart = chunk.Timestamp.Add(-d.cfg.Overlap)
|
||||
}
|
||||
|
||||
case !speech && st.inSpeech:
|
||||
if st.silenceSince.IsZero() {
|
||||
st.silenceSince = chunk.Timestamp
|
||||
}
|
||||
st.buffer = append(st.buffer, chunk.Data...) // keep brief trailing silence for natural cutoff
|
||||
if chunk.Timestamp.Sub(st.silenceSince) >= d.cfg.SilenceHangover {
|
||||
out <- Segment{Track: chunk.Track, Data: st.buffer, Start: st.segStart, End: chunk.Timestamp, IsFinal: true}
|
||||
st.inSpeech = false
|
||||
st.buffer = nil
|
||||
st.silenceSince = time.Time{}
|
||||
}
|
||||
|
||||
default: // !speech && !st.inSpeech
|
||||
st.candidate = nil
|
||||
}
|
||||
}
|
||||
|
||||
// dbFS returns the RMS level of a 16-bit little-endian PCM buffer in dBFS
|
||||
// (0 = full scale for int16). Silence (all-zero input) returns -infinity,
|
||||
// which compares correctly against any real threshold.
|
||||
func dbFS(data []byte) float64 {
|
||||
n := len(data) / 2
|
||||
if n == 0 {
|
||||
return math.Inf(-1)
|
||||
}
|
||||
var sumSquares float64
|
||||
for i := 0; i+1 < len(data); i += 2 {
|
||||
s := int16(binary.LittleEndian.Uint16(data[i : i+2]))
|
||||
v := float64(s)
|
||||
sumSquares += v * v
|
||||
}
|
||||
rms := math.Sqrt(sumSquares / float64(n))
|
||||
if rms == 0 {
|
||||
return math.Inf(-1)
|
||||
}
|
||||
return 20 * math.Log10(rms/32768)
|
||||
}
|
||||
|
||||
func tail(data []byte, n int) []byte {
|
||||
if n <= 0 {
|
||||
return nil
|
||||
}
|
||||
if n >= len(data) {
|
||||
out := make([]byte, len(data))
|
||||
copy(out, data)
|
||||
return out
|
||||
}
|
||||
out := make([]byte, n)
|
||||
copy(out, data[len(data)-n:])
|
||||
return out
|
||||
}
|
||||
Executable
+31
@@ -0,0 +1,31 @@
|
||||
#!/bin/bash
|
||||
# Builds transcriptor-ai and its qwen_asr dependency into dist/, ready to be
|
||||
# embedded by transcriptor-ui's Xcode build phase. Models are NOT part of
|
||||
# this — those are fetched at runtime by the app (see CLAUDE.md), only
|
||||
# binaries are built/staged here.
|
||||
#
|
||||
# Usage: scripts/build-dist.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$REPO_ROOT"
|
||||
|
||||
DIST_DIR="$REPO_ROOT/dist"
|
||||
QWEN_ASR_DIR="$REPO_ROOT/third_party/qwen-asr"
|
||||
|
||||
echo "Ensuring qwen-asr submodule is present (pinned commit)..."
|
||||
git submodule update --init --recursive third_party/qwen-asr
|
||||
|
||||
echo "Building qwen_asr..."
|
||||
make -C "$QWEN_ASR_DIR" blas
|
||||
|
||||
echo "Building transcriptor-ai..."
|
||||
mkdir -p "$DIST_DIR"
|
||||
go build -o "$DIST_DIR/transcriptor-ai" ./cmd/transcriptor-ai
|
||||
|
||||
cp "$QWEN_ASR_DIR/qwen_asr" "$DIST_DIR/qwen_asr"
|
||||
|
||||
echo ""
|
||||
echo "dist/ ready:"
|
||||
ls -la "$DIST_DIR"
|
||||
+1
Submodule third_party/qwen-asr added at b00b789b17
Reference in New Issue
Block a user