add real-time toggle and batch (post-hoc) transcription

Two capture modes: the existing live SSE transcription, and a new
"Transcribe a recording..." flow that runs transcriptor-ai's new
transcribe subcommand against a previously recorded session's PCM
files. The raw PCM recording (previously an undocumented temp-dir
debug tee) is now a real, user-configurable artifact via a new
"Save recordings to" Settings row, since batch transcription depends
on it rather than just "listening back later".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
sttlab-tech
2026-09-18 16:45:16 +02:00
parent 49839b8532
commit b3948e7ca9
5 changed files with 394 additions and 8 deletions
+84
View File
@@ -499,6 +499,90 @@ succeeded."
Not done yet — everything under "Future" in Goal (question-detection/CR-generation
post-processing) — still explicitly out of scope until asked for.
## Real-time toggle + batch (post-hoc) transcription (2026-09-18) — implemented
Motivated by a discussion about splitting the architecture (UI on this MacBook, AI on a local
Ubuntu server): live streaming turned out to bring real complexity (network audio protocol,
buffering/reconnect, no-disk-storage constraints), while **batch transcription after the
meeting sidesteps nearly all of it** — and it turned out the raw audio was already being
recorded, just as an undocumented debug tee (see below). The network/server split itself is
still **not implemented** — this is the local groundwork (two capture modes) that a future
server-side batch pass could build on, not the split itself.
**Found first**: `transcriptor-ai` already wrote every session's raw audio to
`os.TempDir()/transcriptor-ai-{system,mic}.pcm`, unconditionally, undocumented in this repo's
own CLAUDE.md — a debug tee ("for listening back later"), not a designed feature: fixed
filenames (a second session silently overwrote the first), no configurable location, no
cleanup. This session promoted it into a real, deliberate artifact — the user's own framing:
"les PCM sont déjà stockés" — rather than building batch transcription on top of an accidental
side effect.
**`transcriptor-ai` changes** (`cmd/transcriptor-ai/main.go`, new `cmd/transcriptor-ai/transcribe.go`):
- `-recording-dir` (optional, defaults to the OS temp dir if empty) + session-start-timestamped
filenames (`<yyyy-MM-dd_HH-mm-ss>-system.pcm` / `-mic.pcm`, matching the transcript file's
existing timestamp convention) replace the old fixed names — no more silent overwrite between
sessions. The mic file is only created when `-capture-mic` is actually on, so a disabled track
never leaves behind a misleading empty file.
- `-realtime` (default `true`). `false` skips VAD/ASR/SSE entirely (no HTTP server started) and
only captures+records — capture itself, and the PCM tee, run unconditionally either way.
Implemented with nil-channel `select` cases (`segments`/`httpServer` left nil when disabled)
rather than duplicating the run loop — standard Go idiom, kept the diff to `run()` itself
fairly contained rather than forking into two code paths.
- New `transcribe` subcommand (`transcriptor-ai transcribe -system-pcm=... -mic-pcm=...
-asr-binary=... -asr-model-dir=... -transcript-file=...`, at least one of
`-system-pcm`/`-mic-pcm` required): replays a previously recorded PCM file as a stream of
`capture.Chunk`s (synthetic timestamps advanced by bytes-read/audio-duration, not wall clock,
since a file is read as fast as disk I/O allows — see `bytesToDuration`/`readPCMFile`) through
the **same, unmodified** `internal/vad`/`internal/asr` pipeline live mode uses, then
`transcribeWorker` (also unmodified/reused) writes the result to `-transcript-file`. No new
segmentation or ASR logic — the batch mode is entirely a different *source* for
`capture.Chunk`, not a different pipeline. No HTTP server in this mode; a `Broadcaster` is
still constructed (Publish-to-zero-subscribers is a harmless no-op) purely to satisfy
`transcribeWorker`'s existing signature without special-casing a nil broadcaster there.
- **Verified for real, not just "it compiles"**: built a standalone binary, synthesized a PCM
file (silence/tone/silence/tone/silence, tones above VAD's speech threshold) with a throwaway
Python script, and a stub `qwen_asr` shell script that just echoes fixed text reading stdin.
Ran `transcriptor-ai transcribe` against them — got exactly 2 correctly VAD-segmented,
transcribed, file-written lines (matching the 2 tone bursts), clean exit 0. Confirms the
concurrency (two-goroutine file readers → shared chunk channel → VAD → sequential ASR → file)
doesn't deadlock or panic. **Not yet verified against a real recorded session + real
qwen_asr/model** — do that before trusting batch transcription quality, this only proves the
plumbing.
**`transcriptor-ui` (Swift) changes**:
- `TranscriptorProcess.Config` gained `recordingDir` (required, passed as `-recording-dir`) and
`realtime` (passed as `-realtime=\(value)`, same boolean-flag `-flag=value` requirement
documented above for `-capture-system`/`-capture-mic`).
- `ContentView`: new "Real-time transcription" `Toggle`, `@State`-backed (not `@AppStorage`) —
deliberately matching `captureSystem`/`captureMic`'s existing treatment, not the
`@AppStorage`-persisted settings-page values; revisit together if the user ever wants the
main-window toggles remembered across relaunches. When off, `ContentView.start()` skips
building a live transcript file path entirely and never connects `SSEClient` (nothing would
ever arrive on it).
- New `BatchTranscriber.swift` — a separate, one-shot `Process` runner (`@Observable`, async
`run(_:)` that throws on nonzero exit) distinct from `TranscriptorProcess`: no SIGINT-based
graceful-drain protocol to replicate, since batch mode isn't an interactive session.
- New "Transcribe a recording…" button in `ContentView`: `NSOpenPanel` (filtered to `.pcm`,
starting in the recordings directory) lets the user pick *either* file of a session — the
sibling track is located automatically from the shared `<timestamp>-system.pcm` /
`<timestamp>-mic.pcm` naming convention (`ContentView.recordingSession(from:)`). **Scope
deliberately restricted to this app's own recordings, not arbitrary audio files** — an
explicit choice this session ("pour le moment, juste pcm"), not an oversight; supporting
arbitrary formats would need an `ffmpeg`-style decode step this doesn't have. Output transcript
is named `<session-timestamp>-batch.txt` — suffixed distinctly from the live transcript's own
`<timestamp>.txt` so a batch pass can never collide with or silently append into a live
transcript from the same session.
- `SettingsView`: new "Save recordings to" row (`RecordingSaveLocation`, default
`~/Documents/Transcriptor/Recordings`, `@AppStorage`-persisted, `NSOpenPanel` "Choose…" —
same pattern as the existing transcript-directory row), promoting the recording location from
an invisible temp-dir default to a real, user-visible, user-configurable setting.
- **Verification status**: `xcodebuild` succeeds (embedding the freshly rebuilt
`transcriptor-ai` with all the above), and the built app launches and quits cleanly with no
crash. **Not yet verified interactively** — the new toggle/button haven't been clicked, and
batch transcription hasn't been run end to end from the UI against a real recorded session.
Do that (start a short real session, stop it, then use "Transcribe a recording…" on the
resulting `.pcm` pair) before considering this feature done, not just built.
## Installing to /Applications (not automated — do this manually when needed)
There's no install script; this was done by hand once (2026-08-08) and documented here so it
+95
View File
@@ -0,0 +1,95 @@
//
// BatchTranscriber.swift
// transcriptor
//
// Runs `transcriptor-ai transcribe` — post-hoc transcription of a
// previously recorded PCM session (see TranscriptorProcess's
// -recording-dir, which is what produces the .pcm files this reads) — as
// a one-shot subprocess, distinct from TranscriptorProcess (which spawns
// the long-lived live-capture process). No SIGINT-based graceful shutdown
// protocol needed here: this isn't an interactive session, just a
// synchronous batch job that runs to completion or is killed outright.
//
import Foundation
enum BatchTranscriberError: Error {
case alreadyRunning
case binaryNotFound(String)
case processFailed(Int32)
}
@Observable
final class BatchTranscriber {
private(set) var isRunning = false
private var process: Process?
struct Config {
var binaryPath: String
var systemPCMPath: String?
var micPCMPath: String?
var asrBinaryPath: String
var asrModelDir: String
var prompt: String?
var transcriptFilePath: String
}
/// Runs the batch transcription to completion. Throws on a nonzero
/// exit status; stderr (transcriptor-ai's own progress lines, plus
/// qwen_asr's) is forwarded to Console output the same way
/// TranscriptorProcess does for the live process.
func run(_ config: Config) async throws {
guard process == nil else { throw BatchTranscriberError.alreadyRunning }
for path in [config.binaryPath, config.asrBinaryPath] {
guard FileManager.default.fileExists(atPath: path) else {
throw BatchTranscriberError.binaryNotFound(path)
}
}
let process = Process()
process.executableURL = URL(fileURLWithPath: config.binaryPath)
var args = [
"transcribe",
"-asr-binary", config.asrBinaryPath,
"-asr-model-dir", config.asrModelDir,
"-transcript-file", config.transcriptFilePath,
]
if let systemPCMPath = config.systemPCMPath {
args += ["-system-pcm", systemPCMPath]
}
if let micPCMPath = config.micPCMPath {
args += ["-mic-pcm", micPCMPath]
}
if let prompt = config.prompt, !prompt.isEmpty {
args += ["-prompt", prompt]
}
process.arguments = args
process.standardError = FileHandle.standardError
self.process = process
isRunning = true
defer {
self.process = nil
isRunning = false
}
try process.run()
try await withCheckedThrowingContinuation { (continuation: CheckedContinuation<Void, Error>) in
process.terminationHandler = { p in
if p.terminationStatus == 0 {
continuation.resume()
} else {
continuation.resume(throwing: BatchTranscriberError.processFailed(p.terminationStatus))
}
}
}
}
/// Interrupts an in-progress batch job (e.g. the user closing the
/// progress sheet mid-run). No graceful-drain protocol on the other
/// end like TranscriptorProcess.stop() relies on — this just kills it.
func cancel() {
process?.interrupt()
}
}
+156 -3
View File
@@ -10,7 +10,9 @@
// oversight, see CLAUDE.md "Goal".
//
import AppKit
import SwiftUI
import UniformTypeIdentifiers
private enum BundledPaths {
static var audioteeBinary: String {
@@ -47,13 +49,23 @@ struct ContentView: View {
// mid-session and produced a confusing, uncontrolled test run.
@AppStorage("captureSystem") private var captureSystem = true
@AppStorage("captureMic") private var captureMic = true
// @State, not @AppStorage: same treatment as captureSystem/captureMic
// before their persistence fix wasn't asked for — revisit if the user
// wants this remembered across relaunches too.
@State private var realtimeEnabled = true
@AppStorage(TranscriptSaveLocation.defaultsKey)
private var transcriptSaveDirectory = TranscriptSaveLocation.defaultDirectory.path
@AppStorage(RecordingSaveLocation.defaultsKey)
private var recordingSaveDirectory = RecordingSaveLocation.defaultDirectory.path
@AppStorage(GlossaryFileLocation.defaultsKey)
private var glossaryFilePath = GlossaryFileLocation.defaultPath.path
@State private var batchTranscriber = BatchTranscriber()
@State private var batchStatus: String?
@State private var batchError: String?
private var controlsDisabled: Bool {
transcriptorProcess.isRunning || modelManager.isDownloading
transcriptorProcess.isRunning || modelManager.isDownloading || batchTranscriber.isRunning
}
var body: some View {
@@ -74,6 +86,10 @@ struct ContentView: View {
.disabled(controlsDisabled)
Toggle("Microphone", isOn: $captureMic)
.disabled(controlsDisabled)
Divider().frame(height: 16)
Toggle("Real-time transcription", isOn: $realtimeEnabled)
.disabled(controlsDisabled)
}
HStack {
@@ -84,7 +100,12 @@ struct ContentView: View {
Task { await start() }
}
}
.disabled(modelManager.isDownloading || (!captureSystem && !captureMic))
.disabled(modelManager.isDownloading || batchTranscriber.isRunning || (!captureSystem && !captureMic))
Button("Transcribe a recording…") {
chooseBatchRecording()
}
.disabled(controlsDisabled)
Text(statusText)
.foregroundStyle(.secondary)
@@ -108,6 +129,29 @@ struct ContentView: View {
.padding([.horizontal, .bottom])
}
if batchTranscriber.isRunning || batchStatus != nil || batchError != nil {
VStack(alignment: .leading, spacing: 4) {
if batchTranscriber.isRunning {
HStack(spacing: 8) {
ProgressView().controlSize(.small)
Text("Transcribing recording… (see Console for progress)")
}
.font(.caption)
.foregroundStyle(.secondary)
} else if let batchStatus {
Text(batchStatus)
.font(.caption)
.foregroundStyle(.secondary)
}
if let batchError {
Text(batchError)
.font(.caption)
.foregroundStyle(.red)
}
}
.padding([.horizontal, .bottom])
}
Divider()
ScrollViewReader { proxy in
@@ -154,13 +198,21 @@ struct ContentView: View {
return
}
// No live transcript file (and no point connecting the SSE client
// below) when real-time transcription is off — transcriptor-ai
// skips VAD/ASR/SSE entirely in that mode and only records, see
// -realtime in TranscriptorProcess.
let transcriptFilePath: String?
if realtimeEnabled {
do {
transcriptFilePath = try makeTranscriptFilePath(startedAt: Date())
} catch {
errorMessage = "Couldn't prepare transcript save location: \(error)"
return
}
} else {
transcriptFilePath = nil
}
let config = TranscriptorProcess.Config(
binaryPath: BundledPaths.transcriptorAiBinary,
@@ -171,10 +223,13 @@ struct ContentView: View {
transcriptFilePath: transcriptFilePath,
captureSystem: captureSystem,
captureMic: captureMic,
glossary: readGlossaryFile()
glossary: readGlossaryFile(),
recordingDir: recordingSaveDirectory,
realtime: realtimeEnabled
)
do {
try transcriptorProcess.start(config)
guard realtimeEnabled else { return }
// transcriptor-ai's HTTP server starts early in its own
// startup, but not instantly — give it a moment before
// connecting rather than building retry logic for this
@@ -232,6 +287,104 @@ struct ContentView: View {
guard !trimmed.isEmpty else { return nil }
return "Preserve spelling: \(trimmed)"
}
/// Lets the user pick either .pcm file of a recorded session (see
/// -recording-dir); the sibling track is located automatically from the
/// shared "<timestamp>-system.pcm" / "<timestamp>-mic.pcm" naming
/// convention transcriptor-ai writes (see readPCMFile/main.go there) —
/// per spec, batch transcription only targets sessions this app itself
/// recorded, not arbitrary audio files, so this naming convention is a
/// safe assumption rather than a format to validate defensively.
private func chooseBatchRecording() {
let panel = NSOpenPanel()
panel.canChooseDirectories = false
panel.canChooseFiles = true
panel.allowsMultipleSelection = false
panel.prompt = "Transcribe"
panel.directoryURL = URL(fileURLWithPath: recordingSaveDirectory)
if let pcmType = UTType(filenameExtension: "pcm") {
panel.allowedContentTypes = [pcmType]
}
guard panel.runModal() == .OK, let url = panel.url else { return }
let session = recordingSession(from: url)
guard session.systemPath != nil || session.micPath != nil else {
batchError = "Couldn't find a matching recording for \(url.lastPathComponent)."
return
}
Task { await runBatchTranscription(session) }
}
private struct RecordingSession {
let stamp: String
let systemPath: String?
let micPath: String?
}
private func recordingSession(from pickedFile: URL) -> RecordingSession {
let directory = pickedFile.deletingLastPathComponent()
let name = pickedFile.deletingPathExtension().lastPathComponent
let stamp: String
if name.hasSuffix("-system") {
stamp = String(name.dropLast("-system".count))
} else if name.hasSuffix("-mic") {
stamp = String(name.dropLast("-mic".count))
} else {
stamp = name
}
let systemCandidate = directory.appendingPathComponent("\(stamp)-system.pcm").path
let micCandidate = directory.appendingPathComponent("\(stamp)-mic.pcm").path
let fm = FileManager.default
return RecordingSession(
stamp: stamp,
systemPath: fm.fileExists(atPath: systemCandidate) ? systemCandidate : nil,
micPath: fm.fileExists(atPath: micCandidate) ? micCandidate : nil
)
}
private func runBatchTranscription(_ session: RecordingSession) async {
batchError = nil
batchStatus = nil
let modelDir: URL
do {
modelDir = try await modelManager.ensureModel(selectedModel)
} catch {
batchError = "Model download failed: \(error)"
return
}
// Suffixed distinctly from the live transcript's own
// "<timestamp>.txt" naming so a batch pass never collides with (or
// silently appends into) a live transcript from the same session.
let transcriptPath: String
do {
let directory = URL(fileURLWithPath: transcriptSaveDirectory)
try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true)
transcriptPath = directory.appendingPathComponent("\(session.stamp)-batch.txt").path
} catch {
batchError = "Couldn't prepare transcript save location: \(error)"
return
}
let config = BatchTranscriber.Config(
binaryPath: BundledPaths.transcriptorAiBinary,
systemPCMPath: session.systemPath,
micPCMPath: session.micPath,
asrBinaryPath: BundledPaths.qwenAsrBinary,
asrModelDir: modelDir.path,
prompt: readGlossaryFile(),
transcriptFilePath: transcriptPath
)
do {
try await batchTranscriber.run(config)
batchStatus = "Transcribed -> \(transcriptPath)"
} catch {
batchError = "Batch transcription failed: \(error)"
}
}
}
#Preview {
+50
View File
@@ -21,6 +21,20 @@ enum TranscriptSaveLocation {
}
}
/// Where transcriptor-ai's raw PCM recording (system/mic .pcm, one pair per
/// session, timestamped) is saved. Promoted from an undocumented debug tee
/// into `os.TempDir()` to a real, user-visible artifact: it's now the input
/// to batch transcription (see BatchTranscriber), not just "for listening
/// back later" — so it needs a stable, discoverable location like the
/// transcript directory, not a temp path that could be cleared any time.
enum RecordingSaveLocation {
static let defaultsKey = "recordingSaveDirectory"
static var defaultDirectory: URL {
TranscriptSaveLocation.defaultDirectory.appendingPathComponent("Recordings", isDirectory: true)
}
}
/// The glossary lives in a plain text file the user edits directly (in
/// whatever app they like), not a text field in this Settings window —
/// simpler for anything longer than a few words, and avoids SwiftUI
@@ -38,6 +52,8 @@ enum GlossaryFileLocation {
struct SettingsView: View {
@AppStorage(TranscriptSaveLocation.defaultsKey)
private var transcriptSaveDirectory = TranscriptSaveLocation.defaultDirectory.path
@AppStorage(RecordingSaveLocation.defaultsKey)
private var recordingSaveDirectory = RecordingSaveLocation.defaultDirectory.path
@AppStorage(GlossaryFileLocation.defaultsKey)
private var glossaryFilePath = GlossaryFileLocation.defaultPath.path
@@ -73,6 +89,27 @@ struct SettingsView: View {
}
}
GridRow {
Text("Save recordings to")
Text(recordingSaveDirectory)
.lineLimit(1)
.truncationMode(.middle)
.foregroundStyle(.secondary)
.frame(minWidth: pathMinWidth, alignment: .leading)
Button("Choose…") {
chooseRecordingDirectory()
}
}
GridRow {
Color.clear.gridCellUnsizedAxes([.horizontal, .vertical])
Text("Raw audio (system/mic) recorded during every session, used as the input for batch transcription (Transcribe a recording…) — separate from the transcript text above.")
.font(.callout)
.foregroundStyle(.secondary)
.fixedSize(horizontal: false, vertical: true)
.gridCellColumns(3)
}
GridRow {
Text("Glossary file")
Text(glossaryFilePath)
@@ -120,6 +157,19 @@ struct SettingsView: View {
}
}
private func chooseRecordingDirectory() {
let panel = NSOpenPanel()
panel.canChooseDirectories = true
panel.canChooseFiles = false
panel.canCreateDirectories = true
panel.prompt = "Choose"
panel.directoryURL = URL(fileURLWithPath: recordingSaveDirectory)
if panel.runModal() == .OK, let url = panel.url {
recordingSaveDirectory = url.path
}
}
/// NSSavePanel, not NSOpenPanel: lets the user pick a location for a
/// file that may not exist yet (a fresh glossary), the same way a
/// "Save As…" dialog would — NSOpenPanel can only select files that
+4
View File
@@ -29,6 +29,8 @@ final class TranscriptorProcess {
var captureSystem = true
var captureMic = true
var glossary: String?
var recordingDir: String
var realtime = true
}
func start(_ config: Config) throws {
@@ -54,6 +56,8 @@ final class TranscriptorProcess {
"-http-addr", config.httpAddr,
"-capture-system=\(config.captureSystem)",
"-capture-mic=\(config.captureMic)",
"-recording-dir", config.recordingDir,
"-realtime=\(config.realtime)",
]
if let transcriptFilePath = config.transcriptFilePath {
args += ["-transcript-file", transcriptFilePath]