diff --git a/CLAUDE.md b/CLAUDE.md index e1c0a04..a29917f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -499,6 +499,90 @@ succeeded." Not done yet — everything under "Future" in Goal (question-detection/CR-generation post-processing) — still explicitly out of scope until asked for. +## Real-time toggle + batch (post-hoc) transcription (2026-09-18) — implemented + +Motivated by a discussion about splitting the architecture (UI on this MacBook, AI on a local +Ubuntu server): live streaming turned out to bring real complexity (network audio protocol, +buffering/reconnect, no-disk-storage constraints), while **batch transcription after the +meeting sidesteps nearly all of it** — and it turned out the raw audio was already being +recorded, just as an undocumented debug tee (see below). The network/server split itself is +still **not implemented** — this is the local groundwork (two capture modes) that a future +server-side batch pass could build on, not the split itself. + +**Found first**: `transcriptor-ai` already wrote every session's raw audio to +`os.TempDir()/transcriptor-ai-{system,mic}.pcm`, unconditionally, undocumented in this repo's +own CLAUDE.md — a debug tee ("for listening back later"), not a designed feature: fixed +filenames (a second session silently overwrote the first), no configurable location, no +cleanup. This session promoted it into a real, deliberate artifact — the user's own framing: +"les PCM sont déjà stockés" — rather than building batch transcription on top of an accidental +side effect. + +**`transcriptor-ai` changes** (`cmd/transcriptor-ai/main.go`, new `cmd/transcriptor-ai/transcribe.go`): +- `-recording-dir` (optional, defaults to the OS temp dir if empty) + session-start-timestamped + filenames (`-system.pcm` / `-mic.pcm`, matching the transcript file's + existing timestamp convention) replace the old fixed names — no more silent overwrite between + sessions. The mic file is only created when `-capture-mic` is actually on, so a disabled track + never leaves behind a misleading empty file. +- `-realtime` (default `true`). `false` skips VAD/ASR/SSE entirely (no HTTP server started) and + only captures+records — capture itself, and the PCM tee, run unconditionally either way. + Implemented with nil-channel `select` cases (`segments`/`httpServer` left nil when disabled) + rather than duplicating the run loop — standard Go idiom, kept the diff to `run()` itself + fairly contained rather than forking into two code paths. +- New `transcribe` subcommand (`transcriptor-ai transcribe -system-pcm=... -mic-pcm=... + -asr-binary=... -asr-model-dir=... -transcript-file=...`, at least one of + `-system-pcm`/`-mic-pcm` required): replays a previously recorded PCM file as a stream of + `capture.Chunk`s (synthetic timestamps advanced by bytes-read/audio-duration, not wall clock, + since a file is read as fast as disk I/O allows — see `bytesToDuration`/`readPCMFile`) through + the **same, unmodified** `internal/vad`/`internal/asr` pipeline live mode uses, then + `transcribeWorker` (also unmodified/reused) writes the result to `-transcript-file`. No new + segmentation or ASR logic — the batch mode is entirely a different *source* for + `capture.Chunk`, not a different pipeline. No HTTP server in this mode; a `Broadcaster` is + still constructed (Publish-to-zero-subscribers is a harmless no-op) purely to satisfy + `transcribeWorker`'s existing signature without special-casing a nil broadcaster there. +- **Verified for real, not just "it compiles"**: built a standalone binary, synthesized a PCM + file (silence/tone/silence/tone/silence, tones above VAD's speech threshold) with a throwaway + Python script, and a stub `qwen_asr` shell script that just echoes fixed text reading stdin. + Ran `transcriptor-ai transcribe` against them — got exactly 2 correctly VAD-segmented, + transcribed, file-written lines (matching the 2 tone bursts), clean exit 0. Confirms the + concurrency (two-goroutine file readers → shared chunk channel → VAD → sequential ASR → file) + doesn't deadlock or panic. **Not yet verified against a real recorded session + real + qwen_asr/model** — do that before trusting batch transcription quality, this only proves the + plumbing. + +**`transcriptor-ui` (Swift) changes**: +- `TranscriptorProcess.Config` gained `recordingDir` (required, passed as `-recording-dir`) and + `realtime` (passed as `-realtime=\(value)`, same boolean-flag `-flag=value` requirement + documented above for `-capture-system`/`-capture-mic`). +- `ContentView`: new "Real-time transcription" `Toggle`, `@State`-backed (not `@AppStorage`) — + deliberately matching `captureSystem`/`captureMic`'s existing treatment, not the + `@AppStorage`-persisted settings-page values; revisit together if the user ever wants the + main-window toggles remembered across relaunches. When off, `ContentView.start()` skips + building a live transcript file path entirely and never connects `SSEClient` (nothing would + ever arrive on it). +- New `BatchTranscriber.swift` — a separate, one-shot `Process` runner (`@Observable`, async + `run(_:)` that throws on nonzero exit) distinct from `TranscriptorProcess`: no SIGINT-based + graceful-drain protocol to replicate, since batch mode isn't an interactive session. +- New "Transcribe a recording…" button in `ContentView`: `NSOpenPanel` (filtered to `.pcm`, + starting in the recordings directory) lets the user pick *either* file of a session — the + sibling track is located automatically from the shared `-system.pcm` / + `-mic.pcm` naming convention (`ContentView.recordingSession(from:)`). **Scope + deliberately restricted to this app's own recordings, not arbitrary audio files** — an + explicit choice this session ("pour le moment, juste pcm"), not an oversight; supporting + arbitrary formats would need an `ffmpeg`-style decode step this doesn't have. Output transcript + is named `-batch.txt` — suffixed distinctly from the live transcript's own + `.txt` so a batch pass can never collide with or silently append into a live + transcript from the same session. +- `SettingsView`: new "Save recordings to" row (`RecordingSaveLocation`, default + `~/Documents/Transcriptor/Recordings`, `@AppStorage`-persisted, `NSOpenPanel` "Choose…" — + same pattern as the existing transcript-directory row), promoting the recording location from + an invisible temp-dir default to a real, user-visible, user-configurable setting. +- **Verification status**: `xcodebuild` succeeds (embedding the freshly rebuilt + `transcriptor-ai` with all the above), and the built app launches and quits cleanly with no + crash. **Not yet verified interactively** — the new toggle/button haven't been clicked, and + batch transcription hasn't been run end to end from the UI against a real recorded session. + Do that (start a short real session, stop it, then use "Transcribe a recording…" on the + resulting `.pcm` pair) before considering this feature done, not just built. + ## Installing to /Applications (not automated — do this manually when needed) There's no install script; this was done by hand once (2026-08-08) and documented here so it diff --git a/transcriptor/BatchTranscriber.swift b/transcriptor/BatchTranscriber.swift new file mode 100644 index 0000000..13965a4 --- /dev/null +++ b/transcriptor/BatchTranscriber.swift @@ -0,0 +1,95 @@ +// +// BatchTranscriber.swift +// transcriptor +// +// Runs `transcriptor-ai transcribe` — post-hoc transcription of a +// previously recorded PCM session (see TranscriptorProcess's +// -recording-dir, which is what produces the .pcm files this reads) — as +// a one-shot subprocess, distinct from TranscriptorProcess (which spawns +// the long-lived live-capture process). No SIGINT-based graceful shutdown +// protocol needed here: this isn't an interactive session, just a +// synchronous batch job that runs to completion or is killed outright. +// + +import Foundation + +enum BatchTranscriberError: Error { + case alreadyRunning + case binaryNotFound(String) + case processFailed(Int32) +} + +@Observable +final class BatchTranscriber { + private(set) var isRunning = false + private var process: Process? + + struct Config { + var binaryPath: String + var systemPCMPath: String? + var micPCMPath: String? + var asrBinaryPath: String + var asrModelDir: String + var prompt: String? + var transcriptFilePath: String + } + + /// Runs the batch transcription to completion. Throws on a nonzero + /// exit status; stderr (transcriptor-ai's own progress lines, plus + /// qwen_asr's) is forwarded to Console output the same way + /// TranscriptorProcess does for the live process. + func run(_ config: Config) async throws { + guard process == nil else { throw BatchTranscriberError.alreadyRunning } + for path in [config.binaryPath, config.asrBinaryPath] { + guard FileManager.default.fileExists(atPath: path) else { + throw BatchTranscriberError.binaryNotFound(path) + } + } + + let process = Process() + process.executableURL = URL(fileURLWithPath: config.binaryPath) + + var args = [ + "transcribe", + "-asr-binary", config.asrBinaryPath, + "-asr-model-dir", config.asrModelDir, + "-transcript-file", config.transcriptFilePath, + ] + if let systemPCMPath = config.systemPCMPath { + args += ["-system-pcm", systemPCMPath] + } + if let micPCMPath = config.micPCMPath { + args += ["-mic-pcm", micPCMPath] + } + if let prompt = config.prompt, !prompt.isEmpty { + args += ["-prompt", prompt] + } + process.arguments = args + process.standardError = FileHandle.standardError + + self.process = process + isRunning = true + defer { + self.process = nil + isRunning = false + } + + try process.run() + try await withCheckedThrowingContinuation { (continuation: CheckedContinuation) in + process.terminationHandler = { p in + if p.terminationStatus == 0 { + continuation.resume() + } else { + continuation.resume(throwing: BatchTranscriberError.processFailed(p.terminationStatus)) + } + } + } + } + + /// Interrupts an in-progress batch job (e.g. the user closing the + /// progress sheet mid-run). No graceful-drain protocol on the other + /// end like TranscriptorProcess.stop() relies on — this just kills it. + func cancel() { + process?.interrupt() + } +} diff --git a/transcriptor/ContentView.swift b/transcriptor/ContentView.swift index 3ebbd7e..d3fdeb1 100644 --- a/transcriptor/ContentView.swift +++ b/transcriptor/ContentView.swift @@ -10,7 +10,9 @@ // oversight, see CLAUDE.md "Goal". // +import AppKit import SwiftUI +import UniformTypeIdentifiers private enum BundledPaths { static var audioteeBinary: String { @@ -47,13 +49,23 @@ struct ContentView: View { // mid-session and produced a confusing, uncontrolled test run. @AppStorage("captureSystem") private var captureSystem = true @AppStorage("captureMic") private var captureMic = true + // @State, not @AppStorage: same treatment as captureSystem/captureMic + // before their persistence fix wasn't asked for — revisit if the user + // wants this remembered across relaunches too. + @State private var realtimeEnabled = true @AppStorage(TranscriptSaveLocation.defaultsKey) private var transcriptSaveDirectory = TranscriptSaveLocation.defaultDirectory.path + @AppStorage(RecordingSaveLocation.defaultsKey) + private var recordingSaveDirectory = RecordingSaveLocation.defaultDirectory.path @AppStorage(GlossaryFileLocation.defaultsKey) private var glossaryFilePath = GlossaryFileLocation.defaultPath.path + @State private var batchTranscriber = BatchTranscriber() + @State private var batchStatus: String? + @State private var batchError: String? + private var controlsDisabled: Bool { - transcriptorProcess.isRunning || modelManager.isDownloading + transcriptorProcess.isRunning || modelManager.isDownloading || batchTranscriber.isRunning } var body: some View { @@ -74,6 +86,10 @@ struct ContentView: View { .disabled(controlsDisabled) Toggle("Microphone", isOn: $captureMic) .disabled(controlsDisabled) + + Divider().frame(height: 16) + Toggle("Real-time transcription", isOn: $realtimeEnabled) + .disabled(controlsDisabled) } HStack { @@ -84,7 +100,12 @@ struct ContentView: View { Task { await start() } } } - .disabled(modelManager.isDownloading || (!captureSystem && !captureMic)) + .disabled(modelManager.isDownloading || batchTranscriber.isRunning || (!captureSystem && !captureMic)) + + Button("Transcribe a recording…") { + chooseBatchRecording() + } + .disabled(controlsDisabled) Text(statusText) .foregroundStyle(.secondary) @@ -108,6 +129,29 @@ struct ContentView: View { .padding([.horizontal, .bottom]) } + if batchTranscriber.isRunning || batchStatus != nil || batchError != nil { + VStack(alignment: .leading, spacing: 4) { + if batchTranscriber.isRunning { + HStack(spacing: 8) { + ProgressView().controlSize(.small) + Text("Transcribing recording… (see Console for progress)") + } + .font(.caption) + .foregroundStyle(.secondary) + } else if let batchStatus { + Text(batchStatus) + .font(.caption) + .foregroundStyle(.secondary) + } + if let batchError { + Text(batchError) + .font(.caption) + .foregroundStyle(.red) + } + } + .padding([.horizontal, .bottom]) + } + Divider() ScrollViewReader { proxy in @@ -154,12 +198,20 @@ struct ContentView: View { return } + // No live transcript file (and no point connecting the SSE client + // below) when real-time transcription is off — transcriptor-ai + // skips VAD/ASR/SSE entirely in that mode and only records, see + // -realtime in TranscriptorProcess. let transcriptFilePath: String? - do { - transcriptFilePath = try makeTranscriptFilePath(startedAt: Date()) - } catch { - errorMessage = "Couldn't prepare transcript save location: \(error)" - return + if realtimeEnabled { + do { + transcriptFilePath = try makeTranscriptFilePath(startedAt: Date()) + } catch { + errorMessage = "Couldn't prepare transcript save location: \(error)" + return + } + } else { + transcriptFilePath = nil } let config = TranscriptorProcess.Config( @@ -171,10 +223,13 @@ struct ContentView: View { transcriptFilePath: transcriptFilePath, captureSystem: captureSystem, captureMic: captureMic, - glossary: readGlossaryFile() + glossary: readGlossaryFile(), + recordingDir: recordingSaveDirectory, + realtime: realtimeEnabled ) do { try transcriptorProcess.start(config) + guard realtimeEnabled else { return } // transcriptor-ai's HTTP server starts early in its own // startup, but not instantly — give it a moment before // connecting rather than building retry logic for this @@ -232,6 +287,104 @@ struct ContentView: View { guard !trimmed.isEmpty else { return nil } return "Preserve spelling: \(trimmed)" } + + /// Lets the user pick either .pcm file of a recorded session (see + /// -recording-dir); the sibling track is located automatically from the + /// shared "-system.pcm" / "-mic.pcm" naming + /// convention transcriptor-ai writes (see readPCMFile/main.go there) — + /// per spec, batch transcription only targets sessions this app itself + /// recorded, not arbitrary audio files, so this naming convention is a + /// safe assumption rather than a format to validate defensively. + private func chooseBatchRecording() { + let panel = NSOpenPanel() + panel.canChooseDirectories = false + panel.canChooseFiles = true + panel.allowsMultipleSelection = false + panel.prompt = "Transcribe" + panel.directoryURL = URL(fileURLWithPath: recordingSaveDirectory) + if let pcmType = UTType(filenameExtension: "pcm") { + panel.allowedContentTypes = [pcmType] + } + + guard panel.runModal() == .OK, let url = panel.url else { return } + let session = recordingSession(from: url) + guard session.systemPath != nil || session.micPath != nil else { + batchError = "Couldn't find a matching recording for \(url.lastPathComponent)." + return + } + Task { await runBatchTranscription(session) } + } + + private struct RecordingSession { + let stamp: String + let systemPath: String? + let micPath: String? + } + + private func recordingSession(from pickedFile: URL) -> RecordingSession { + let directory = pickedFile.deletingLastPathComponent() + let name = pickedFile.deletingPathExtension().lastPathComponent + let stamp: String + if name.hasSuffix("-system") { + stamp = String(name.dropLast("-system".count)) + } else if name.hasSuffix("-mic") { + stamp = String(name.dropLast("-mic".count)) + } else { + stamp = name + } + + let systemCandidate = directory.appendingPathComponent("\(stamp)-system.pcm").path + let micCandidate = directory.appendingPathComponent("\(stamp)-mic.pcm").path + let fm = FileManager.default + return RecordingSession( + stamp: stamp, + systemPath: fm.fileExists(atPath: systemCandidate) ? systemCandidate : nil, + micPath: fm.fileExists(atPath: micCandidate) ? micCandidate : nil + ) + } + + private func runBatchTranscription(_ session: RecordingSession) async { + batchError = nil + batchStatus = nil + + let modelDir: URL + do { + modelDir = try await modelManager.ensureModel(selectedModel) + } catch { + batchError = "Model download failed: \(error)" + return + } + + // Suffixed distinctly from the live transcript's own + // ".txt" naming so a batch pass never collides with (or + // silently appends into) a live transcript from the same session. + let transcriptPath: String + do { + let directory = URL(fileURLWithPath: transcriptSaveDirectory) + try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true) + transcriptPath = directory.appendingPathComponent("\(session.stamp)-batch.txt").path + } catch { + batchError = "Couldn't prepare transcript save location: \(error)" + return + } + + let config = BatchTranscriber.Config( + binaryPath: BundledPaths.transcriptorAiBinary, + systemPCMPath: session.systemPath, + micPCMPath: session.micPath, + asrBinaryPath: BundledPaths.qwenAsrBinary, + asrModelDir: modelDir.path, + prompt: readGlossaryFile(), + transcriptFilePath: transcriptPath + ) + + do { + try await batchTranscriber.run(config) + batchStatus = "Transcribed -> \(transcriptPath)" + } catch { + batchError = "Batch transcription failed: \(error)" + } + } } #Preview { diff --git a/transcriptor/SettingsView.swift b/transcriptor/SettingsView.swift index a0b5646..5f0da33 100644 --- a/transcriptor/SettingsView.swift +++ b/transcriptor/SettingsView.swift @@ -21,6 +21,20 @@ enum TranscriptSaveLocation { } } +/// Where transcriptor-ai's raw PCM recording (system/mic .pcm, one pair per +/// session, timestamped) is saved. Promoted from an undocumented debug tee +/// into `os.TempDir()` to a real, user-visible artifact: it's now the input +/// to batch transcription (see BatchTranscriber), not just "for listening +/// back later" — so it needs a stable, discoverable location like the +/// transcript directory, not a temp path that could be cleared any time. +enum RecordingSaveLocation { + static let defaultsKey = "recordingSaveDirectory" + + static var defaultDirectory: URL { + TranscriptSaveLocation.defaultDirectory.appendingPathComponent("Recordings", isDirectory: true) + } +} + /// The glossary lives in a plain text file the user edits directly (in /// whatever app they like), not a text field in this Settings window — /// simpler for anything longer than a few words, and avoids SwiftUI @@ -38,6 +52,8 @@ enum GlossaryFileLocation { struct SettingsView: View { @AppStorage(TranscriptSaveLocation.defaultsKey) private var transcriptSaveDirectory = TranscriptSaveLocation.defaultDirectory.path + @AppStorage(RecordingSaveLocation.defaultsKey) + private var recordingSaveDirectory = RecordingSaveLocation.defaultDirectory.path @AppStorage(GlossaryFileLocation.defaultsKey) private var glossaryFilePath = GlossaryFileLocation.defaultPath.path @@ -73,6 +89,27 @@ struct SettingsView: View { } } + GridRow { + Text("Save recordings to") + Text(recordingSaveDirectory) + .lineLimit(1) + .truncationMode(.middle) + .foregroundStyle(.secondary) + .frame(minWidth: pathMinWidth, alignment: .leading) + Button("Choose…") { + chooseRecordingDirectory() + } + } + + GridRow { + Color.clear.gridCellUnsizedAxes([.horizontal, .vertical]) + Text("Raw audio (system/mic) recorded during every session, used as the input for batch transcription (Transcribe a recording…) — separate from the transcript text above.") + .font(.callout) + .foregroundStyle(.secondary) + .fixedSize(horizontal: false, vertical: true) + .gridCellColumns(3) + } + GridRow { Text("Glossary file") Text(glossaryFilePath) @@ -120,6 +157,19 @@ struct SettingsView: View { } } + private func chooseRecordingDirectory() { + let panel = NSOpenPanel() + panel.canChooseDirectories = true + panel.canChooseFiles = false + panel.canCreateDirectories = true + panel.prompt = "Choose" + panel.directoryURL = URL(fileURLWithPath: recordingSaveDirectory) + + if panel.runModal() == .OK, let url = panel.url { + recordingSaveDirectory = url.path + } + } + /// NSSavePanel, not NSOpenPanel: lets the user pick a location for a /// file that may not exist yet (a fresh glossary), the same way a /// "Save As…" dialog would — NSOpenPanel can only select files that diff --git a/transcriptor/TranscriptorProcess.swift b/transcriptor/TranscriptorProcess.swift index a01d661..e727865 100644 --- a/transcriptor/TranscriptorProcess.swift +++ b/transcriptor/TranscriptorProcess.swift @@ -29,6 +29,8 @@ final class TranscriptorProcess { var captureSystem = true var captureMic = true var glossary: String? + var recordingDir: String + var realtime = true } func start(_ config: Config) throws { @@ -54,6 +56,8 @@ final class TranscriptorProcess { "-http-addr", config.httpAddr, "-capture-system=\(config.captureSystem)", "-capture-mic=\(config.captureMic)", + "-recording-dir", config.recordingDir, + "-realtime=\(config.realtime)", ] if let transcriptFilePath = config.transcriptFilePath { args += ["-transcript-file", transcriptFilePath]