8bd1a6caf8
SwiftUI app, the one-click way to use the local transcription pipeline — installs and runs with no separate manual steps. The Xcode build phase (scripts/embed-binaries.sh) builds audiotee (git submodule, pinned to a commit on our own Gitea fork) and transcriptor-ai + qwen_asr (sibling checkout, which itself vendors qwen-asr as a submodule), then embeds all three binaries in the app bundle. The AI model is the one thing still fetched at runtime, via ModelManager.swift, since it's multi-GB and doesn't belong baked into a build. Also implements: model download with real progress UI, capture source selection (system/mic/both), a Settings window for the transcript save location and a glossary file (biases ASR toward proper nouns/product names via qwen_asr's --prompt), and a live chat-bubble transcript view. See CLAUDE.md for the full architecture, the reasoning behind each decision, and a fairly long list of real bugs found via actual testing (not just code review) — TCC permission escalation through a subprocess tree, SwiftUI Form/Grid layout quirks, @State vs @AppStorage persistence, a prompt-leakage bug in the glossary feature, and more. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
513 lines
36 KiB
Markdown
513 lines
36 KiB
Markdown
# transcriptor-ui
|
|
|
|
Native macOS SwiftUI app — the "one-click" way to actually use the local meeting
|
|
transcription project. This file is for working on the code: decisions, reasoning,
|
|
conventions, current status. User-facing install/usage docs belong in `README.md` once the
|
|
app exists (see `../transcriptor-ai`'s and `../audiotee`'s READMEs for the pattern this repo
|
|
should follow — don't mix the two kinds of doc in this file).
|
|
|
|
Design/decision history lives in a long chat transcript, not in this repo yet. If something
|
|
here seems arbitrary, assume there was a reason discussed elsewhere rather than
|
|
re-litigating it from scratch — but flag it if it looks wrong given the current code.
|
|
|
|
## Goal
|
|
|
|
**"One-click" means installation too, not just usage** — this was an explicit correction
|
|
from the user, not an assumption to relax. The end state: someone downloads/builds this one
|
|
`.app`, double-clicks it, and it works — no separate manual steps to build or locate
|
|
`audiotee`, `qwen_asr`, or `transcriptor-ai` first. The **only** thing fetched at runtime is
|
|
the AI model (multi-GB, deliberately not bundled — same reasoning as why Ollama/LM Studio
|
|
don't ship models inside their app bundle either).
|
|
|
|
Feature scope for the UI itself:
|
|
- Configure a few options at launch — which ASR model to use, to start.
|
|
- Display the live transcript (SSE client, same protocol `transcript-tail` already uses).
|
|
- Save the transcript to a text file, live, so a crash loses as little as possible.
|
|
**Already implemented on the `transcriptor-ai` side**, not here — see "Architecture" below,
|
|
this was a deliberate decision, don't revisit without reason.
|
|
- Future (not now, don't build preemptively): configuring a per-meeting glossary; a
|
|
post-processing UI for LLM-based features the user described — detecting a
|
|
user-supplied list of questions being asked/answered during the meeting and logging the
|
|
answers, and generating a meeting summary/CR. These are real intended features, not
|
|
speculative — but out of scope until the core "one-click live transcript" app works.
|
|
|
|
## Target hardware
|
|
|
|
Same as `transcriptor-ai`: **MacBook (base) M3, 24 GB unified memory** is the deployment
|
|
target, not the M4 Max/128GB dev machine. Carries over here mainly for any future
|
|
model-size/UI-responsiveness decisions (e.g. don't assume headroom for running something
|
|
heavy inside the UI process itself).
|
|
|
|
## Architecture
|
|
|
|
```
|
|
transcriptor-ui.app (SwiftUI)
|
|
├─ Contents/Resources/audiotee (embedded, prebuilt+signed — see below)
|
|
├─ Contents/Resources/qwen_asr (embedded, prebuilt — see below)
|
|
├─ Contents/Resources/transcriptor-ai (embedded, prebuilt — see below)
|
|
│
|
|
├─ spawns transcriptor-ai as a subprocess (same pattern as
|
|
│ transcriptor-ai → audiotee: each layer manages the one below it via a
|
|
│ subprocess + a clean protocol, never merged into one project/toolchain)
|
|
│
|
|
├─ SSE client of transcriptor-ai's live transcript endpoint (same protocol
|
|
│ cmd/transcript-tail in transcriptor-ai already implements — port from
|
|
│ that Go code, don't redesign the parsing from scratch)
|
|
│
|
|
└─ on first run (or whenever a selected model is missing locally): downloads
|
|
the model, with progress UI — the only runtime fetch, everything else is
|
|
bundled at build time
|
|
```
|
|
|
|
**Deliberately not a monorepo** with `transcriptor-ai` or `audiotee` — each stays its own
|
|
repo with its own native toolchain (Go, Swift-CLI, Swift-app respectively). Mixing Go and
|
|
Swift build steps into one Xcode project would undermine the reason Go was chosen for
|
|
`transcriptor-ai` in the first place (see that repo's CLAUDE.md, "Why these choices"). Xcode
|
|
build phases (Run Script) reach into the sibling repos instead — a standard pattern, not
|
|
exotic, for embedding externally-built dependencies.
|
|
|
|
### Why these choices (don't re-decide without reason)
|
|
|
|
- **Transcript file writing lives in `transcriptor-ai`, not here.** Decided explicitly by the
|
|
user: `transcriptor-ai` is more crash-resistant than a UI process (simpler control flow),
|
|
and the whole point of that file is surviving a crash — so the more crash-resistant process
|
|
should own it. `transcriptor-ai` already implements this via `-transcript-file` (see its
|
|
`internal/transcript/writer.go`); this app just needs to pass that flag with a path when it
|
|
spawns the subprocess, not write the file itself.
|
|
- **SSE, not WebSocket, for the live transcript** — inherited from `transcriptor-ai`'s
|
|
decision (one-directional flow). A native app connects to SSE the same way a browser's
|
|
`EventSource` or `transcript-tail`'s manual line-parsing does — no special client library
|
|
needed in Swift either (a `URLSession` data task reading line-by-line works, same parsing
|
|
logic as `transcript-tail`).
|
|
- **Binaries embedded at build time, not referenced by a fixed external path.** Rejected the
|
|
simpler "just spawn `~/bin/audiotee` and a hardcoded `qwen_asr` path" approach — that's not
|
|
"one-click install," it requires the user to have separately built and placed those first.
|
|
Real cost of the embedded approach: build phases in Xcode need to invoke `go build`, `make`,
|
|
and Swift's own build across three different toolchains, and the resulting `.app` needs
|
|
correct nested code-signing (see "Known open risk").
|
|
- **Models are the one runtime dependency, deliberately.** Multi-GB downloads don't belong
|
|
baked into a build artifact — same reasoning `transcriptor-ai`'s own README already
|
|
documents for why it doesn't try to bundle them either.
|
|
|
|
### Nested code-signing — resolved (2026-08-08), verified empirically
|
|
|
|
Was an open risk; now answered by actually building and inspecting the result, not just
|
|
reasoning about it. Xcode's default app-signing pass (no `--deep`) does **not** touch nested
|
|
pre-signed executables sitting in `Contents/Resources/` — copying `audiotee` in with a plain
|
|
`cp` and letting Xcode sign the outer `.app` normally works cleanly:
|
|
- `codesign -dvvv transcriptor.app/Contents/Resources/audiotee` still shows
|
|
`Authority=sttlab-apps` after the outer app is built and signed — untouched.
|
|
- `codesign --verify --verbose transcriptor.app` → "valid on disk", "satisfies its Designated
|
|
Requirement".
|
|
- `qwen_asr` and `transcriptor-ai` (ad-hoc signed by their own toolchains, Go/clang) sit in
|
|
Resources/ too and run fine — they never call Core Audio themselves, so they don't need a
|
|
real identity, just to be executable at all, which ad-hoc signing satisfies locally.
|
|
|
|
No `--deep`, no re-signing step, no special handling needed. If this ever needs revisiting
|
|
(e.g. after adding Hardened Runtime, or targeting notarization/distribution — neither planned
|
|
now), re-verify rather than assume this still holds.
|
|
|
|
## Dependencies (what this repo's build reaches into)
|
|
|
|
Implemented: `scripts/embed-binaries.sh`, invoked by a "Run Script" build phase named "Embed
|
|
audiotee, transcriptor-ai, qwen_asr" (last phase on the `transcriptor` target, after
|
|
Resources). It:
|
|
- `../audiotee` — runs `scripts/build-signed.sh` there, then copies the resulting
|
|
`~/bin/audiotee` in. Doesn't reimplement audiotee's signing logic, just invokes it.
|
|
- `../transcriptor-ai` — runs `scripts/build-dist.sh` there (builds both `transcriptor-ai`
|
|
and `qwen_asr`, the latter from a pinned git submodule, `third_party/qwen-asr`, inside that
|
|
repo, into `dist/`), then copies `dist/*` in. This repo still doesn't need to know about
|
|
qwen-asr directly — that stays `transcriptor-ai`'s dependency to manage.
|
|
- Copies all three into `Contents/Resources/` in the built app, `chmod +x`'d.
|
|
- Prepends `/opt/homebrew/bin` to `PATH` at the top of the script — Xcode Run Script phases
|
|
don't inherit a normal shell's PATH, so `go` (used by `transcriptor-ai`'s build) isn't found
|
|
otherwise. Bit us once already during manual testing this session; same fix here.
|
|
- Testable standalone outside Xcode — see the script's own header comment for the exact env
|
|
vars to set (`SRCROOT`, `BUILT_PRODUCTS_DIR`, `UNLOCALIZED_RESOURCES_FOLDER_PATH`). Do this
|
|
before debugging inside Xcode if the phase ever fails — much faster iteration loop.
|
|
|
|
Two Xcode project settings had to be disabled for this to work, both build-setting keys (not
|
|
files) — `ENABLE_APP_SANDBOX = NO` (a sandboxed app cannot spawn arbitrary external
|
|
processes, fundamentally incompatible with what this app does) and
|
|
`ENABLE_USER_SCRIPT_SANDBOXING = NO` (a separate, newer Xcode feature that sandboxes Run
|
|
Script build phases themselves — blocked the script from even reading its own file otherwise,
|
|
independent of the app's runtime sandbox). Both are the right call for a personal,
|
|
non-App-Store tool, same reasoning as `audiotee` itself not being sandboxed.
|
|
|
|
`transcriptor-ai` needed a new flag to make this fully self-contained: `-audiotee-binary`
|
|
(otherwise it only found `audiotee` via `PATH` then `~/bin/audiotee` — fine for CLI use, not
|
|
for a bundled app that shouldn't depend on anything pre-installed outside itself). Verified
|
|
by moving `~/bin/audiotee` aside entirely before testing — capture still worked, proving the
|
|
embedded copy was actually being used, not a leftover fallback.
|
|
|
|
## Conventions
|
|
|
|
- Swift/SwiftUI idioms, standard Xcode project conventions once the project exists.
|
|
- Code and comments in English. Conversation with the user in French (matches the sibling
|
|
repos' convention).
|
|
- User-facing install/usage docs belong in `README.md` once one exists, not here.
|
|
- Don't reintroduce a monorepo, WebSocket, or reimplement audiotee/qwen-asr build logic
|
|
locally instead of invoking the sibling repos' own scripts — see "Why these choices."
|
|
|
|
## Project facts (don't regenerate the Xcode project — it exists, edit it)
|
|
|
|
- Xcode project lives at **`transcriptor.xcodeproj`, directly at the repo root** —
|
|
`transcriptor/`, `transcriptorTests/`, `transcriptorUITests/` sit as flat sibling folders
|
|
next to it. Xcode's project-creation flow initially nests everything one level deeper
|
|
(inside a `transcriptor/` wrapper folder named after the product, i.e.
|
|
`transcriptor-ui/transcriptor/transcriptor.xcodeproj`) — that nesting was deliberately
|
|
removed (moved everything up one level, verified with a clean build afterward that nothing
|
|
broke — Xcode's internal paths are relative to the `.xcodeproj`'s own location, so this is
|
|
safe). If you ever regenerate or re-import this project, redo that flattening.
|
|
- Product name: **`transcriptor`** (the built app is `transcriptor.app`; the repo is
|
|
`transcriptor-ui`, deliberately different — don't rename to match, that's already decided).
|
|
- Bundle identifier: **`eu.sttlab.transcriptor`** (organization identifier `eu.sttlab`, not
|
|
`com.stephanetailland` — matches the `sttlab-apps` code-signing certificate already created
|
|
for `audiotee`; this is the namespace for the user's personal apps going forward).
|
|
- Targets: `transcriptor` (the app), `transcriptorTests` (Swift Testing, chosen over XCTest —
|
|
Apple's current default), `transcriptorUITests`. Storage: None (no Core Data/SwiftData —
|
|
config is small enough for `@AppStorage`, the transcript file is owned by
|
|
`transcriptor-ai`, not this app — see "Why these choices").
|
|
- Built and launch-tested once already (`xcodebuild ... build`, then `open` the resulting
|
|
`.app`, confirmed the process actually runs) — this is the stock SwiftUI template
|
|
(`ContentView.swift` / `transcriptorApp.swift`), no project-specific code yet.
|
|
- `.gitignore` added after project creation (Xcode's "create project" flow does not add one
|
|
automatically when you opt out of "Create Git repository on my Mac", which the user did
|
|
since the repo already existed) — `xcuserdata/`, `*.xcuserstate`, `.DS_Store`, etc. Verify
|
|
new xcuserdata files don't creep back into `git status` before committing anything.
|
|
|
|
## Status
|
|
|
|
**The capture→VAD→ASR→SSE circuit is proven end to end from the native UI**, verified with
|
|
real audio (not just "it compiles") — `transcriptor.app` spawns `transcriptor-ai`, which
|
|
spawns `audiotee`, real system audio gets captured (confirmed via `ffmpeg volumedetect`,
|
|
-21dB/-2.8dB, not silence), transcribed, and delivered live over the SSE endpoint to a Swift
|
|
client rendering it in a scrolling list. (Historical note: this paragraph originally described
|
|
a dev-hardcoded-paths version, before embedding and model management existed — see "Resume
|
|
point" below for current, accurate status. Left the bug narrative below as-is since it's still
|
|
useful history, just be aware "Paths are still dev-time hardcoded" no longer applies.)
|
|
|
|
New files: `TranscriptMessage.swift` (mirrors `transcriptor-ai`'s `internal/transcript.Message`
|
|
JSON — kept in sync by hand, no shared schema), `TranscriptorProcess.swift` (subprocess
|
|
spawn/stop), `SSEClient.swift` (Swift port of `cmd/transcript-tail`'s parsing logic),
|
|
`AppDelegate.swift` (see bug below), `Info.plist` (see bug below). `ContentView.swift`
|
|
rewritten with a Start/Stop button and a live-scrolling transcript list.
|
|
|
|
### Two real bugs found via actual testing, not code review
|
|
|
|
- **App Sandbox blocked everything.** The macOS App template enables
|
|
`ENABLE_APP_SANDBOX = YES` by default. A sandboxed app cannot spawn arbitrary external
|
|
processes — fundamentally incompatible with what this app does (spawn `transcriptor-ai`,
|
|
which spawns `audiotee`/`qwen_asr`). Disabled (`ENABLE_APP_SANDBOX = NO` in both Debug and
|
|
Release). Correct call for a personal, non-App-Store tool — same reasoning as why `audiotee`
|
|
itself isn't sandboxed.
|
|
- **TCC "responsible process" escalation silently blocked mic + system audio capture**, with
|
|
**no visible permission prompt** — the most subtle/important finding so far. `audiotee`
|
|
already has a valid, working signed identity (`sttlab-apps`) and its own embedded
|
|
`Info.plist` with the usage-description keys. That was NOT enough once `audiotee` runs as a
|
|
grandchild of a brand-new GUI app: macOS's TCC appears to attribute the permission request
|
|
to the top-level "responsible" app in the process tree (`transcriptor.app`), not just the
|
|
leaf binary that actually calls the Core Audio APIs — and since `transcriptor.app`'s
|
|
Info.plist had no usage-description keys of its own, the request failed outright instead of
|
|
prompting. Confirmed via `log show --predicate 'process == "audiotee"'`
|
|
(`/usr/bin/log`, not the zsh builtin `log` — that shadows it and errors with "too many
|
|
arguments") — a `TCCAccessRequest() IPC` immediately followed by a CoreAudio HAL error whose
|
|
code decodes as ASCII "nope" (`0x6E6F7065` / `1852797029`), with no dialog ever shown.
|
|
**Fix:** `transcriptor.app` needs its own `NSMicrophoneUsageDescription` AND
|
|
`NSAudioCaptureUsageDescription`, same as `audiotee` has. `NSMicrophoneUsageDescription` is
|
|
a recognized `INFOPLIST_KEY_*` build setting Xcode synthesizes automatically, but
|
|
`NSAudioCaptureUsageDescription` is not (same gap `audiotee`'s own CONTEXT.md already
|
|
documented for its Xcode dropdown) — silently dropped if you try `INFOPLIST_KEY_*` for it.
|
|
Switched the target to a real, physical `Info.plist` (`GENERATE_INFOPLIST_FILE = NO`,
|
|
`INFOPLIST_FILE = transcriptor/Info.plist`) with both keys set explicitly, using
|
|
`$(VARIABLE)` placeholders for the usual build-setting-derived values
|
|
(`CFBundleIdentifier`, etc.) so they stay in sync. Had to also add a
|
|
`PBXFileSystemSynchronizedBuildFileExceptionSet` excluding `Info.plist` from "Copy Bundle
|
|
Resources" (the synchronized-group auto-membership otherwise tries to bundle it as a
|
|
resource too, alongside using it as the actual Info.plist — build warning, not fatal, but
|
|
worth doing cleanly). After a `tccutil reset Microphone eu.sttlab.transcriptor` (needed
|
|
because the earlier silent failure may have already recorded a denial) and a rebuild, real
|
|
permission prompts appeared and — once granted — real audio was captured. **Implication for
|
|
later:** the eventual embedded/bundled version needs this too; don't assume `audiotee`'s own
|
|
Info.plist is sufficient just because it worked standalone via CLI.
|
|
- **Quitting the app orphaned `transcriptor-ai`/`audiotee` as background processes** — normal
|
|
`Cmd+Q`/Quit did not stop them, confirmed by quitting and checking `pgrep` afterward. Fixed
|
|
by moving `TranscriptorProcess` ownership to a proper `NSApplicationDelegate`
|
|
(`AppDelegate.swift`) so `applicationWillTerminate` can call `.stop()` (sends SIGINT, same
|
|
graceful shutdown `transcriptor-ai` already has). `ContentView` now receives the shared
|
|
instance via init instead of creating its own `@State`. Verified: start capture, quit via
|
|
the app's own Quit (tested via `osascript ... quit`, which triggers the same termination
|
|
path as the user doing it), `pgrep` shows nothing left running. Known gap, not fixed (can't
|
|
be, at this layer): a force-kill (Activity Monitor, SIGKILL) skips
|
|
`applicationWillTerminate` entirely and would still orphan the children.
|
|
|
|
## ASR quality benchmark (2026-08-09) — real-world WER against YouTube subtitles
|
|
|
|
Ad hoc, not a built tool (deliberately — a one-off validation exercise, don't build a
|
|
permanent benchmark harness unless asked): tested the app against real spoken content by
|
|
playing 5 minutes of two YouTube videos (`test/test-en.webm`, English tutorial; `test/test-fr.mkv`,
|
|
French tutorial) through system audio while `transcriptor.app` captured with mic disabled,
|
|
then compared the saved transcript against each video's auto-generated YouTube subtitles as
|
|
reference (`test/test-en.vtt`, `test/test-fr.vtt` — gitignored, large media, not committed).
|
|
|
|
**Reference-text extraction from YouTube's auto-caption VTT format**: these files use
|
|
word-by-word rolling reveal (`<c>` inline tags) with the growing line repeated across
|
|
consecutive cues — naively joining cue text produces massive duplication. The correct
|
|
reconstruction (validated by inspection): take only the **last** text line of each cue block,
|
|
strip `<...>` tags, and skip consecutive duplicate lines. This is a general fact about
|
|
YouTube's auto-caption format, not project-specific — worth remembering for any future
|
|
subtitle-scraping task, not just this one.
|
|
|
|
**Results**: English WER 6.6%, French WER 8.9% (Levenshtein word-edit-distance / reference
|
|
word count, computed with a throwaway Python script, not committed to the repo). Both
|
|
"quite good" for real, unscripted speech.
|
|
- English: the dominant error was **"Claude" consistently mistranscribed as "cloud"/"claw"/
|
|
"clawed"/"clause"** — a single recurring word accounting for a meaningful share of the
|
|
errors, and exactly the kind of thing lexical biasing (`--prompt`) exists to fix.
|
|
- French: no single dominant error class — singular/plural agreement (often genuinely
|
|
inaudible in spoken French: "agent"/"agents"), proper nouns garbled on *both* sides
|
|
(including in YouTube's own reference — e.g. "SWOT" was itself mistranscribed as "swat" in
|
|
the YouTube reference, while our ASR got it right), and one real content-loss segment
|
|
("être honnête avec vous" → "un intagou") worth keeping an eye on if it recurs.
|
|
- Pipeline mechanics (capture → VAD → ASR → file save) held up correctly through both runs,
|
|
no crashes or dropped audio.
|
|
|
|
## Glossary (2026-08-09) — implemented
|
|
|
|
Directly motivated by the "Claude" mishearing above. **Design: a plain text file, not a
|
|
Settings text field** — the user explicitly rejected an in-app text field after seeing it
|
|
(see the two SwiftUI layout bugs below) in favor of something editable in any text editor.
|
|
- `SettingsView.swift`'s `GlossaryFileLocation` enum: default path
|
|
`~/Library/Application Support/eu.sttlab.transcriptor/glossary.txt`, location persisted via
|
|
`@AppStorage`. Settings UI has "Choose…" (`NSSavePanel`, not `NSOpenPanel` — lets picking a
|
|
location for a file that doesn't exist yet, the way "Save As…" does; `NSOpenPanel` can only
|
|
select existing files) and "Open" (`NSWorkspace.shared.open`, launches the user's default
|
|
text editor for `.txt`).
|
|
- `ContentView.readGlossaryFile()` reads the file's content fresh at every Start (not cached),
|
|
trims whitespace, wraps it as `"Preserve spelling: <content>"`, and passes that as
|
|
`TranscriptorProcess.Config.glossary` → `-prompt` to `transcriptor-ai` → `--prompt` to
|
|
`qwen_asr`. Missing/empty file → `nil`, silently no `-prompt` passed — not a startup error,
|
|
glossary is optional. **The file itself holds only the bare comma-separated terms** (matches
|
|
the Settings UI help text) — the `"Preserve spelling: "` wrapping is applied in code, not
|
|
stored, so the file stays simple to hand-edit.
|
|
|
|
### Glossary prompt-leakage bug (2026-08-09) — found via real testing, mitigated
|
|
|
|
Ran a real WER benchmark (5 min, English test video, qwen3-asr-0.6b + the glossary from
|
|
`--prompt` above) to check whether the glossary actually fixed the earlier "Claude" →
|
|
"cloud"/"claw"/"clause" mishearing. It did — 100% of "Claude" occurrences correct in that run
|
|
— but surfaced a different, real problem: **one transcript segment came back as the glossary
|
|
terms verbatim** (`"claude claude code mcp context7 supabase stripe vercel github typescript
|
|
playwright ui"`), replacing what should have been actual transcribed speech
|
|
(`"assistants are only as good as their training data enforcing it to use web search to
|
|
fetch"`), and that hallucinated segment was marked `is_final: true` like a normal one — the
|
|
model echoed the biasing prompt instead of transcribing the audio for that segment. This is a
|
|
known general failure mode of prompt-conditioned generation ("prompt leakage"), not a
|
|
qwen_asr-specific bug and not something fixable from `transcriptor-ai`'s side (it's the
|
|
underlying model's behavior, not something the Go orchestration layer controls).
|
|
|
|
**Mitigation applied**: the glossary was being passed as a bare comma list
|
|
(`"Claude, Claude Code, MCP, ..."`). `qwen_asr`'s own documented `--prompt` example is
|
|
`"Preserve spelling: CPU, CUDA, PostgreSQL, Redis"` — an instruction-framed prompt, not a bare
|
|
list — and we'd deviated from that. Fixed to match (see `readGlossaryFile()` above). **Not
|
|
proven to eliminate the leak, just: follow the tool's own recommended format instead of an
|
|
untested deviation from it** — re-verify if a future test surfaces leakage again, don't assume
|
|
this fully solved it.
|
|
|
|
**The prefix mitigation alone was NOT sufficient** — re-ran the same French 5-min test with
|
|
the "Preserve spelling: " framing active, and the leak recurred: one segment came back as
|
|
almost the entire glossary verbatim (`"Mistral, IA Studio, SWOT, PESTEL, JSON, TypeScript,
|
|
Playwright, UI/UX, Claude, Claude Code, MCP, Context7, Stripe, Vercel, GitHub"` — 15 of 16
|
|
glossary terms, in the file's exact order), contributing measurably to that run's WER (13.5%
|
|
vs. the original 8.9% 1.7B baseline — though as always, confounded by the model-size
|
|
difference too, see below).
|
|
|
|
**Real fix implemented: `internal/asr/leak.go`, `looksLikePromptLeak(text, prompt string) bool`**
|
|
in `transcriptor-ai`. Called from `Transcriber.Transcribe` right after getting qwen_asr's
|
|
output — if a segment looks like a leak, it's discarded (returns `""`, which the existing
|
|
`if text == "" { continue }` caller logic in `main.go` already treats as "nothing to publish
|
|
this round" — no separate wiring needed there) and a diagnostic line is written to stderr so
|
|
it's visible for debugging without polluting the live transcript or the saved file.
|
|
|
|
Heuristic: normalize both the segment text and the prompt into lowercase word lists (unicode
|
|
letter/digit aware, so accented French words don't get mis-split), and if ≥80% of the
|
|
segment's words also appear among the prompt's words AND the segment has ≥4 words, treat it
|
|
as a leak. The ≥4-word floor exists specifically so a short, legitimate mention of one
|
|
glossary term ("Use Mistral", "in Claude Code") is never flagged — real leaks observed were
|
|
near-total reproductions of the term list, giving a lot of margin between real leaks and
|
|
normal usage. Verified against both real captured leaks (English and French) plus several
|
|
normal-sentence-mentioning-a-glossary-term cases with a throwaway Go test program before
|
|
wiring it in — all classified correctly, not just "it compiled."
|
|
|
|
**Re-verified end to end with a real capture**: reran the same French 5-min benchmark with the
|
|
filter active — the saved transcript file no longer contains the leak pattern, and WER
|
|
improved to 12.5% (from 13.5% with the leak present), consistent with removing the ~15-word
|
|
leaked segment's contribution to the edit distance. Model was still 0.6B in this run (see
|
|
"Model quality" note below — same confound applies, this number isn't a clean 1.7B comparison
|
|
either), but the *qualitative* result — no more verbatim glossary dumps in the output — is the
|
|
actual point of this fix and was directly confirmed, not inferred from the WER delta alone.
|
|
|
|
### Model quality: not actually compared apples-to-apples (2026-08-09)
|
|
|
|
Have three WER data points now, but **still no clean isolated comparison** — every run has
|
|
at least two variables differing at once (model size, glossary presence, and/or the leak bug
|
|
above): 1.7B/no-glossary: 6.6% (EN) / 8.9% (FR); 0.6B/with-glossary/leak-present: 7.3% (EN) /
|
|
13.5% (FR); 0.6B/with-glossary/leak-fixed: 12.5% (FR). None of these isolate the model-size
|
|
variable — an isolated test (0.6B, no glossary, same 5-min clips) was proposed twice this
|
|
session to get a clean comparison but still not run. Do that first if model-size quality is
|
|
ever actually decided; don't reuse any of these numbers as if they isolate the model variable.
|
|
|
|
### Settings persistence bugs (2026-08-09) — `@State` vs `@AppStorage`, two found and fixed
|
|
|
|
Both found the same way: mid-session, the app had to be relaunched (to test a different
|
|
config), and a setting the user had explicitly picked reverted to its hardcoded default
|
|
without warning — read as the app ignoring input, caused real, justified frustration both
|
|
times before the actual cause (not persisted at all, silently) was identified.
|
|
|
|
- `selectedModel` (the ASR model picker) was `@State`, defaulting to `.large` (1.7B) — every
|
|
relaunch silently discarded a `.small` (0.6B) selection back to 1.7B. Fixed:
|
|
`@AppStorage("selectedModel")` — `ModelChoice` is `RawRepresentable` (`String`), which
|
|
`@AppStorage` supports directly, no separate string-backed key needed.
|
|
- `captureSystem`/`captureMic` (the two capture-source toggles) had the identical bug,
|
|
defaulting back to both-on every relaunch. Fixed the same way:
|
|
`@AppStorage("captureSystem")` / `@AppStorage("captureMic")`.
|
|
|
|
**Verified for real, not just "it compiled"**: for `selectedModel`, set the picker to small,
|
|
fully quit the app (not just Stop), relaunched, and confirmed via
|
|
`/usr/libexec/PlistBuddy -c "Print" ~/Library/Preferences/eu.sttlab.transcriptor.plist` that
|
|
`selectedModel = small` persisted on disk and the picker showed 0.6B on the next launch — not
|
|
just eyeballing the UI once. Note: `defaults read eu.sttlab.transcriptor ...` resolves to the
|
|
wrong (sandboxed-style `~/Library/Containers/...`) path for this non-sandboxed app and reports
|
|
"does not exist" even though the real prefs file is fine at the standard
|
|
`~/Library/Preferences/eu.sttlab.transcriptor.plist` location — use `PlistBuddy` directly
|
|
against that path instead of trusting `defaults read`'s domain resolution here.
|
|
|
|
**Check `@State` vs `@AppStorage` deliberately for any new setting added to this view** — this
|
|
was the same bug twice in a row because it's an easy default to reach for; anything the user
|
|
configures and would expect to survive a relaunch needs `@AppStorage`, not `@State`.
|
|
|
|
**Two real SwiftUI layout bugs hit and fixed while building the Settings UI for this** — both
|
|
found by the user looking at actual screenshots, not caught by a successful build:
|
|
1. A multi-line `TextField(..., axis: .vertical)` inside `LabeledContent` inside `Form`
|
|
rendered the placeholder floating above a large empty box instead of inside it — visually
|
|
broken. This was the earlier "text field in Settings" design, since abandoned for the
|
|
plain-file approach above, so this specific bug is moot now — but the general lesson
|
|
(verify below) isn't.
|
|
2. **`Form`'s automatic label-column width calculation broke badly** once a row's content grew
|
|
taller than one line (the glossary row's caption text) — the "Glossary file:" label
|
|
rendered wrapped one letter per line ("G" / "lo" / "s" / "s" / "a" / "ry" / "fil" / "e:"),
|
|
stacked vertically. Fixed by abandoning `Form`/`LabeledContent` entirely for this view in
|
|
favor of a hand-rolled `VStack` with a fixed-width label column
|
|
(`.frame(width: labelWidth, alignment: .trailing)`) — simpler and predictable, not worth
|
|
fighting Form's automatic sizing further. **If this view needs new rows in the future,
|
|
keep using the hand-rolled layout, don't reintroduce `Form`/`LabeledContent` here** — it's
|
|
already demonstrated to break under multi-line content in this exact view.
|
|
|
|
## Resume point
|
|
|
|
**The one-click install goal is fully met, verified end to end with a real cold start.**
|
|
`transcriptor.app` is completely self-contained:
|
|
- Building it (one Xcode build phase) builds and embeds `audiotee`, `transcriptor-ai`, and
|
|
`qwen_asr` — no separate manual step. Verified by moving `~/bin/audiotee` aside and
|
|
confirming capture still worked from the embedded copy alone.
|
|
- Nested code-signing, the biggest open unknown, resolved cleanly with no special handling
|
|
needed (see above).
|
|
- **Model management is implemented** (`ModelManager.swift`): downloads from HuggingFace
|
|
natively (`URLSessionDownloadDelegate`, real per-file progress, resumable — already-present
|
|
files are skipped), stores under `~/Library/Application Support/eu.sttlab.transcriptor/models/`,
|
|
lets the user pick 0.6B vs 1.7B via a `Picker`. Clicking "Start" downloads-then-starts as one
|
|
action if the model isn't there yet — no separate "download" step to remember. **Verified
|
|
with a genuine cold start**: emptied the target directory, clicked Start, watched the 1.7B
|
|
model (~4.7GB, 7 files) download for real, confirmed it auto-started capture immediately
|
|
after, and confirmed real transcription worked against the freshly-downloaded model. Not
|
|
simulated or assumed — this is as close to "what a new user's first launch looks like" as
|
|
testing on the dev machine allows.
|
|
- Two `@Observable`/Swift gotchas hit along the way: the macro doesn't support `lazy var`
|
|
(init-accessor synthesis conflict — fixed with `@ObservationIgnored` on the one lazy
|
|
property, `URLSession`, which isn't UI-relevant state anyway), and a ternary expression
|
|
can't mix a `Void`-returning branch with a `Task { }`-returning branch (fixed with a plain
|
|
`if/else` in the button action instead).
|
|
|
|
**Transcript save location is implemented** (`SettingsView.swift`): a real macOS Settings
|
|
window (`Cmd+,`, the `Settings { }` scene in `transcriptorApp.swift` — not a control bolted
|
|
onto the main window), lets the user pick the save *directory* via `NSOpenPanel`
|
|
(`@AppStorage`-persisted, defaults to `~/Documents/Transcriptor`). The filename itself is not
|
|
user-configurable, per explicit spec — `ContentView.makeTranscriptFilePath` derives it from
|
|
the transcription's start timestamp (`yyyy-MM-dd_HH-mm-ss.txt`), computed fresh each time
|
|
"Start" is pressed. Verified with a real session end to end: file created at the expected
|
|
path with the expected name, content matched the live SSE transcript exactly.
|
|
|
|
Two things worth remembering from getting this verified:
|
|
- **Terminal (and this Bash tool) can't read `~/Documents` on this machine** — a separate,
|
|
per-app macOS "Files and Folders" TCC grant that Terminal doesn't have, unrelated to
|
|
whether `transcriptor.app` itself can write there (it can — the file got created and
|
|
written to correctly). Don't mistake "my shell can't read X" for "the app failed to write
|
|
X" — verify via Finder/the app's own behavior instead when this comes up again.
|
|
- **A "0 bytes" or "no SSE messages" observation immediately after clicking Start is not
|
|
necessarily a bug** — it can simply mean no segment has been transcribed *yet* (the first
|
|
segment takes several seconds: VAD needs speech, then up to ~4s to force-cut, then the ASR
|
|
call itself). This tripped up verification here more than once this session already (see
|
|
the "Two real bugs" section above for the first two times) — when a live-audio test comes
|
|
back empty, redo it with `curl` started *before* the audio plays and a generous timeout,
|
|
rather than assuming something regressed.
|
|
|
|
**Capture source selection and message visual distinction are implemented**, per explicit
|
|
request:
|
|
- Two `Toggle`s ("System audio", "Microphone") in the main window, `@State`-backed (not
|
|
persisted — resets to both-on each launch, unlike the transcript directory setting; that
|
|
wasn't specified either way, revisit if the user wants it remembered). Start is disabled if
|
|
both are off. Wired to two new `transcriptor-ai` flags, `-capture-system`/`-capture-mic`
|
|
(both default `true`) — passed as `-flag=value`, not `-flag value`, because **Go's `flag`
|
|
package does not accept a space-separated value for boolean flags**: `-flag value` leaves
|
|
the flag at its default and the value as a stray positional argument. Verified empirically
|
|
with a throwaway Go program before relying on it, not assumed — see
|
|
`TranscriptorProcess.swift`'s comment at the call site.
|
|
- Important asymmetry, also documented in `transcriptor-ai/CLAUDE.md`: `-capture-mic=false`
|
|
skips mic capture at the `audiotee` level entirely (no permission requested). `-capture-system=false`
|
|
**cannot** do the same — audiotee has no flag to skip its system tap, so it keeps running;
|
|
disabling it just stops `transcriptor-ai` from transcribing that track. Verified via CLI
|
|
(not just via the app) with system audio playing while `-capture-system=false`: zero
|
|
`track: system` messages reached the SSE stream, confirming the drop actually works.
|
|
- `MessageBubble` (new private view in `ContentView.swift`) replaced the plain
|
|
track-label-prefix row with a Messages.app-style layout — mic ("Me") right-aligned in accent
|
|
color, system ("Them") left-aligned in a neutral gray, matching the "me vs them" framing the
|
|
two-track design has used since `audiotee`. Verified functionally (the underlying toggle
|
|
logic, via SSE content) but the visual layout itself was not independently re-confirmed by
|
|
the user after a code change late in this session — worth a quick visual glance next time
|
|
the app is open, not assumed broken, just not re-verified.
|
|
|
|
**Glossary is now implemented** — see "Glossary (2026-08-09)" above, no longer future work.
|
|
Its Settings UI layout was last verified visually to have fixed the label-wrapping bug (build
|
|
succeeded, app relaunched with the hand-rolled VStack layout) but **the user had not yet
|
|
confirmed the fix on screen when this session paused** — do a quick visual check (open
|
|
Settings, `Cmd+,`) before assuming it's resolved, don't just trust the earlier "build
|
|
succeeded."
|
|
|
|
Not done yet — everything under "Future" in Goal (question-detection/CR-generation
|
|
post-processing) — still explicitly out of scope until asked for.
|
|
|
|
## Installing to /Applications (not automated — do this manually when needed)
|
|
|
|
There's no install script; this was done by hand once (2026-08-08) and documented here so it
|
|
doesn't need re-deriving:
|
|
```bash
|
|
xcodebuild -project transcriptor.xcodeproj -scheme transcriptor -configuration Release build
|
|
ditto "$(xcodebuild -project transcriptor.xcodeproj -scheme transcriptor -configuration Release -showBuildSettings | awk -F'= ' '/ CONFIGURATION_BUILD_DIR /{print $2; exit}')/transcriptor.app" /Applications/transcriptor.app
|
|
/System/Library/Frameworks/CoreServices.framework/Versions/Current/Frameworks/LaunchServices.framework/Versions/Current/Support/lsregister -f /Applications/transcriptor.app
|
|
```
|
|
(The exact `CONFIGURATION_BUILD_DIR` used in practice was the DerivedData path directly —
|
|
the `awk`/`-showBuildSettings` version above is untested, just a cleaner-looking equivalent;
|
|
verify it before trusting it blindly.)
|
|
|
|
**Known cosmetic issue, not fixed**: every Xcode build (Debug and Release) registers that
|
|
build's `transcriptor.app` with Launch Services, so Spotlight's raw filename search lists
|
|
multiple entries (DerivedData Debug/Release + the `/Applications` copy) with generic blank
|
|
icons (no `AppIcon` image set in `Assets.xcassets` yet — that's a separate, unaddressed gap).
|
|
**This is cosmetic only** — `open -a transcriptor` (what actually fires when you hit Enter on
|
|
a Spotlight result, or use the Dock/Launchpad) correctly resolves to `/Applications/transcriptor.app`
|
|
regardless, verified by killing all instances and confirming `open -a transcriptor` launched
|
|
the `/Applications` one specifically. Can't be fully cleaned up without deleting the
|
|
DerivedData build products, which would break `⌘R` in Xcode — not worth doing. If it bothers
|
|
the user again, the practical fix is dragging `/Applications/transcriptor.app` to the Dock,
|
|
not chasing the Spotlight listing further.
|