diff --git a/README.md b/README.md index ef465db..a316ea8 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,10 @@ # AudioTee -AudioTee captures your Mac's system audio output and writes it in PCM encoded chunks to `stdout` at regular intervals, either in base64-encoded JSON (good for humans, easy on terminals) or binary (good for other programs). It uses the [Core Audio taps](https://developer.apple.com/documentation/coreaudio/capturing-system-audio-with-core-audio-taps) API introduced in macOS 14.2 (released in December 2023). You can do whatever you want with this audio - stream it somewhere else, save it to disk, visualize it, etc. +**⚠️ API Instability Warning: The AudioTee API is incredibly unstable at present and subject to change without notice.** -By default, it taps the audio output from **all** running process and selects the most appropriate audio chunk output format to use based on the presence of a tty. Tap output is forced to `mono` (not yet configurable) and preserves your output device's sample rate (configurable via the `--sample-rate` flag). Only the default output device is currently supported. +AudioTee captures your Mac's system audio output and writes raw PCM audio data directly to `stdout`. All logging and metadata information is written to `stderr`, allowing for clean redirection of audio data to files or pipes. It uses the [Core Audio taps](https://developer.apple.com/documentation/coreaudio/capturing-system-audio-with-core-audio-taps) API introduced in macOS 14.2 (released in December 2023). You can do whatever you want with this audio - stream it somewhere else, save it to disk, visualise it, etc. + +By default, it taps the audio output from **all** running processes. Tap output is forced to `mono` (not yet configurable) and preserves your output device's sample rate (configurable via the `--sample-rate` flag). Only the default output device is currently supported. My original (and so far only) use case is streaming audio to a parent process which communicates with a realtime ASR service, so AudioTee makes some design decisions you might not agree with. Open an issue or a PR and we can talk about them. I'm also no Swift developer, so contributions improving codebase idioms and general hygiene are welcome. @@ -16,7 +18,7 @@ Recording system audio is harder than it should be on macOS, and folks often wre ## Quick start -The following will start capturing audio output from all running programs and write base64-encoded chunks of it to your terminal every 200ms: +The following will start capturing audio output from all running programs and write raw PCM audio data to your terminal: ```bash git clone git@github.com:makeusabrew/audiotee.git @@ -24,7 +26,13 @@ cd audiotee swift run ``` -If you're not playing audio when you run it, you'll just see packets full of `AAAAA...` - the base64 version of a bunch of zeroes. +More usefully, you can redirect the output to a file: + +```bash +swift run > captured_audio.pcm +``` + +If you're not playing audio when you run it, the PCM file will contain silence (zeroes). ## Build @@ -40,14 +48,17 @@ swift build -c release Replace the path below with `.build///audiotee`, e.g. `build/arm64-apple-macosx/release/audiotee` for a release build on Apple Silicon. ```bash -# Auto-detect output format (JSON in terminal, binary when piped) +# Write raw PCM audio to stdout (logs go to stderr) ./audiotee -# Always use JSON format (terminal-safe) -./audiotee --format json +# Redirect audio to a file +./audiotee > captured_audio.pcm -# Always use binary format (pipe-optimised) -./audiotee --format binary +# Pipe to another program +./audiotee | your_audio_processing_tool + +# Redirect logs as well +./audiotee > captured_audio.pcm 2> audiotee.log ``` ### Audio conversion @@ -94,147 +105,52 @@ Note that trying to include or exclude a PID which isn't currently playing audio ./audiotee --chunk-duration 0.1 ``` -## Output formats +## Output -AudioTee supports two output formats optimised for different use cases: +AudioTee writes raw PCM audio data directly to `stdout` in chunks. All logging, metadata, and status information is written to `stderr`, allowing for clean separation of audio data from program output. -### JSON format (`--format json` or auto in terminal) +### Audio format -JSON messages to stdout, one per line. Audio data is base64-encoded for terminal safety. - -### Binary format (`--format binary` or auto when piped) - -JSON metadata lines followed by raw binary audio data. More efficient for piping to other processes. - -## Protocol - -### Message types - -All messages (except raw binary audio chunks) follow this envelope structure: - -```json -{ - "timestamp": "2024-03-21T15:30:45.123Z", - "message_type": "...", - "data": { ... } -} -``` - -#### 1. Metadata - -Sent once at startup to describe the audio format: - -```json -{ - "timestamp": "2024-03-21T15:30:45.123Z", - "message_type": "metadata", - "data": { - "sample_rate": 48000, - "channels_per_frame": 1, - "bits_per_channel": 32, - "is_float": true, - "capture_mode": "audio", - "device_name": null, - "device_uid": null, - "encoding": "pcm_f32le" - } -} -``` - -#### 2. Stream start - -Indicates audio data will follow: - -```json -{ - "timestamp": "2024-03-21T15:30:45.123Z", - "message_type": "stream_start", - "data": null -} -``` - -#### 3. Audio data - -**JSON format:** - -```json -{ - "timestamp": "2024-03-21T15:30:45.123Z", - "message_type": "audio", - "data": { - "timestamp": "2024-03-21T15:30:45.123Z", - "duration": 0.2, - "peak_amplitude": 0.45, - "audio_data": "base64_encoded_raw_audio..." - } -} -``` - -**Binary format:** - -```json -{ - "timestamp": "2024-03-21T15:30:45.123Z", - "message_type": "audio", - "data": { - "timestamp": "2024-03-21T15:30:45.123Z", - "duration": 0.2, - "peak_amplitude": 0.45, - "audio_length": 9600 - } -} -``` - -_Followed immediately by 9600 bytes of raw binary audio data_ - -#### 4. Stream stop - -Sent when recording stops: - -```json -{ - "timestamp": "2024-03-21T15:30:45.123Z", - "message_type": "stream_stop", - "data": null -} -``` - -#### 5. Log messages - -Info, error, and debug messages (useful for monitoring): - -```json -{ - "timestamp": "2024-03-21T15:30:45.123Z", - "message_type": "info", - "data": { - "message": "Starting AudioTee...", - "context": { "output_format": "auto" } - } -} -``` +- **Format**: Raw PCM audio data +- **Channels**: Mono (1 channel) +- **Sample rate**: Matches your output device's sample rate by default (configurable) +- **Bit depth**: 32-bit float by default, or 16-bit when sample rate conversion is performed +- **Endianness**: Little-endian +- **Chunk duration**: 200ms by default (configurable) ### Consuming output -**JSON format:** +The simplest way to consume AudioTee's output is to redirect it to a file: -1. Parse each line as JSON using the envelope structure -2. Use `metadata` message to understand the audio format -3. For `audio` messages, decode `audio_data` from base64 to get raw PCM data -4. Do something with each chunk of data +```bash +./audiotee > captured_audio.pcm +``` -**Binary format:** +You can then process this raw PCM file with tools like: +- **FFmpeg**: `ffmpeg -f f32le -ar 48000 -ac 1 -i captured_audio.pcm output.wav` +- **SoX**: `sox -t f32 -r 48000 -c 1 captured_audio.pcm output.wav` +- **Audacity**: Import as raw data with the appropriate settings -1. Parse JSON metadata lines using the envelope structure -2. Use `metadata` message to understand the audio format -3. For `audio` messages, read `audio_length` bytes of raw binary data after the JSON line -4. Do something with each chunk of data +For real-time processing, pipe the output to your application: -**Note**: binary is actually a mixed mode; JSON during boot, JSON packet header information preceding each binary chunk. +```bash +./audiotee | your_audio_processing_tool +``` + +### Logs and monitoring + +All program logs are written to `stderr` and can be captured separately: + +```bash +# Capture audio and logs separately +./audiotee > audio.pcm 2> audiotee.log + +# View logs in real-time while capturing audio +./audiotee > audio.pcm 2>&1 | grep "AudioTee" +``` ## Command Line options -- `--format, -f`: Output format (`json`, `binary`, `auto`) [default: `auto`] - `--include-processes`: Process IDs to tap (space-separated, empty = all processes) - `--exclude-processes`: Process IDs to exclude (space-separated, empty = none) - `--mute`: Mute processes being tapped diff --git a/Sources/CLI/ArgumentParser.swift b/Sources/CLI/ArgumentParser.swift index 5917463..260e164 100644 --- a/Sources/CLI/ArgumentParser.swift +++ b/Sources/CLI/ArgumentParser.swift @@ -189,11 +189,6 @@ class SimpleArgumentParser { throw ArgumentParserError.invalidValue(optionName, value) } return doubleValue as! T - } else if type == OutputFormat.self { - guard let format = OutputFormat(rawValue: value) else { - throw ArgumentParserError.invalidValue(optionName, value) - } - return format as! T } throw ArgumentParserError.invalidValue(optionName, value) diff --git a/Sources/CLI/AudioTee.swift b/Sources/CLI/AudioTee.swift index 51da1d7..10895a4 100644 --- a/Sources/CLI/AudioTee.swift +++ b/Sources/CLI/AudioTee.swift @@ -2,7 +2,6 @@ import CoreAudio import Foundation struct AudioTee { - var format: OutputFormat = .auto var includeProcesses: [Int32] = [] var excludeProcesses: [Int32] = [] var mute: Bool = false @@ -18,11 +17,6 @@ struct AudioTee { discussion: """ AudioTee captures system audio using Core Audio taps and streams it as structured output. - Output formats: - • json: Base64-encoded audio in JSON messages (safe for terminals) - • binary: Raw binary audio with JSON metadata headers (efficient for pipes) - • auto: Automatically choose based on whether stdout is a terminal (default) - Process filtering: • include-processes: Only tap specified process IDs (empty = all processes) • exclude-processes: Tap all processes except specified ones @@ -30,8 +24,6 @@ struct AudioTee { Examples: audiotee # Auto format, tap all processes - audiotee --format=json # Always use JSON format - audiotee --format=binary # Always use binary format audiotee --sample-rate=16000 # Convert to 16kHz mono for ASR audiotee --sample-rate=8000 # Convert to 8kHz for telephony audiotee --include-processes 1234 # Only tap process 1234 @@ -42,7 +34,6 @@ struct AudioTee { ) // Configure arguments - parser.addOption(name: "format", shortName: "f", help: "Output format", defaultValue: "auto") parser.addArrayOption( name: "include-processes", help: "Process IDs to include (space-separated, empty = all processes)") @@ -62,7 +53,6 @@ struct AudioTee { var audioTee = AudioTee() // Extract values - audioTee.format = try parser.getValue("format", as: OutputFormat.self) audioTee.includeProcesses = try parser.getArrayValue("include-processes", as: Int32.self) audioTee.excludeProcesses = try parser.getArrayValue("exclude-processes", as: Int32.self) audioTee.mute = parser.getFlag("mute") @@ -102,7 +92,6 @@ struct AudioTee { setupSignalHandlers() Logger.info("Starting AudioTee...") - Logger.debug("Using output format: \(format)") // Validate chunk duration guard chunkDuration > 0 && chunkDuration <= 5.0 else { @@ -143,7 +132,7 @@ struct AudioTee { throw ExitCode.failure } - let outputHandler = createOutputHandler(for: format) + let outputHandler = BinaryAudioOutputHandler() let recorder = AudioRecorder( deviceID: deviceID, outputHandler: outputHandler, convertToSampleRate: sampleRate, chunkDuration: chunkDuration) @@ -172,17 +161,6 @@ struct AudioTee { } } - private func createOutputHandler(for format: OutputFormat) -> AudioOutputHandler { - switch format { - case .json: - return JSONAudioOutputHandler() - case .binary: - return BinaryAudioOutputHandler() - case .auto: - return AutoAudioOutputHandler() - } - } - private func convertProcessFlags() -> ([Int32], Bool) { if !includeProcesses.isEmpty { // Include specific processes only diff --git a/Sources/CLI/OutputFormat.swift b/Sources/CLI/OutputFormat.swift deleted file mode 100644 index a9646bc..0000000 --- a/Sources/CLI/OutputFormat.swift +++ /dev/null @@ -1,16 +0,0 @@ -enum OutputFormat: String, CaseIterable { - case json = "json" - case binary = "binary" - case auto = "auto" - - var description: String { - switch self { - case .json: - return "Base64-encoded JSON (terminal-safe)" - case .binary: - return "Binary with JSON headers (pipe-optimised)" - case .auto: - return "Auto-detect based on TTY (default)" - } - } -} diff --git a/Sources/Core/AudioFormatConverter.swift b/Sources/Core/AudioFormatConverter.swift index c413f2f..d394ac2 100644 --- a/Sources/Core/AudioFormatConverter.swift +++ b/Sources/Core/AudioFormatConverter.swift @@ -119,7 +119,6 @@ public class AudioFormatConverter { return AudioPacket( timestamp: packet.timestamp, duration: packet.duration, - peakAmplitude: packet.peakAmplitude, rawAudioData: outputData ) } diff --git a/Sources/Core/AudioPacket.swift b/Sources/Core/AudioPacket.swift index 0a703cc..5ae6471 100644 --- a/Sources/Core/AudioPacket.swift +++ b/Sources/Core/AudioPacket.swift @@ -3,18 +3,15 @@ import Foundation public struct AudioPacket { public let timestamp: Date public let duration: Double - public let peakAmplitude: Float // useful for level monitoring public let rawAudioData: Data public init( timestamp: Date, duration: Double, - peakAmplitude: Float, rawAudioData: Data ) { self.timestamp = timestamp self.duration = duration - self.peakAmplitude = peakAmplitude self.rawAudioData = rawAudioData } } diff --git a/Sources/Output/Handlers/AutoOutputHandler.swift b/Sources/Output/Handlers/AutoOutputHandler.swift deleted file mode 100644 index 1b38abd..0000000 --- a/Sources/Output/Handlers/AutoOutputHandler.swift +++ /dev/null @@ -1,31 +0,0 @@ -import Foundation - -/// Auto-detecting output handler based on TTY -public class AutoAudioOutputHandler: AudioOutputHandler { - private let handler: AudioOutputHandler - - public init() { - // Auto-detect based on whether stdout is a terminal - if isatty(STDOUT_FILENO) != 0 { - handler = JSONAudioOutputHandler() - } else { - handler = BinaryAudioOutputHandler() - } - } - - public func handleAudioPacket(_ packet: AudioPacket) { - handler.handleAudioPacket(packet) - } - - public func handleMetadata(_ metadata: AudioStreamMetadata) { - handler.handleMetadata(metadata) - } - - public func handleStreamStart() { - handler.handleStreamStart() - } - - public func handleStreamStop() { - handler.handleStreamStop() - } -} diff --git a/Sources/Output/Handlers/BinaryOutputHandler.swift b/Sources/Output/Handlers/BinaryOutputHandler.swift index 7f4e7bb..781c0e2 100644 --- a/Sources/Output/Handlers/BinaryOutputHandler.swift +++ b/Sources/Output/Handlers/BinaryOutputHandler.swift @@ -5,12 +5,6 @@ public class BinaryAudioOutputHandler: AudioOutputHandler { public init() {} public func handleAudioPacket(_ packet: AudioPacket) { - // Create metadata without the audio data - let metadata = BinaryPacketHeader(from: packet) - - // Write JSON metadata line - Logger.writeMessage(.audio, data: metadata) - // Write raw binary audio data directly to stdout FileHandle.standardOutput.write(packet.rawAudioData) } diff --git a/Sources/Output/Handlers/JSONOutputHandler.swift b/Sources/Output/Handlers/JSONOutputHandler.swift deleted file mode 100644 index cb43e9d..0000000 --- a/Sources/Output/Handlers/JSONOutputHandler.swift +++ /dev/null @@ -1,23 +0,0 @@ -import Foundation - -/// Base64-encoded JSON output (terminal-safe) -public class JSONAudioOutputHandler: AudioOutputHandler { - public init() {} - - public func handleAudioPacket(_ packet: AudioPacket) { - let jsonPacket = JSONAudioPacket(from: packet) - Logger.writeMessage(.audio, data: jsonPacket) - } - - public func handleMetadata(_ metadata: AudioStreamMetadata) { - Logger.writeMessage(.metadata, data: metadata) - } - - public func handleStreamStart() { - Logger.writeMessage(.streamStart, data: Optional.none) - } - - public func handleStreamStop() { - Logger.writeMessage(.streamStop, data: Optional.none) - } -} diff --git a/Sources/Output/OutputPacketTypes.swift b/Sources/Output/OutputPacketTypes.swift deleted file mode 100644 index 205a68b..0000000 --- a/Sources/Output/OutputPacketTypes.swift +++ /dev/null @@ -1,45 +0,0 @@ -import Foundation - -/// JSON-serializable version of AudioPacket with base64-encoded audio data -public struct JSONAudioPacket: Codable { - public let timestamp: Date - public let duration: Double - public let peakAmplitude: Float - public let audioData: String // base64 encoded audio data - - public enum CodingKeys: String, CodingKey { - case timestamp - case duration - case peakAmplitude = "peak_amplitude" - case audioData = "audio_data" - } - - public init(from packet: AudioPacket) { - self.timestamp = packet.timestamp - self.duration = packet.duration - self.peakAmplitude = packet.peakAmplitude - self.audioData = packet.rawAudioData.base64EncodedString() - } -} - -/// Metadata-only packet for binary output (without base64 audio data) -public struct BinaryPacketHeader: Codable { - public let timestamp: Date - public let duration: Double - public let peakAmplitude: Float - public let audioLength: Int // Length of raw audio data in bytes - - public enum CodingKeys: String, CodingKey { - case timestamp - case duration - case peakAmplitude = "peak_amplitude" - case audioLength = "audio_length" - } - - public init(from packet: AudioPacket) { - self.timestamp = packet.timestamp - self.duration = packet.duration - self.peakAmplitude = packet.peakAmplitude - self.audioLength = packet.rawAudioData.count - } -} diff --git a/Sources/Utils/Logger.swift b/Sources/Utils/Logger.swift index a89ff67..b19139e 100644 --- a/Sources/Utils/Logger.swift +++ b/Sources/Utils/Logger.swift @@ -24,8 +24,8 @@ public class Logger { let message = Message(type: type, data: data) do { let jsonData = try jsonEncoder.encode(message) - FileHandle.standardOutput.write(jsonData) - FileHandle.standardOutput.write("\n".data(using: .utf8)!) + FileHandle.standardError.write(jsonData) + FileHandle.standardError.write("\n".data(using: .utf8)!) } catch { // TODO: handle at some point }