only write PCM to stdout
This commit is contained in:
Nick Payne
@@ -1,8 +1,10 @@
|
||||
# AudioTee
|
||||
|
||||
AudioTee captures your Mac's system audio output and writes it in PCM encoded chunks to `stdout` at regular intervals, either in base64-encoded JSON (good for humans, easy on terminals) or binary (good for other programs). It uses the [Core Audio taps](https://developer.apple.com/documentation/coreaudio/capturing-system-audio-with-core-audio-taps) API introduced in macOS 14.2 (released in December 2023). You can do whatever you want with this audio - stream it somewhere else, save it to disk, visualize it, etc.
|
||||
**⚠️ API Instability Warning: The AudioTee API is incredibly unstable at present and subject to change without notice.**
|
||||
|
||||
By default, it taps the audio output from **all** running process and selects the most appropriate audio chunk output format to use based on the presence of a tty. Tap output is forced to `mono` (not yet configurable) and preserves your output device's sample rate (configurable via the `--sample-rate` flag). Only the default output device is currently supported.
|
||||
AudioTee captures your Mac's system audio output and writes raw PCM audio data directly to `stdout`. All logging and metadata information is written to `stderr`, allowing for clean redirection of audio data to files or pipes. It uses the [Core Audio taps](https://developer.apple.com/documentation/coreaudio/capturing-system-audio-with-core-audio-taps) API introduced in macOS 14.2 (released in December 2023). You can do whatever you want with this audio - stream it somewhere else, save it to disk, visualise it, etc.
|
||||
|
||||
By default, it taps the audio output from **all** running processes. Tap output is forced to `mono` (not yet configurable) and preserves your output device's sample rate (configurable via the `--sample-rate` flag). Only the default output device is currently supported.
|
||||
|
||||
My original (and so far only) use case is streaming audio to a parent process which communicates with a realtime ASR service, so AudioTee makes some design decisions you might not agree with. Open an issue or a PR and we can talk about them. I'm also no Swift developer, so contributions improving codebase idioms and general hygiene are welcome.
|
||||
|
||||
@@ -16,7 +18,7 @@ Recording system audio is harder than it should be on macOS, and folks often wre
|
||||
|
||||
## Quick start
|
||||
|
||||
The following will start capturing audio output from all running programs and write base64-encoded chunks of it to your terminal every 200ms:
|
||||
The following will start capturing audio output from all running programs and write raw PCM audio data to your terminal:
|
||||
|
||||
```bash
|
||||
git clone git@github.com:makeusabrew/audiotee.git
|
||||
@@ -24,7 +26,13 @@ cd audiotee
|
||||
swift run
|
||||
```
|
||||
|
||||
If you're not playing audio when you run it, you'll just see packets full of `AAAAA...` - the base64 version of a bunch of zeroes.
|
||||
More usefully, you can redirect the output to a file:
|
||||
|
||||
```bash
|
||||
swift run > captured_audio.pcm
|
||||
```
|
||||
|
||||
If you're not playing audio when you run it, the PCM file will contain silence (zeroes).
|
||||
|
||||
## Build
|
||||
|
||||
@@ -40,14 +48,17 @@ swift build -c release
|
||||
Replace the path below with `.build/<arch>/<target>/audiotee`, e.g. `build/arm64-apple-macosx/release/audiotee` for a release build on Apple Silicon.
|
||||
|
||||
```bash
|
||||
# Auto-detect output format (JSON in terminal, binary when piped)
|
||||
# Write raw PCM audio to stdout (logs go to stderr)
|
||||
./audiotee
|
||||
|
||||
# Always use JSON format (terminal-safe)
|
||||
./audiotee --format json
|
||||
# Redirect audio to a file
|
||||
./audiotee > captured_audio.pcm
|
||||
|
||||
# Always use binary format (pipe-optimised)
|
||||
./audiotee --format binary
|
||||
# Pipe to another program
|
||||
./audiotee | your_audio_processing_tool
|
||||
|
||||
# Redirect logs as well
|
||||
./audiotee > captured_audio.pcm 2> audiotee.log
|
||||
```
|
||||
|
||||
### Audio conversion
|
||||
@@ -94,147 +105,52 @@ Note that trying to include or exclude a PID which isn't currently playing audio
|
||||
./audiotee --chunk-duration 0.1
|
||||
```
|
||||
|
||||
## Output formats
|
||||
## Output
|
||||
|
||||
AudioTee supports two output formats optimised for different use cases:
|
||||
AudioTee writes raw PCM audio data directly to `stdout` in chunks. All logging, metadata, and status information is written to `stderr`, allowing for clean separation of audio data from program output.
|
||||
|
||||
### JSON format (`--format json` or auto in terminal)
|
||||
### Audio format
|
||||
|
||||
JSON messages to stdout, one per line. Audio data is base64-encoded for terminal safety.
|
||||
|
||||
### Binary format (`--format binary` or auto when piped)
|
||||
|
||||
JSON metadata lines followed by raw binary audio data. More efficient for piping to other processes.
|
||||
|
||||
## Protocol
|
||||
|
||||
### Message types
|
||||
|
||||
All messages (except raw binary audio chunks) follow this envelope structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2024-03-21T15:30:45.123Z",
|
||||
"message_type": "...",
|
||||
"data": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
#### 1. Metadata
|
||||
|
||||
Sent once at startup to describe the audio format:
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2024-03-21T15:30:45.123Z",
|
||||
"message_type": "metadata",
|
||||
"data": {
|
||||
"sample_rate": 48000,
|
||||
"channels_per_frame": 1,
|
||||
"bits_per_channel": 32,
|
||||
"is_float": true,
|
||||
"capture_mode": "audio",
|
||||
"device_name": null,
|
||||
"device_uid": null,
|
||||
"encoding": "pcm_f32le"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 2. Stream start
|
||||
|
||||
Indicates audio data will follow:
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2024-03-21T15:30:45.123Z",
|
||||
"message_type": "stream_start",
|
||||
"data": null
|
||||
}
|
||||
```
|
||||
|
||||
#### 3. Audio data
|
||||
|
||||
**JSON format:**
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2024-03-21T15:30:45.123Z",
|
||||
"message_type": "audio",
|
||||
"data": {
|
||||
"timestamp": "2024-03-21T15:30:45.123Z",
|
||||
"duration": 0.2,
|
||||
"peak_amplitude": 0.45,
|
||||
"audio_data": "base64_encoded_raw_audio..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Binary format:**
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2024-03-21T15:30:45.123Z",
|
||||
"message_type": "audio",
|
||||
"data": {
|
||||
"timestamp": "2024-03-21T15:30:45.123Z",
|
||||
"duration": 0.2,
|
||||
"peak_amplitude": 0.45,
|
||||
"audio_length": 9600
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
_Followed immediately by 9600 bytes of raw binary audio data_
|
||||
|
||||
#### 4. Stream stop
|
||||
|
||||
Sent when recording stops:
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2024-03-21T15:30:45.123Z",
|
||||
"message_type": "stream_stop",
|
||||
"data": null
|
||||
}
|
||||
```
|
||||
|
||||
#### 5. Log messages
|
||||
|
||||
Info, error, and debug messages (useful for monitoring):
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2024-03-21T15:30:45.123Z",
|
||||
"message_type": "info",
|
||||
"data": {
|
||||
"message": "Starting AudioTee...",
|
||||
"context": { "output_format": "auto" }
|
||||
}
|
||||
}
|
||||
```
|
||||
- **Format**: Raw PCM audio data
|
||||
- **Channels**: Mono (1 channel)
|
||||
- **Sample rate**: Matches your output device's sample rate by default (configurable)
|
||||
- **Bit depth**: 32-bit float by default, or 16-bit when sample rate conversion is performed
|
||||
- **Endianness**: Little-endian
|
||||
- **Chunk duration**: 200ms by default (configurable)
|
||||
|
||||
### Consuming output
|
||||
|
||||
**JSON format:**
|
||||
The simplest way to consume AudioTee's output is to redirect it to a file:
|
||||
|
||||
1. Parse each line as JSON using the envelope structure
|
||||
2. Use `metadata` message to understand the audio format
|
||||
3. For `audio` messages, decode `audio_data` from base64 to get raw PCM data
|
||||
4. Do something with each chunk of data
|
||||
```bash
|
||||
./audiotee > captured_audio.pcm
|
||||
```
|
||||
|
||||
**Binary format:**
|
||||
You can then process this raw PCM file with tools like:
|
||||
- **FFmpeg**: `ffmpeg -f f32le -ar 48000 -ac 1 -i captured_audio.pcm output.wav`
|
||||
- **SoX**: `sox -t f32 -r 48000 -c 1 captured_audio.pcm output.wav`
|
||||
- **Audacity**: Import as raw data with the appropriate settings
|
||||
|
||||
1. Parse JSON metadata lines using the envelope structure
|
||||
2. Use `metadata` message to understand the audio format
|
||||
3. For `audio` messages, read `audio_length` bytes of raw binary data after the JSON line
|
||||
4. Do something with each chunk of data
|
||||
For real-time processing, pipe the output to your application:
|
||||
|
||||
**Note**: binary is actually a mixed mode; JSON during boot, JSON packet header information preceding each binary chunk.
|
||||
```bash
|
||||
./audiotee | your_audio_processing_tool
|
||||
```
|
||||
|
||||
### Logs and monitoring
|
||||
|
||||
All program logs are written to `stderr` and can be captured separately:
|
||||
|
||||
```bash
|
||||
# Capture audio and logs separately
|
||||
./audiotee > audio.pcm 2> audiotee.log
|
||||
|
||||
# View logs in real-time while capturing audio
|
||||
./audiotee > audio.pcm 2>&1 | grep "AudioTee"
|
||||
```
|
||||
|
||||
## Command Line options
|
||||
|
||||
- `--format, -f`: Output format (`json`, `binary`, `auto`) [default: `auto`]
|
||||
- `--include-processes`: Process IDs to tap (space-separated, empty = all processes)
|
||||
- `--exclude-processes`: Process IDs to exclude (space-separated, empty = none)
|
||||
- `--mute`: Mute processes being tapped
|
||||
|
||||
@@ -189,11 +189,6 @@ class SimpleArgumentParser {
|
||||
throw ArgumentParserError.invalidValue(optionName, value)
|
||||
}
|
||||
return doubleValue as! T
|
||||
} else if type == OutputFormat.self {
|
||||
guard let format = OutputFormat(rawValue: value) else {
|
||||
throw ArgumentParserError.invalidValue(optionName, value)
|
||||
}
|
||||
return format as! T
|
||||
}
|
||||
|
||||
throw ArgumentParserError.invalidValue(optionName, value)
|
||||
|
||||
@@ -2,7 +2,6 @@ import CoreAudio
|
||||
import Foundation
|
||||
|
||||
struct AudioTee {
|
||||
var format: OutputFormat = .auto
|
||||
var includeProcesses: [Int32] = []
|
||||
var excludeProcesses: [Int32] = []
|
||||
var mute: Bool = false
|
||||
@@ -18,11 +17,6 @@ struct AudioTee {
|
||||
discussion: """
|
||||
AudioTee captures system audio using Core Audio taps and streams it as structured output.
|
||||
|
||||
Output formats:
|
||||
• json: Base64-encoded audio in JSON messages (safe for terminals)
|
||||
• binary: Raw binary audio with JSON metadata headers (efficient for pipes)
|
||||
• auto: Automatically choose based on whether stdout is a terminal (default)
|
||||
|
||||
Process filtering:
|
||||
• include-processes: Only tap specified process IDs (empty = all processes)
|
||||
• exclude-processes: Tap all processes except specified ones
|
||||
@@ -30,8 +24,6 @@ struct AudioTee {
|
||||
|
||||
Examples:
|
||||
audiotee # Auto format, tap all processes
|
||||
audiotee --format=json # Always use JSON format
|
||||
audiotee --format=binary # Always use binary format
|
||||
audiotee --sample-rate=16000 # Convert to 16kHz mono for ASR
|
||||
audiotee --sample-rate=8000 # Convert to 8kHz for telephony
|
||||
audiotee --include-processes 1234 # Only tap process 1234
|
||||
@@ -42,7 +34,6 @@ struct AudioTee {
|
||||
)
|
||||
|
||||
// Configure arguments
|
||||
parser.addOption(name: "format", shortName: "f", help: "Output format", defaultValue: "auto")
|
||||
parser.addArrayOption(
|
||||
name: "include-processes",
|
||||
help: "Process IDs to include (space-separated, empty = all processes)")
|
||||
@@ -62,7 +53,6 @@ struct AudioTee {
|
||||
var audioTee = AudioTee()
|
||||
|
||||
// Extract values
|
||||
audioTee.format = try parser.getValue("format", as: OutputFormat.self)
|
||||
audioTee.includeProcesses = try parser.getArrayValue("include-processes", as: Int32.self)
|
||||
audioTee.excludeProcesses = try parser.getArrayValue("exclude-processes", as: Int32.self)
|
||||
audioTee.mute = parser.getFlag("mute")
|
||||
@@ -102,7 +92,6 @@ struct AudioTee {
|
||||
setupSignalHandlers()
|
||||
|
||||
Logger.info("Starting AudioTee...")
|
||||
Logger.debug("Using output format: \(format)")
|
||||
|
||||
// Validate chunk duration
|
||||
guard chunkDuration > 0 && chunkDuration <= 5.0 else {
|
||||
@@ -143,7 +132,7 @@ struct AudioTee {
|
||||
throw ExitCode.failure
|
||||
}
|
||||
|
||||
let outputHandler = createOutputHandler(for: format)
|
||||
let outputHandler = BinaryAudioOutputHandler()
|
||||
let recorder = AudioRecorder(
|
||||
deviceID: deviceID, outputHandler: outputHandler, convertToSampleRate: sampleRate,
|
||||
chunkDuration: chunkDuration)
|
||||
@@ -172,17 +161,6 @@ struct AudioTee {
|
||||
}
|
||||
}
|
||||
|
||||
private func createOutputHandler(for format: OutputFormat) -> AudioOutputHandler {
|
||||
switch format {
|
||||
case .json:
|
||||
return JSONAudioOutputHandler()
|
||||
case .binary:
|
||||
return BinaryAudioOutputHandler()
|
||||
case .auto:
|
||||
return AutoAudioOutputHandler()
|
||||
}
|
||||
}
|
||||
|
||||
private func convertProcessFlags() -> ([Int32], Bool) {
|
||||
if !includeProcesses.isEmpty {
|
||||
// Include specific processes only
|
||||
|
||||
@@ -1,16 +0,0 @@
|
||||
enum OutputFormat: String, CaseIterable {
|
||||
case json = "json"
|
||||
case binary = "binary"
|
||||
case auto = "auto"
|
||||
|
||||
var description: String {
|
||||
switch self {
|
||||
case .json:
|
||||
return "Base64-encoded JSON (terminal-safe)"
|
||||
case .binary:
|
||||
return "Binary with JSON headers (pipe-optimised)"
|
||||
case .auto:
|
||||
return "Auto-detect based on TTY (default)"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -119,7 +119,6 @@ public class AudioFormatConverter {
|
||||
return AudioPacket(
|
||||
timestamp: packet.timestamp,
|
||||
duration: packet.duration,
|
||||
peakAmplitude: packet.peakAmplitude,
|
||||
rawAudioData: outputData
|
||||
)
|
||||
}
|
||||
|
||||
@@ -3,18 +3,15 @@ import Foundation
|
||||
public struct AudioPacket {
|
||||
public let timestamp: Date
|
||||
public let duration: Double
|
||||
public let peakAmplitude: Float // useful for level monitoring
|
||||
public let rawAudioData: Data
|
||||
|
||||
public init(
|
||||
timestamp: Date,
|
||||
duration: Double,
|
||||
peakAmplitude: Float,
|
||||
rawAudioData: Data
|
||||
) {
|
||||
self.timestamp = timestamp
|
||||
self.duration = duration
|
||||
self.peakAmplitude = peakAmplitude
|
||||
self.rawAudioData = rawAudioData
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,31 +0,0 @@
|
||||
import Foundation
|
||||
|
||||
/// Auto-detecting output handler based on TTY
|
||||
public class AutoAudioOutputHandler: AudioOutputHandler {
|
||||
private let handler: AudioOutputHandler
|
||||
|
||||
public init() {
|
||||
// Auto-detect based on whether stdout is a terminal
|
||||
if isatty(STDOUT_FILENO) != 0 {
|
||||
handler = JSONAudioOutputHandler()
|
||||
} else {
|
||||
handler = BinaryAudioOutputHandler()
|
||||
}
|
||||
}
|
||||
|
||||
public func handleAudioPacket(_ packet: AudioPacket) {
|
||||
handler.handleAudioPacket(packet)
|
||||
}
|
||||
|
||||
public func handleMetadata(_ metadata: AudioStreamMetadata) {
|
||||
handler.handleMetadata(metadata)
|
||||
}
|
||||
|
||||
public func handleStreamStart() {
|
||||
handler.handleStreamStart()
|
||||
}
|
||||
|
||||
public func handleStreamStop() {
|
||||
handler.handleStreamStop()
|
||||
}
|
||||
}
|
||||
@@ -5,12 +5,6 @@ public class BinaryAudioOutputHandler: AudioOutputHandler {
|
||||
public init() {}
|
||||
|
||||
public func handleAudioPacket(_ packet: AudioPacket) {
|
||||
// Create metadata without the audio data
|
||||
let metadata = BinaryPacketHeader(from: packet)
|
||||
|
||||
// Write JSON metadata line
|
||||
Logger.writeMessage(.audio, data: metadata)
|
||||
|
||||
// Write raw binary audio data directly to stdout
|
||||
FileHandle.standardOutput.write(packet.rawAudioData)
|
||||
}
|
||||
|
||||
@@ -1,23 +0,0 @@
|
||||
import Foundation
|
||||
|
||||
/// Base64-encoded JSON output (terminal-safe)
|
||||
public class JSONAudioOutputHandler: AudioOutputHandler {
|
||||
public init() {}
|
||||
|
||||
public func handleAudioPacket(_ packet: AudioPacket) {
|
||||
let jsonPacket = JSONAudioPacket(from: packet)
|
||||
Logger.writeMessage(.audio, data: jsonPacket)
|
||||
}
|
||||
|
||||
public func handleMetadata(_ metadata: AudioStreamMetadata) {
|
||||
Logger.writeMessage(.metadata, data: metadata)
|
||||
}
|
||||
|
||||
public func handleStreamStart() {
|
||||
Logger.writeMessage(.streamStart, data: Optional<String>.none)
|
||||
}
|
||||
|
||||
public func handleStreamStop() {
|
||||
Logger.writeMessage(.streamStop, data: Optional<String>.none)
|
||||
}
|
||||
}
|
||||
@@ -1,45 +0,0 @@
|
||||
import Foundation
|
||||
|
||||
/// JSON-serializable version of AudioPacket with base64-encoded audio data
|
||||
public struct JSONAudioPacket: Codable {
|
||||
public let timestamp: Date
|
||||
public let duration: Double
|
||||
public let peakAmplitude: Float
|
||||
public let audioData: String // base64 encoded audio data
|
||||
|
||||
public enum CodingKeys: String, CodingKey {
|
||||
case timestamp
|
||||
case duration
|
||||
case peakAmplitude = "peak_amplitude"
|
||||
case audioData = "audio_data"
|
||||
}
|
||||
|
||||
public init(from packet: AudioPacket) {
|
||||
self.timestamp = packet.timestamp
|
||||
self.duration = packet.duration
|
||||
self.peakAmplitude = packet.peakAmplitude
|
||||
self.audioData = packet.rawAudioData.base64EncodedString()
|
||||
}
|
||||
}
|
||||
|
||||
/// Metadata-only packet for binary output (without base64 audio data)
|
||||
public struct BinaryPacketHeader: Codable {
|
||||
public let timestamp: Date
|
||||
public let duration: Double
|
||||
public let peakAmplitude: Float
|
||||
public let audioLength: Int // Length of raw audio data in bytes
|
||||
|
||||
public enum CodingKeys: String, CodingKey {
|
||||
case timestamp
|
||||
case duration
|
||||
case peakAmplitude = "peak_amplitude"
|
||||
case audioLength = "audio_length"
|
||||
}
|
||||
|
||||
public init(from packet: AudioPacket) {
|
||||
self.timestamp = packet.timestamp
|
||||
self.duration = packet.duration
|
||||
self.peakAmplitude = packet.peakAmplitude
|
||||
self.audioLength = packet.rawAudioData.count
|
||||
}
|
||||
}
|
||||
@@ -24,8 +24,8 @@ public class Logger {
|
||||
let message = Message(type: type, data: data)
|
||||
do {
|
||||
let jsonData = try jsonEncoder.encode(message)
|
||||
FileHandle.standardOutput.write(jsonData)
|
||||
FileHandle.standardOutput.write("\n".data(using: .utf8)!)
|
||||
FileHandle.standardError.write(jsonData)
|
||||
FileHandle.standardError.write("\n".data(using: .utf8)!)
|
||||
} catch {
|
||||
// TODO: handle at some point
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user