> ## Documentation Index
> Fetch the complete documentation index at: https://daily-docs-pr-5356.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# AudioBufferProcessor
> Process and buffer audio frames from conversations with flexible event handling
## Overview
The `AudioBufferProcessor` captures and buffers audio frames from both input (user) and output (bot) sources during conversations. It provides synchronized audio streams with configurable sample rates, supports both mono and stereo output, and offers flexible event handlers for various audio processing workflows.
## Constructor
```python theme={null}
AudioBufferProcessor(
sample_rate=None,
num_channels=1,
buffer_size=0,
enable_turn_audio=False,
auto_start_recording=False,
**kwargs
)
```
### Parameters
The desired output sample rate in Hz. If `None`, uses the transport's sample
rate from the `StartFrame`.
Number of output audio channels:
* `1`: Mono output (user and bot audio are mixed together)
* `2`: Stereo output (user audio on left channel, bot audio on right channel)
Buffer size in bytes that triggers audio data events:
* `0`: Events only trigger when recording stops
* `>0`: Events trigger whenever buffer reaches this size (useful for chunked processing)
Whether to enable per-turn audio event handlers (`on_user_turn_audio`,
`on_user_turn_audio_data`, `on_bot_turn_audio`, `on_bot_turn_audio_data`).
Whether to start recording automatically when the pipeline starts, without
requiring a call to `start_recording()` or an
`AudioBufferStartRecordingFrame`.
## Properties
### sample\_rate
```python theme={null}
@property
def sample_rate(self) -> int
```
The current sample rate of the audio processor in Hz.
### num\_channels
```python theme={null}
@property
def num_channels(self) -> int
```
The number of channels in the audio output (1 for mono, 2 for stereo).
## Methods
### start\_recording()
```python theme={null}
async def start_recording()
```
Start recording audio from both user and bot sources. Initializes recording state, resets audio buffers, and triggers the `on_recording_started` event. Does nothing when recording is already active.
### stop\_recording()
```python theme={null}
async def stop_recording()
```
Stop recording and trigger final audio data handlers with any remaining buffered audio, then triggers the `on_recording_stopped` event. Does nothing when recording is not active.
### has\_audio()
```python theme={null}
def has_audio() -> bool
```
Check if both user and bot audio buffers contain data.
**Returns:** `True` if both buffers contain audio data.
## TurnAudioData
The `TurnAudioData` dataclass holds one speaker's audio for a complete conversation turn:
```python theme={null}
@dataclass
class TurnAudioData:
turn_number: int # The turn this audio belongs to
audio: bytes # Complete turn audio as raw PCM (pauses removed)
sample_rate: int # Sample rate in Hz
num_channels: int # Number of channels
```
This dataclass is passed to the `on_user_turn_audio` and `on_bot_turn_audio` event handlers. Unlike the deprecated per-run events, these handlers fire once per turn with the complete audio, and include the turn number for tracking.
## Event Handlers
The processor supports multiple event handlers for different audio processing workflows. Register handlers using the `@processor.event_handler()` decorator.
### on\_audio\_data
Triggered when `buffer_size` is reached or recording stops, providing merged audio.
```python theme={null}
@audiobuffer.event_handler("on_audio_data")
async def on_audio_data(buffer, audio: bytes, sample_rate: int, num_channels: int):
# Handle merged audio data
pass
```
**Parameters:**
* `buffer`: The AudioBufferProcessor instance
* `audio`: Merged audio data (format depends on `num_channels` setting)
* `sample_rate`: Sample rate in Hz
* `num_channels`: Number of channels (1 or 2)
### on\_track\_audio\_data
Triggered alongside `on_audio_data`, providing separate user and bot audio tracks.
```python theme={null}
@audiobuffer.event_handler("on_track_audio_data")
async def on_track_audio_data(buffer, user_audio: bytes, bot_audio: bytes,
sample_rate: int, num_channels: int):
# Handle separate audio tracks
pass
```
**Parameters:**
* `buffer`: The AudioBufferProcessor instance
* `user_audio`: Raw user audio bytes (always mono)
* `bot_audio`: Raw bot audio bytes (always mono)
* `sample_rate`: Sample rate in Hz
* `num_channels`: Always 1 for individual tracks
### on\_user\_turn\_audio
Triggered when a turn ends, providing everything the user said during it. Requires `enable_turn_audio=True` and `enable_turn_tracking=True` on the PipelineWorker.
```python theme={null}
@audiobuffer.event_handler("on_user_turn_audio")
async def on_user_turn_audio(buffer, turn_data: TurnAudioData):
# Handle complete user turn audio
print(f"Turn {turn_data.turn_number}: {len(turn_data.audio)} bytes")
```
**Parameters:**
* `buffer`: The AudioBufferProcessor instance
* `turn_data`: A `TurnAudioData` object containing the turn number, complete audio (with pauses removed), sample rate, and number of channels
### on\_bot\_turn\_audio
Triggered when a turn ends, providing everything the bot said during it. Requires `enable_turn_audio=True` and `enable_turn_tracking=True` on the PipelineWorker.
```python theme={null}
@audiobuffer.event_handler("on_bot_turn_audio")
async def on_bot_turn_audio(buffer, turn_data: TurnAudioData):
# Handle complete bot turn audio
print(f"Turn {turn_data.turn_number}: {len(turn_data.audio)} bytes")
```
**Parameters:**
* `buffer`: The AudioBufferProcessor instance
* `turn_data`: A `TurnAudioData` object containing the turn number, complete audio (with pauses removed), sample rate, and number of channels
### on\_user\_turn\_audio\_data
**Deprecated since 1.8.0**. Use `on_user_turn_audio` instead. Will be removed
in 2.0.0.
Triggered when the user stops speaking, providing that run of speech. Requires `enable_turn_audio=True`.
```python theme={null}
@audiobuffer.event_handler("on_user_turn_audio_data")
async def on_user_turn_audio_data(buffer, audio: bytes, sample_rate: int, num_channels: int):
# Handle user turn audio
pass
```
**Parameters:**
* `buffer`: The AudioBufferProcessor instance
* `audio`: Audio data from the user's speaking turn
* `sample_rate`: Sample rate in Hz
* `num_channels`: Always 1 (mono)
### on\_bot\_turn\_audio\_data
**Deprecated since 1.8.0**. Use `on_bot_turn_audio` instead. Will be removed
in 2.0.0.
Triggered when the bot stops speaking, providing that run of speech. Requires `enable_turn_audio=True`.
```python theme={null}
@audiobuffer.event_handler("on_bot_turn_audio_data")
async def on_bot_turn_audio_data(buffer, audio: bytes, sample_rate: int, num_channels: int):
# Handle bot turn audio
pass
```
**Parameters:**
* `buffer`: The AudioBufferProcessor instance
* `audio`: Audio data from the bot's speaking turn
* `sample_rate`: Sample rate in Hz
* `num_channels`: Always 1 (mono)
### on\_recording\_started
Triggered when recording starts (state transitions to active).
```python theme={null}
@audiobuffer.event_handler("on_recording_started")
async def on_recording_started(buffer):
# Handle recording start
pass
```
**Parameters:**
* `buffer`: The AudioBufferProcessor instance
### on\_recording\_stopped
Triggered after recording stops and the final buffered audio has been emitted.
```python theme={null}
@audiobuffer.event_handler("on_recording_stopped")
async def on_recording_stopped(buffer):
# Handle recording stop
pass
```
**Parameters:**
* `buffer`: The AudioBufferProcessor instance
## Audio Processing Features
* **Automatic resampling**: Converts incoming audio to the specified sample rate
* **Buffer synchronization**: Aligns user and bot audio streams temporally
* **Silence insertion**: Fills gaps in non-continuous audio streams to maintain timing
* **Turn tracking**: Monitors speaking turns when `enable_turn_audio=True`
## Recording Control
Recording can be controlled in three ways:
1. **Automatically on pipeline start**: Set `auto_start_recording=True` in the constructor
2. **Programmatically**: Call `await processor.start_recording()` and `await processor.stop_recording()`
3. **Via control frames**: Push `AudioBufferStartRecordingFrame` and `AudioBufferStopRecordingFrame` through the pipeline
```python theme={null}
from pipecat.frames.frames import AudioBufferStartRecordingFrame, AudioBufferStopRecordingFrame
# Start recording via frame
await worker.queue_frames([AudioBufferStartRecordingFrame()])
# Stop recording via frame
await worker.queue_frames([AudioBufferStopRecordingFrame()])
```
## Working with Captured Audio
Event handlers receive raw PCM audio as `bytes`. To save or upload this audio, wrap it in a WAV container using `pcm_to_wav()` from `pipecat.audio.utils`:
```python theme={null}
from pipecat.audio.utils import pcm_to_wav
@audiobuffer.event_handler("on_audio_data")
async def on_audio_data(buffer, audio: bytes, sample_rate: int, num_channels: int):
# Convert PCM to WAV format
wav_data = pcm_to_wav(audio, sample_rate, num_channels)
# Save to file
with open("recording.wav", "wb") as f:
f.write(wav_data)
# Or upload to a service
# await upload_to_storage(wav_data)
```
The `pcm_to_wav()` function accepts `bytes`, `bytearray`, or `memoryview`, matching the types used in Pipecat pipelines.
## Integration Notes
### STT Audio Passthrough
If using an STT service in your pipeline, enable audio passthrough to make audio available to the AudioBufferProcessor:
```python theme={null}
stt = DeepgramSTTService(
api_key=os.getenv("DEEPGRAM_API_KEY"),
audio_passthrough=True,
)
```
`audio_passthrough` is enabled by default.
### Pipeline Placement
Add the AudioBufferProcessor after `transport.output()` to capture both user and bot audio:
```python theme={null}
pipeline = Pipeline([
transport.input(),
# ... other processors ...
transport.output(),
audiobuffer, # Place after audio output
# ... remaining processors ...
])
```