| config | s | c |
|---|
+-+-+-+-+-+-+-+-+ `
The top 5 bits (config) map to an internal lookup table that specifies one of three operating modes, alongside bandwidth and frame duration. Bits 6 and 7 (c) indicate whether the packet contains one, two, or an arbitrary number of frames.
The TOC Configuration Space
The 32 possible values of config split the operational space into distinct execution paths:
1. SILK-only Mode (Configs 0–11): Covers narrowband (8000 Hz sample reference), mediumband (12000 Hz), and wideband (16000 Hz). Frame sizes are typically 10 ms, 20 ms, 40 ms, or 60 ms. The MDCT engine is completely powered down. 2. Hybrid Mode (Configs 12–15): Covers super-wideband (24000 Hz) and fullband (48000 Hz). Frame durations are restricted strictly to 10 ms and 20 ms. The input signal is split via a quadrature mirror filter (QMF); the lower band is processed by SILK, while the upper band is processed by CELT. 3. CELT-only Mode (Configs 16–31): Covers narrowband through fullband (up to 48000 Hz). Supported frame sizes drop down to 2.5 ms and 5 ms, extending up to 20 ms. The linear predictive state is bypassed.
If your telemetry shows an encoder frequently flipping between Config 11 (SILK-only wideband, 20 ms) and Config 15 (Hybrid fullband, 20 ms), the application layer is modifying target bitrates across the threshold where CELT engagement becomes computationally viable.
Dynamic Switching and Asymmetric Frames
The decoder must instantiate both decoders if it advertises super-wideband or fullband support. When an encoder switches from SILK to CELT, it does not reinitialize memory. SILK maintains its historical synthesis filter states, and CELT retains its overlap-and-add buffers.
Because CELT uses time-domain aliasing cancellation (TDAC), switching into CELT requires an overlap window. The specification defines an explicit transition step where CELT generates a short windowed burst to cancel the aliasing of the initial frame. If an implementation violates the TOC state transition logic, the result is an audible click caused by non-cancelled time-domain aliasing at the frame boundary.
---
What does sILK: Predictive Source Modeling for Speech actually mean?
SILK assumes the input signal corresponds to the human vocal tract. It relies on an Auto-Regressive (AR) filter model driven by either an excitation signal (unvoiced sounds) or a periodic impulse train (voiced sounds).
Vocal Tract Synthesis and Pitch Filtering
SILK processes audio in internal subframes of 5 ms. For a standard 20 ms audio frame, it calculates four sets of parameters. It begins by performing linear predictive analysis to generate LPC coefficients: an 8th-to-16th order filter that tracks the spectral envelope (formants). For voiced segments, SILK applies a Long-Term Prediction (LTP) filter—a 5-tap pitch predictor that isolates harmonic periodicity.
The residual error left after LPC and LTP analysis is quantized using a variable-order vector quantizer and arithmetic entropy coding. At bitrates like 12 kbps to 20 kbps, SILK excels because it does not attempt to reproduce the original audio waveform with mathematical precision. Instead, it transmits the minimal parameters needed by the decoder's synthesis filter to construct an acoustically plausible facsimile of the human voice.
If you feed non-speech content into SILK—such as a piano chord, background crowd noise, or complex synthesizer waveforms—the LPC filter fails to isolate discrete formants, the LTP analysis fails to lock onto a single pitch lag, and the residual energy overwhelms the low-bitrate entropy coder. The result is severe "swishing," hollow phasing, and unnatural modulation.
In-Band Forward Error Correction
SILK includes a native, non-standardized mechanism for resilience: Low Bitrate Redundancy (LBRR). When enabled via the useinbandfec=1 SDP parameter and activated by the application, SILK encodes a heavily quantized, lower-bitrate copy of frame $N-1$ into the bitstream of frame $N$.
If packet $N-1$ is lost in transit, the decoder detects the sequence number gap, inspects packet $N$, pulls out the LBRR payload, and synthesizes the missing audio. This reduces loss footprint without doubling transport packet overhead. However, LBRR only protects the SILK band; in Hybrid mode, the CELT layer contains no LBRR data.
---
CELT: Transform Energy Coding for Complex Signals
CELT abandons the physiological model entirely. It is a pure frequency-domain transform codec designed to eliminate delay. Where SILK requires a 5 ms lookahead for its analysis window plus additional framing delay, CELT operates with algorithmic delays as low as 2.5 ms.
Energy-Shape Separation and Algebraic Quantization
CELT computes a modified discrete cosine transform (MDCT) over critically sampled windows. The spectrum is divided into 21 critical bands matching the Bark scale. CELT's fundamental operational rule is the absolute separation of band energy from band shape:
1. Band Energy: The root-mean-square (RMS) energy of each of the 21 bands is calculated, transformed to a log scale, and quantized coarsely. This establishes the rough spectral tilt. 2. Spherical Vector Quantization: The remaining detailed frequency bins within each band form a unit vector representing "shape." CELT quantizes this vector by projecting it onto the surface of an $N$-dimensional sphere using a Pyramid Vector Quantizer (PVQ).
This approach prevents the traditional failure mode of transform codecs at low bitrates: "birdies" and fluctuating musical noise caused by transform bins randomly dropping to absolute zero. CELT never quantizes a band to absolute silence if there is energy allocated to it; it scales the noise floor to preserve the envelope.
At high bitrates (e.g., 64 kbps to 128 kbps per channel), CELT delivers full-fidelity audio transparency across complex musical signals that would completely break an LPC coder.
Crossover Mechanics in Hybrid Mode
In Hybrid mode, the input is split at an 8000 Hz crossover point. SILK encodes the lower spectrum (0 to 8000 Hz) using wideband LPC modeling. CELT encodes the upper spectrum (8000 Hz to 20000/24000 Hz) using its MDCT bands.
This introduces phase-alignment challenges. SILK’s IIR and FIR filters impart non-linear phase delays to the lower spectrum. CELT’s MDCT introduces linear phase delay through its framing window. The hybrid decoder runs an internal delay-compensation line on the CELT side to align the high-frequency packet data with the output of the SILK synthesis filter. When writing a custom pipeline, tampering with the sample clocking between these paths will destroy the transient cohesion of wideband speech.
---
Network Degradation and Packet Loss
When an RTP stream experiences severe network impairment, the structural differences between SILK and CELT manifest as totally distinct failure modes.
SILK relies entirely on continuity of state. The LPC synthesis filter maintains an ongoing auto-regressive memory, and the LTP filter relies on a historical pitch buffer spanning up to 18 ms. When a packet is lost, SILK’s Packet Loss Concealment (PLC) must extrapolate these filters forward in time. Under a single 20 ms loss, the pitch filter holds its track, merely drifting slightly. If loss bursts exceed 60 ms to 80 ms, or if jitter forces the jitter buffer to starve repeatedly, the LPC filter parameters drift out of alignment with the actual acoustic signal. When valid packets resume, the decoder's filter state does not match the incoming residual signal, generating high-amplitude pops or muffled audio while the state converges.
CELT fails differently. Because CELT does not use an auto-regressive vocal tract model, it does not suffer from long-term filter state divergence. Its PLC repeats the previous frame’s MDCT band energies while randomizing the vector phases. Under 15% random packet loss, CELT produces metallic, robotic ringing as it continually fades and regenerates synthetic phase data, but it resynchronizes cleanly on the first uncorrupted packet.
In Hybrid mode, network loss degrades both engines simultaneously. A lost frame triggers SILK PLC on the low-band and CELT PLC on the high-band. If the packet loss concealment algorithms diverge in their energy attenuation curves—SILK typically decays energy faster than CELT—the audio collapses into an unbalanced, high-frequency hiss.
---
Practical Rules for System Tuning
Configuring an Opus-based transport pipeline requires understanding how libopus routes decisions between these engines. The implementation is sensitive to your operational constraints:
* Do not force CELT-only mode for low-bitrate speech. Restricting an encoder to CELT to minimize latency (e.g., forcing 5 ms frames at 16 kbps) will cause catastrophic degradation in voice comprehension. CELT requires at least 32 kbps to encode intelligible wideband speech reliably. If latency allows a 20 ms frame, let SILK handle it. * Check bitrate boundaries for Hybrid mode. The reference encoder usually engages Hybrid mode only when the configured bitrate exceeds approximately 24 kbps to 32 kbps for super-wideband signals. Below that threshold, the encoder silently drops high-frequency data and defaults to SILK-only wideband. (I would check the exact version behavior of your underlying library, as dynamic bitrate thresholds have changed slightly between libopus 1.1 and 1.5). * Evaluate CPU constraints across modes. SILK’s linear prediction, pitch search, and noise shaping filters consume far more CPU cycles per millisecond of audio than CELT’s unrolled FFT-based MDCT. If you are multiplexing 500 channels on a single media server, pinning channels to CELT-only at fixed bitrates will drastically reduce compute load compared to SILK or Hybrid processing, at the cost of higher network bandwidth.
The practical rule is simple: treat SILK as an acoustic model that requires stable filter state, and treat CELT as an energy-preserving transform that trades bandwidth efficiency for low latency. Never debug Opus audio artifacts without first reading the TOC byte to verify which engine actually produced the failing sound.
Terms used in this note
A codec is an algorithm that compresses audio for transmission. A jitter buffer is a queue that absorbs variable arrival times before playback. Packet loss concealment is the substitution a decoder performs when a frame never arrives.
- Sample rate — how many times per second the signal is measured
- Frame size — the block of samples a codec encodes at once
- Lookahead — the delay a codec needs before it can encode
- Bitrate — bits per second the stream consumes
- Jitter — variation in packet arrival time
- Playout delay — total time from capture to speaker
- Concealment — what the decoder invents for a missing frame
- Transcoding — decoding and re-encoding between formats
- Comfort noise — synthetic background added during silence
- RTP — the transport that carries real-time media
- RTCP — the control protocol reporting on an RTP session
- Narrowband — speech-band audio, roughly 300 to 3400 Hz