Baseline

Last Updated: October 22, 2026

Table of Contents

1. Overview
2. VoCodec architecture
3. Track 1 and Track 2 models
4. Baseline checkpoints
5. DAC-style residual vector quantizer
6. Analytical model-design sheets and resource accounting
7. Streaming design and latency checker
8. Objective results
   1. 1 kbps
   2. 6 kbps
   3. 12 kbps
   4. 24 kbps
9. Reference

Overview

LRAC 2.0 provides open-source baseline models for both challenge tracks. The models are adapted from VoCodec [1], a lightweight, low-bitrate time-frequency neural speech codec, and have been revised for the operating points and resource constraints of LRAC 2.0.

The complete LRAC 2.0 baseline recipe is available in ESPnet. It contains the Track 1 and Track 2 configurations, training and evaluation recipes, analytical model-design sheets, and supporting validation tools. The baseline training data can be reproduced with the official LRAC 2.0 data-preparation pipeline.

The baseline is intended to provide participants with a reproducible reference system and worked resource-accounting examples. Participants are not required to use the baseline architecture. The official Rules remain authoritative for all submission requirements and constraints.

VoCodec architecture

VoCodec operates on 24-kHz audio in the time-frequency domain. Its analysis path uses a 720-sample (30-ms) STFT window and a 240-sample (10-ms) hop to produce a 100-Hz representation. Causal ConvNeXt blocks and a PosNet module, which combines residual convolutions with multi-head self-attention, transform the STFT features into a latent sequence. A residual vector quantizer (RVQ) compresses that sequence, after which a mirrored neural decoder and inverse STFT reconstruct the waveform.

Both baseline models use four-head self-attention with a bounded 300-frame sliding window, corresponding to approximately three seconds of history at 100 Hz. The attention has zero lookahead and uses rotary position embeddings (RoPE). Bounding the context prevents attention memory and compute from growing with utterance duration. The decoder PosNet uses residual groups [1, 1], [1, 1], [2, 2], and [2, 2] in its four blocks.

Training uses a multi-scale STFT discriminator together with adversarial, feature-matching, multi-scale mel-spectrogram, commitment, and codebook losses. The LRAC 2.0 implementation retains the main VoCodec design while increasing the RVQ depth from the original paper’s six layers to 24 and adjusting the model dimensions to support the new challenge operating points and Track 2 parameter limit.

Track 1 and Track 2 models

Track 1 – Transparency uses the larger baseline configuration. It has a 176-dimensional model representation, an encoder ConvNeXt expansion dimension of 216, and a decoder expansion dimension of 200. Track 1 has no parameter cap or memory-bandwidth penalty, but it must satisfy the common bitrate, compute, and latency limits.

Track 2 – Memory-Efficient Transparency uses a smaller 112-dimensional representation and a 128-dimensional ConvNeXt expansion in both the encoder and decoder. This configuration is designed to remain below the Track 2 cap of 1.5 million under the challenge parameter-accounting convention, while satisfying the common constraints. Track 2 is also subject to the theoretical memory-bandwidth penalty defined in the challenge rules.

Setting	Track 1	Track 2
Model dimension	176	112
Encoder / decoder expansion	216 / 200	128 / 128
Attention heads	4	4
Attention history	300 frames	300 frames
Attention lookahead	0 frames	0 frames
RoPE rotary dimension	44	28
RVQ layers	24	24
Native target bitrates	1, 6, 12, 24 kbps	1, 6, 12, 24 kbps
Trainable parameters	3,179,362	1,478,626

The analytical calculations included with the recipe give the following resource values:

Quantity	Track 1	Track 2
Transmit compute	384.286 MFLOPS	190.748 MFLOPS
Receive compute	294.276 MFLOPS	132.073 MFLOPS
Total latency (buffering + algorithmic)	30 ms	30 ms
Memory bandwidth	694.883 MB/s	333.536 MB/s
Track 2 bandwidth penalty	Not applicable	9.502

Under the assumptions documented in the baseline analytical model-design sheets, both models meet the challenge compute and latency limits, and the Track 2 model also meets the 1.5-million-value cap.

Baseline checkpoints

The final epoch-1700 baseline checkpoints and their license files are available from the public ESPnet-based baseline repository:

• Track 1 baseline model directory
• Track 2 baseline model directory

Each directory contains the trained model checkpoint and its accompanying license file.

DAC-style residual vector quantizer

The baseline uses a DAC-style residual vector quantizer. Codebook embeddings are learned through an explicit codebook loss, while a straight-through estimator and commitment loss train the encoder path.

Nearest-neighbor lookup may compare normalized encoder and codebook vectors, while reconstruction uses the selected unnormalized embedding. Learned input and output projections are applied independently around each RVQ layer in the baseline. At inference, a requested bitrate deterministically selects a prefix of the RVQ stack, so the lower-bitrate representations are nested within the higher-bitrate representation.

The LRAC 2.0 RVQ contains 24 layers with 1,024 entries per codebook. At a 100-Hz frame rate, each active layer contributes 1 kbps. Prefixes of 1, 6, 12, and 24 layers therefore provide the four native challenge operating points of 1, 6, 12, and 24 kbps.

LRAC 2.0 also evaluates a 32-kbps cap. Because each listed bitrate is an upper limit, the final baseline serves this cap using its 24-layer, 24-kbps operating point and reuses the 24-kbps reconstructions for the 32-kbps evaluation. During initial baseline development, training an additional 32-layer, 32-kbps operating point did not yield a meaningful improvement over 24 kbps, so it was omitted from the final configuration.

During training, the baseline uses a quantizer-dropout probability of 0.5 and samples only from the target prefixes [1, 6, 12, 24]. Half of the examples use all 24 RVQ layers. The other half sample uniformly from the four target operating points; with a sufficiently large batch, the expected proportions are therefore 12.5% each at 1, 6, and 12 kbps and 62.5% at 24 kbps. Intermediate prefix lengths are not sampled by this configuration. Quantizer dropout is disabled during evaluation, where the requested bitrate selects a deterministic prefix.

Additional model and quantizer-training details are available in the recipe’s DAC-style RVQ documentation.

Analytical model-design sheets and resource accounting

Every challenge entry must include a completed analytical model-design sheet with hyperparameters and layer-by-layer analytical calculations of field of view, compute complexity, and latency. Track 2 entries must additionally include memory-bandwidth calculations. See the Rules for the complete submission requirements.

The baseline recipe includes the following completed analytical model-design sheets as worked examples:

• Track 1 baseline analytical model-design sheet
• Track 2 baseline analytical model-design sheet

The sheets report calculations per layer or block and summarize parameters, persistent model values, transmit and receive compute, field of view, latency, and memory bandwidth. They are generated automatically from a shared analytical calculator and sheet generator, with separate generation entry points for Track 1 and Track 2.

These sheets and scripts are specific to the baseline VoCodec design. Their formulas encode assumptions about its STFT and inverse STFT, ConvNeXt and PosNet blocks, attention, DAC-style RVQ, numerical precision, and cached state. They are not general calculators or submission templates for arbitrary codec architectures. Participants must provide calculations that correctly represent their own submitted system.

Streaming design and latency checker

The VoCodec convolutions are causal, and attention uses three seconds of past context with no attention lookahead. For the centre-aligned STFT used by the baseline, total latency consists of a 10-ms frame-hop buffer, 10 ms of STFT analysis lookahead, and 10 ms of inverse-STFT synthesis lookahead, for a total of 30 ms. The retained overlap and attention histories are state, rather than additional lookahead.

The recipe includes a sample-based streamability checker. It perturbs input audio after randomly selected cutoff points and checks that earlier output remains unchanged outside the declared future-dependency allowance. Usage, tolerance selection, and limitations are described in the streaming and streamability section of the recipe documentation.

The checker is provided only as a helper and may have limitations. Its result is empirical and depends on the tested inputs, cutoff positions, numerical precision, device, and comparison tolerances. It does not verify compute, bitrate, parameter count, memory bandwidth, packetization, scheduling, or every possible runtime behavior. Each participant is responsible for verifying that the submitted codec satisfies all official challenge requirements, including the latency and streamability rules.

Objective results

The tables below report mean objective scores for the final epoch-1700 Track 1 and Track 2 checkpoints at their four native operating bitrates. Results are shown for the public LRAC 1.0 Open Test Set and the now-public LRAC 1.0 Official Test Set, which was the blind test set during LRAC 1.0. Scores are rounded to two decimal places. The abbreviated column labels are PESQ, reference-based SCOREQ (SCOREQ-ref), UTMOS, SHEET-SSQA (SHEET), and Audiobox Aesthetics CE (ABox CE). The 32-kbps challenge cap reuses the 24-kbps baseline operating point and therefore has the same results as the 24-kbps tables.

1 kbps

Dataset	Condition	Track 1					Track 2
		PESQ	SCOREQ-ref	UTMOS	SHEET	ABox CE	PESQ	SCOREQ-ref	UTMOS	SHEET	ABox CE
LRAC 1.0 Open Test Set	Clean	1.29	0.83	1.88	2.50	4.40	1.19	1.04	1.56	2.16	4.00
	Noisy	1.26	0.72	1.64	2.27	3.75	1.17	0.86	1.44	2.06	3.46
	Reverberant	1.22	0.65	1.35	2.23	3.29	1.17	0.70	1.31	2.23	3.15
LRAC 1.0 Official Test Set	Clean	1.29	0.75	2.03	2.90	4.28	1.19	1.00	1.66	2.39	3.85
	Real-world	1.28	0.66	1.58	2.58	3.59	1.22	0.81	1.42	2.33	3.26
	Simultaneous talkers	1.15	0.55	1.27	2.10	3.13	1.11	0.63	1.26	2.24	3.03

6 kbps

Dataset	Condition	Track 1					Track 2
		PESQ	SCOREQ-ref	UTMOS	SHEET	ABox CE	PESQ	SCOREQ-ref	UTMOS	SHEET	ABox CE
LRAC 1.0 Open Test Set	Clean	3.37	0.17	3.73	4.22	5.62	3.10	0.23	3.53	4.07	5.52
	Noisy	3.03	0.19	3.09	3.42	4.72	2.77	0.24	2.94	3.30	4.64
	Reverberant	2.18	0.23	1.55	2.94	4.10	2.07	0.29	1.53	2.88	4.02
LRAC 1.0 Official Test Set	Clean	3.20	0.16	3.66	4.46	5.64	2.82	0.24	3.29	4.33	5.51
	Real-world	2.87	0.18	2.73	3.56	4.61	2.61	0.23	2.59	3.43	4.51
	Simultaneous talkers	2.03	0.28	1.34	2.42	4.02	1.90	0.33	1.32	2.44	3.99

12 kbps

Dataset	Condition	Track 1					Track 2
		PESQ	SCOREQ-ref	UTMOS	SHEET	ABox CE	PESQ	SCOREQ-ref	UTMOS	SHEET	ABox CE
LRAC 1.0 Open Test Set	Clean	3.83	0.11	3.85	4.31	5.71	3.47	0.16	3.71	4.23	5.65
	Noisy	3.59	0.15	3.19	3.51	4.84	3.17	0.19	3.08	3.42	4.78
	Reverberant	2.62	0.19	1.56	3.04	4.22	2.34	0.24	1.58	2.97	4.14
LRAC 1.0 Official Test Set	Clean	3.66	0.11	3.77	4.51	5.75	3.25	0.16	3.53	4.43	5.65
	Real-world	3.40	0.14	2.82	3.66	4.73	3.02	0.17	2.75	3.58	4.65
	Simultaneous talkers	2.54	0.23	1.34	2.52	4.14	2.23	0.28	1.33	2.45	4.06

24 kbps

Dataset	Condition	Track 1					Track 2
		PESQ	SCOREQ-ref	UTMOS	SHEET	ABox CE	PESQ	SCOREQ-ref	UTMOS	SHEET	ABox CE
LRAC 1.0 Open Test Set	Clean	3.94	0.10	3.89	4.33	5.74	3.57	0.14	3.76	4.26	5.68
	Noisy	3.73	0.14	3.21	3.52	4.88	3.25	0.18	3.11	3.44	4.81
	Reverberant	2.75	0.17	1.56	3.08	4.27	2.41	0.23	1.59	2.99	4.17
LRAC 1.0 Official Test Set	Clean	3.81	0.10	3.81	4.52	5.78	3.35	0.15	3.57	4.45	5.69
	Real-world	3.54	0.13	2.85	3.70	4.78	3.12	0.16	2.78	3.61	4.67
	Simultaneous talkers	2.72	0.22	1.34	2.51	4.17	2.30	0.26	1.32	2.47	4.09

Reference

[1] L. Yang, R. Hu, Y. Xu, and J. Lu, “VoCodec: An Efficient Lightweight Low-Bitrate Speech Codec,” 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 2026, pp. 21982–21986, doi: 10.1109/ICASSP55912.2026.11464230.

Back to top