Edge0 has released Audio8-TTS-Realistic, a 601M-parameter open-weight text-to-speech model for zero-shot voice cloning under Apache 2.0, supporting 23 languages—12 more than its predecessor. The dual-autoregressive architecture employs a Qwen2.5 0.5B slow branch (24 layers, width 896, ~21.5 fps) predicting text tokens and the first audio codebook, followed by a shallow 4-layer fast branch generating the remaining nine codebooks in parallel, operating within a 2,048-token context window. Training proceeds in three stages: multilingual supervised fine-tuning, GRPO with a 0.7 WER / 0.3 SIM reward, and GRPO guided by a learned realism critic. The critic, a 3M-parameter head on a frozen WavLM-large encoder, was trained on a quality ladder from six open-source TTS systems ranked via blind listening, incorporating synthetic pause compression and an identity-erasure loss to prevent fingerprinting; a duration penalty beyond 1.35x reference length prevents gaming. Self-reported benchmarks show 4.86% average CV3 error across nine languages (SIM 0.661), outperforming Qwen3-TTS-1.7B (7.73%) and VoxCPM2 (5.35%), and 5.89% on MiniMax across 24 languages (SIM 0.754). Seed-TTS results are 1.46% WER (EN) and 1.03% CER (ZH). Higgs TTS 3 (~4B) leads with 4.41% CV3 and 2.74% MiniMax. Low-resource languages remain weak: Thai hits 48.57% WER (CV3), Cantonese 31.35% CER (MiniMax). The realism stage, trained only on Chinese/English, still cut aggregate error on 13 lower-resource languages from 22.53% to 20.77% and raised SIM from 0.637 to 0.679. Deployment requires PyTorch 2.5+, Transformers 4.57+, and trust_remote_code; an SGLang Omni adapter enables OpenAI-compatible serving, though throughput and streaming are unbenchmarked. Peak memory is 1.1–1.2 GiB on Apple M2 CPU (INT4 ONNX); CUDA GPU recommended.
Read original
marktechpost