Production-grade voice AI requires more than speech collection

Kyutai was pushing the boundaries of open voice and multimodal AI, including real-time conversational models and advanced speech generation.As the lab moved toward emotionally expressive text-to-speech, available datasets presented a limitation. Basic speech databases could support intelligibility, but they often lacked the emotional nuance, conversational variation, and multilingual consistency needed to produce natural-sounding voices.

Building the datasets introduced significant operational complexity. Speakers needed to maintain consistent identities and recording quality across sessions while delivering varied emotions and speaking styles. Transcripts, emotion labels, non-speech events, pronunciation, and metadata also had to remain accurate across languages and recording batches. To scale, Kyutai needed a governed production process capable of creating expressive multilingual data consistently.

Creating a unified production pipeline for multilingual voice data

Kyutai partnered with Uber AI Solutions to operationalise voice data creation as one coordinated system rather than managing recording, transcription, annotation and QA independently.

UAIS coordinated multilingual recording programmes across French, Brazilian Portuguese, European and Latin American Spanish and German. Voice talent received structured guidance designed to expand emotional and prosodic range, including empathic, sarcastic, angry, sad, crying and other high-intensity speech styles.

The workflow combined AI-assisted transcription with human verification for verbatim transcripts, emotion labels, non-speech events and other metadata. Standardised quality criteria were applied across speakers and recording batches, making QA a continuous part of production rather than a final checkpoint.

This operating model helped Kyutai maintain consistency as the programme expanded across languages, speakers and increasingly nuanced voice requirements.

Delivering a detailed dataset for expressive multilingual TTS

The engagement produced a structured source of multilingual training data tailored to Kyutai's expressive TTS requirements.

Beyond the dataset itself, the collaboration established a repeatable process for managing talent, recording quality, annotation and acceptance criteria across distributed language programmes. That foundation reduced variability before data entered model training and gave Kyutai a more consistent way to generate future speech datasets as requirements evolved.