多语言多说话人单元声码器：离散语音表示的系统分析
Multilingual Multi-Speaker Unit Vocoders: A Systematic Analysis of Discrete Speech Representations
AI总结：
分析基于BigVGAN的单元声码器在多语言多说话人语音生成中的表现，发现聚类大小控制可懂度，显式说话人条件防止身份崩溃，语言监督在低聚类大小时有益。
摘要：
通过k-means聚类自监督嵌入获得的离散语音单元纠缠了音素、说话人和语言信息，导致多语言多说话人语音生成中的说话人混合和跨语言干扰。尽管在音频大语言模型和语音到语音系统中使用日益增多，单元声码器仍然研究不足。我们分析了基于BigVGAN的单元声码器，涵盖四种印度语言。我们使用WER、说话人相似度和单元级指标研究了聚类大小与条件策略之间的相互作用。结果表明，聚类大小通过提高音素区分性来控制可懂度，而显式说话人条件对于防止身份崩溃不可或缺。语言监督主要在单元仍模糊的较小聚类大小时带来进一步收益。我们的分析显示，在较小库存时，不同语言中相似音素会坍缩到相同的聚类ID，而较大的聚类会逐渐将它们分离。
英文摘要：
Discrete speech units obtained via k-means clustering of self supervised embeddings entangle phonetic, speaker, and language information, causing speaker mixing and cross-lingual interference in multilingual multi-speaker speech generation. Despite growing use in Audio LLMs and speech to speech systems, unit vocoders remain underexplored. We analyze a BigVGAN based unit vocoder, across four Indian languages. We study the interaction between cluster size and conditioning strategies using WER, speaker similarity, and unit level metrics. Results show that cluster size governs intelligibility by improving phonetic discriminability, while explicit speaker conditioning is indispensable for preventing identity collapse. Language supervision yields further gains mainly at lower cluster sizes where units remain ambiguous. Our analysis shows similar phonemes across languages collapse to the same cluster IDs at smaller inventories, with larger clusters progressively separating them.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
