语音优先的印度语言国家偏好：面向印度语言TTS的大规模成对评估与偏好分析
Preferences of a Voice-First Nation: Large-Scale Pairwise Evaluation and Preference Analysis for TTS in Indian Languages
AI总结：
本文通过大规模成对评估和分析，探讨印度语言TTS系统在多语言环境下的偏好，结合语言控制与感知注解，评估7种先进系统，揭示人类偏好与模型性能的多维关系。
摘要：
众包成对评估已成为评估基础模型的可扩展方法。然而，将其应用于文本到语音（TTS）时，由于语言多样性及语音感知的多维特性，引入了高方差。我们提出了一种受控的多维成对评估框架，用于多语言TTS，结合语言控制与感知导向的注解。使用5000+个本地和混合语言句子，覆盖10种印度语言，评估7种最先进的TTS系统，并从1900名本地评估者中收集超过120,000次成对比较。除了总体偏好外，评估者还提供了六个感知维度的判断：可理解性、表现力、声音质量、活力、噪声和幻觉。使用Bradley-Terry建模，我们构建了多语言排行榜，通过SHAP分析解释人类偏好，并分析排行榜的可靠性以及模型在感知维度上的优势与权衡。
英文摘要：
Crowdsourced pairwise evaluation has emerged as a scalable approach for assessing foundation models. However, applying it to Text to Speech(TTS) introduces high variance due to linguistic diversity and multidimensional nature of speech perception. We present a controlled multidimensional pairwise evaluation framework for multilingual TTS that combines linguistic control with perceptually grounded annotation. Using 5K+ native and code-mixed sentences across 10 Indic languages, we evaluate 7 state-of-the-art TTS systems and collect over 120K pairwise comparisons from over 1900 native raters. In addition to overall preference, raters provide judgments across 6 perceptual dimensions: intelligibility, expressiveness, voice quality, liveliness, noise, and hallucinations. Using Bradley-Terry modeling, we construct a multilingual leaderboard, interpret human preference using SHAP analysis and analyze leaderboard reliability alongside model strengths and trade-offs across perceptual dimensions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
