SpeechJudge: 向人类水平的语音自然度判断迈进
SpeechJudge: Towards Human-Level Judgment for Speech Naturalness
AI总结：
SpeechJudge通过构建大规模人类反馈数据集和生成奖励模型，提升语音自然度判断的准确性，显著优于现有方法。
摘要：
将大型生成模型与人类反馈对齐是一个关键挑战。在语音合成中，这一问题尤为突出，因为缺乏大规模的人类偏好数据集，阻碍了真正与人类感知对齐的模型发展。为此，我们引入SpeechJudge，一个包含数据集、基准和以自然度为核心的奖励模型的综合体系——自然度是语音合成中最基本的主观度量指标之一。首先，我们提出了SpeechJudge-Data，一个包含99,000对语音的大型人类反馈语料库。该数据集通过使用多样化的先进零样本文本到语音（TTS）模型，涵盖多种语音风格和多种语言，进行构建，并对可懂度和自然度偏好进行人工标注。从该数据集出发，我们建立了SpeechJudge-Eval，一个语音自然度判断的挑战性基准。我们的评估发现，现有指标和AudioLLMs在该任务上表现不佳；领先的模型Gemini-2.5-Flash在与人类判断的一致性上仅达到不到70%，凸显了改进的必要性。为弥合这一差距，我们开发了SpeechJudge-GRM，一个基于Qwen2.5-Omni-7B的生成奖励模型（GRM）。它通过两阶段的微调过程在SpeechJudge-Data上进行训练：首先通过Chain-of-Thought推理进行监督微调，然后通过GRPO在具有挑战性的案例上进行强化学习。在SpeechJudge-Eval基准上，所提出的SpeechJudge-GRM表现出色，其准确率为77.2%（在推理时扩展至10倍后达到79.4%），优于经典的Bradley-Terry奖励模型（72.7%）。此外，SpeechJudge-GRM也可作为语音生成模型后训练期间的奖励函数，以促进其与人类偏好的对齐。
英文摘要：
Aligning large generative models with human feedback is a critical challenge. In speech synthesis, this is particularly pronounced due to the lack of a large-scale human preference dataset, which hinders the development of models that truly align with human perception. To address this, we introduce SpeechJudge, a comprehensive suite comprising a dataset, a benchmark, and a reward model centered on naturalness--one of the most fundamental subjective metrics for speech synthesis. First, we present SpeechJudge-Data, a large-scale human feedback corpus of 99K speech pairs. The dataset is constructed using a diverse set of advanced zero-shot text-to-speech (TTS) models across diverse speech styles and multiple languages, with human annotations for both intelligibility and naturalness preference. From this, we establish SpeechJudge-Eval, a challenging benchmark for speech naturalness judgment. Our evaluation reveals that existing metrics and AudioLLMs struggle with this task; the leading model, Gemini-2.5-Flash, achieves less than 70% agreement with human judgment, highlighting a significant gap for improvement. To bridge this gap, we develop SpeechJudge-GRM, a generative reward model (GRM) based on Qwen2.5-Omni-7B. It is trained on SpeechJudge-Data via a two-stage post-training process: Supervised Fine-Tuning (SFT) with Chain-of-Thought rationales followed by Reinforcement Learning (RL) with GRPO on challenging cases. On the SpeechJudge-Eval benchmark, the proposed SpeechJudge-GRM demonstrates superior performance, achieving 77.2% accuracy (and 79.4% after inference-time scaling @10) compared to a classic Bradley-Terry reward model (72.7%). Furthermore, SpeechJudge-GRM can be also employed as a reward function during the post-training of speech generation models to facilitate their alignment with human preferences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
