双轴生成奖励模型：面向交互口语对话模型中语义和轮流鲁棒性的研究
Dual-Axis Generative Reward Model Toward Semantic and Turn-taking Robustness in Interactive Spoken Dialogue Models
AI总结：
本文提出双轴生成奖励模型，通过详细分类和标注数据集理解复杂交互动态，提供语义质量和交互时间的独立评估，提升口语对话模型的鲁棒性与性能。
摘要：
实现无缝、类人交互仍是全双工口语对话模型（SDMs）的关键挑战。强化学习（RL）显著提升了文本和视觉语言模型，而精心设计的奖励信号对RL性能至关重要。我们考虑RL是解决SDMs关键挑战的有希望策略。然而，一个基本障碍仍然存在：现有的自动评估交互质量的指标依赖于表面代理，如行为统计或时间预测准确性，无法提供可靠的RL奖励信号。另一方面，人类评估尽管丰富，但成本高、不一致且难以扩展。我们通过提出双轴生成奖励模型来解决这一关键障碍，该模型通过详细分类和标注数据集理解复杂交互动态，生成单一评分，并关键地提供语义质量和交互时间的独立评估。此类双输出为SDMs提供精确的诊断反馈，并提供可靠的、指导性的奖励信号，适用于在线强化学习。我们的模型在交互质量评估上实现了跨广泛数据集的最先进性能，涵盖合成对话和复杂的现实交互。
英文摘要：
Achieving seamless, human-like interaction remains a key challenge for full-duplex spoken dialogue models (SDMs). Reinforcement learning (RL) has substantially enhanced text- and vision-language models, while well-designed reward signals are crucial for the performance of RL. We consider RL a promising strategy to address the key challenge for SDMs. However, a fundamental barrier persists: prevailing automated metrics for assessing interaction quality rely on superficial proxies, such as behavioral statistics or timing-prediction accuracy, failing to provide reliable reward signals for RL. On the other hand, human evaluations, despite their richness, remain costly, inconsistent, and difficult to scale. We tackle this critical barrier by proposing a Dual-Axis Generative Reward Model, which is trained to understand complex interaction dynamics using a detailed taxonomy and an annotated dataset, produces a single score and, crucially, provides separate evaluations for semantic quality and interaction timing. Such dual outputs furnish precise diagnostic feedback for SDMs and deliver a dependable, instructive reward signal suitable for online reinforcement learning. Our model achieves state-of-the-art performance on interaction-quality assessment across a wide spectrum of datasets, spanning synthetic dialogues and complex real-world interactions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
