SageLM：一种用于语音评判的多方面可解释大语言模型
SageLM: A Multi-aspect and Explainable Large Language Model for Speech Judgement
AI总结：
SageLM提出一种端到端、多方面可解释的语音大语言模型，通过联合评估语义与声学维度、引入理由监督和合成偏好数据集SpeechFeedback，在语音到语音模型评估中与人类一致率达82.79%，显著优于现有基线。
摘要：
语音到语音（S2S）大语言模型（LLMs）是自然人机交互的基础，能够实现端到端口语对话系统。然而，评估这些模型仍然是一个根本性挑战。我们提出SageLM，一个端到端、多方面且可解释的语音LLM，用于全面的S2S LLMs评估。首先，与忽略声学特征的级联方法不同，SageLM联合评估语义和声学两个维度。其次，它利用基于理由的监督来增强可解释性并引导模型学习，与基于规则的强化学习方法相比，实现了与评估结果更优的一致性。第三，我们引入了SpeechFeedback，一个合成的偏好数据集，并采用两阶段训练范式来缓解语音偏好数据的稀缺问题。在语义和声学两个维度上训练后，SageLM与人类评估者达到了82.79%的一致率，分别比级联基线和基于SLM的基线至少高出7.42%和26.20%。
英文摘要：
Speech-to-Speech (S2S) Large Language Models (LLMs) are foundational to natural human-computer interaction, enabling end-to-end spoken dialogue systems. However, evaluating these models remains a fundamental challenge. We propose \texttt{SageLM}, an end-to-end, multi-aspect, and explainable speech LLM for comprehensive S2S LLMs evaluation. First, unlike cascaded approaches that disregard acoustic features, SageLM jointly assesses both semantic and acoustic dimensions. Second, it leverages rationale-based supervision to enhance explainability and guide model learning, achieving superior alignment with evaluation outcomes compared to rule-based reinforcement learning methods. Third, we introduce \textit{SpeechFeedback}, a synthetic preference dataset, and employ a two-stage training paradigm to mitigate the scarcity of speech preference data. Trained on both semantic and acoustic dimensions, SageLM achieves an 82.79\% agreement rate with human evaluators, outperforming cascaded and SLM-based baselines by at least 7.42\% and 26.20\%, respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
