arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.01467eess.AS
收藏

会话语音自然度预测器

Conversational Speech Naturalness Predictor

University of Southern California(南加州大学);Meta Platforms, Inc(Meta平台公司)

Anfeng Xu, Yashesh Gaur, Naoyuki Kanda, Zhicheng Ouyang, Katerina Zmolikova, Desh Raj, Simone Merello, Anna Sun, Ozlem Kalinli

AI总结:

本文提出了一种双通道自然度预测器,用于多轮对话中的语音自然度评估,通过预训练编码器和数据增强提升预测性能。

摘要:

评估会话的自然度对于开发类人语音代理至关重要。然而,现有的语音自然度预测器往往设计用于评估单个说话人的语音,无法捕捉到对话层面的自然度特征。在本文中,我们提出了一种自动自然度预测框架,用于双说话人、多轮对话。我们首先展示现有自然度估计器与会话自然度之间的相关性较低,甚至为负,基于带有人类评分的会话录音。然后,我们提出了一种双通道自然度估计器,在其中我们研究了多个预训练编码器与数据增强。我们提出的模型在域内和域外条件下都比现有自然度预测器与人类判断具有显著更高的相关性。

英文摘要:

Evaluation of conversational naturalness is essential for developing human-like speech agents. However, existing speech naturalness predictors are often designed to assess utterances from a single speaker, failing to capture conversation-level naturalness qualities. In this paper, we present a framework for an automatic naturalness predictor for two-speaker, multi-turn conversations. We first show that existing naturalness estimators have low, or sometimes even negative, correlations with conversational naturalness, based on conversational recordings annotated with human ratings. We then propose a dual-channel naturalness estimator, in which we investigate multiple pre-trained encoders with data augmentation. Our proposed model achieves substantially higher correlation with human judgments compared to existing naturalness predictors for both in-domain and out-of-domain conditions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑