arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.28618eess.AS
收藏

多样场景下长篇语音生成的综合基准测试

Comprehensive Benchmarking of Long-Form Speech Generation in Diverse Scenarios

Zhejiang University(浙江大学);Bytedance(字节跳动)

Changhao Pan, Rui Yang, Han Wang, Zhuan Zhou, Xuming He, Wenxiang Guo, Ziyue Jiang, Ruiqi Li, Yu Zhang, Chenyuhao Wen, Ke Lei, Xiang Yin, Jingyu Lu, Zhiyuan Zhu, Zhou Zhao

AI总结:

针对现有评估局限于有限领域且忽略一致性与连贯性等问题,提出Swanbench-Speech基准,从声学、语义和表现力三个维度分解长篇语音质量,涵盖17种场景、1101个样本,并定义七项自动评估指标,揭示当前模型在高表现力场景中的不足及与真实录音的差距。

摘要:

近期语音生成的进展实现了高保真合成,但在长上下文条件下对模型的系统评估仍 largely 未充分探索。长篇语音的综合评估基准不可或缺,原因有二:1)现有测试场景通常局限于有限领域,与多样化的下游应用之间存在显著差距;2)现有指标忽视了诸如一致性和连贯性等关键的长文本因素,无法可靠地泛化。为此,我们提出Swanbench-Speech,一个将长篇语音质量分解为特定、解耦维度的综合基准。SwanBench-Speech具有三个关键特性:1)丰富的语音场景:聚焦于长篇语音生成和对话生成,SwanBench-Speech涵盖声学、语义和表现力挑战,由跨越17种常见语音场景的1101个样本组成;2)全面的评估维度:沿声学、语义和表现力轴,SwanBench-Speech定义了一个包含七项指标的自动评估协议,以提供全面、准确和标准化的评估;3)有价值的见解:通过大量实验,我们揭示当前模型在高表现力场景中仍存在困难,并在一致性和层次结构方面与真实录音存在显著差距。

英文摘要:

Recent advances in speech generation have enabled high-fidelity synthesis, yet systematic evaluation of models under long-context conditions remains largely underexplored. A comprehensive evaluation benchmark for long-form speech is indispensable for two reasons: 1) existing test scenarios are often confined to limited domains, creating a significant gap with the diverse downstream applications; 2) existing metrics overlook critical long-text factors such as consistency and coherence, failing to generalize reliably. To this end, we propose Swanbench-Speech, a comprehensive benchmark that decomposes long-form speech quality into specific, disentangled dimensions. SwanBench-Speech has three key properties. 1) Rich speech scenarios: Focusing on long-form speech generation and dialog generation, SwanBench-Speech covers acoustics, semantics, and expressiveness challenges, and consists of 1,101 samples spanning 17 common speech scenarios; 2) Comprehensive evaluation dimensions: Along the acoustics, semantics, and expressiveness axes, SwanBench-Speech defines an automated evaluation protocol with seven metrics to provide a comprehensive, accurate, and standardized assessment; 3) Valuable Insights: Through extensive experiments, we reveal that current models still struggle in highly expressive scenarios and exhibit a notable gap in consistency and hierarchy compared to real recordings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑