STEB：用于评估超越翻译保真度的语音到语音翻译表现力基准
STEB: A Speech-to-Speech Translation Expressiveness Benchmark for Evaluating Beyond Translation Fidelity
AI总结：
提出STEB基准，通过描述-总结框架评估语音到语音翻译在情感、场景风格和非语言发声方面的表现力，发现现有系统在语义保真度上表现良好但表现力保留不足。
摘要：
语音到语音翻译（S2ST）不仅应保留词汇意义，还应保留表现力属性：情感、场景风格（例如，新闻报道与戏剧对话）和非语言发声（NVs）。此外，大规模收集既忠实于翻译又在表现力上与源语言对齐的跨语言目标语音是困难的，这使得基于参考的评估不切实际。我们引入了STEB（语音到语音翻译表现力基准），这是一个32.6小时的中英基准，评估标准维度（翻译保真度、说话人相似性、时长对齐）和表现力维度（情感、场景风格、NV保留）。对于表现力评估，STEB使用描述-总结框架，将语音转换为结构化的表现力属性，并通过LLM评判器比较源语言和假设的属性。人工验证显示，在所有表现力维度上，与听众判断存在统计显著的相关性。我们评估了六个S2ST系统，涵盖级联系统、端到端模型和语音大语言模型。许多系统，尤其是级联系统，实现了强大的翻译保真度，但在情感保留（最佳：3.82/5）和NV保留（最佳：2.31/5）方面仍然困难。这些结果揭示了语义迁移和表现力迁移之间的差距，将表现力保留确定为S2ST的一个开放挑战。音频样本可在以下网址获取：此 https URL。
英文摘要：
Speech-to-speech translation (S2ST) should preserve not only lexical meaning, but also expressive attributes: emotion, scenario style (e.g., news reporting vs. dramatic dialogue), and nonverbal vocalizations (NVs). Moreover, collecting cross-lingual target speech that is both translation-faithful and expressively aligned with the source is difficult at scale, making reference-based evaluation impractical. We introduce STEB (Speech-to-Speech Translation Expressiveness Benchmark), a 32.6-hour Chinese--English benchmark that evaluates both standard dimensions (translation fidelity, speaker similarity, duration alignment) and expressiveness dimensions (emotion, scenario style, NV preservation). For expressiveness evaluation, STEB uses a caption-then-summarize framework that converts speech into structured expressive attributes and compares source and hypothesis attributes with an LLM judge. Human validation shows statistically significant correlations with listener judgments across all expressive dimensions. We evaluate six S2ST systems covering cascaded systems, end-to-end models, and speech large language models. Many systems, especially cascaded ones, achieve strong translation fidelity, but they still struggle with emotion preservation (best: 3.82/5) and NV preservation (best: 2.31/5). These results reveal a gap between semantic transfer and expressive transfer, identifying expressiveness preservation as an open challenge for S2ST. Audio samples are available at https://cmots.github.io/steb.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
