韵律在翻译中会丢失吗？跨语言的细粒度韵律相似性研究
Is Prosody Lost in Translation? Fine-Grained Cross-Lingual Prosody Similarity Across Languages
浏览论文内容
中文总结 AI 辅助
本研究利用多语言配音数据开展跨语言韵律细粒度分析，发现特定语言间韵律结构存在内在相关性，为表达性语音到语音翻译系统提供了实证指导。
中文摘要 AI 辅助
韵律在语音翻译中发挥重要作用，可传达强调、情感、意图等超出词汇内容的信息。尽管近期表达性语音到语音翻译（S2ST）取得进展，但人们对不同语言间韵律模式的相似性或差异知之甚少。理解这些跨语言异同对将韵律有效融入表达性S2ST系统至关重要。本研究首次利用英语-德语、英语-西班牙语、英语-法语的多语言配音数据，开展跨语言韵律的细粒度分析，分析源语言与目标语言语音的音高、能量及时序特征模式的相似性，探究影响该相似性的语言及对齐相关因素。分析发现特定语言间韵律结构存在内在跨语言相关性，研究结果为韵律跨语言迁移提供重要见解，并为未来表达性语音到语音翻译系统提供实证指导。
英文摘要
Prosody plays an important role in speech translation, conveying information such as emphasis, emotion, and intent beyond lexical content. However, despite recent progress in expressive speech-to-speech translation (S2ST), little is known about how prosodic patterns are similar/different across languages. Understanding these cross-lingual similarities and differences is crucial for effectively incorporating prosody into expressive S2ST systems. In this work, we present the first fine-grained cross-lingual analysis of prosody using multilingual dubbing data across English-German, English-Spanish, and English-French language pairs. We analyze the similarity of pitch, energy, and temporal feature patterns between source and target speech and investigate the linguistic and alignment-related factors affecting this similarity. Our analysis reveals inherent cross-lingual correlations in prosodic structure between certain languages. The findings provide important insights into the transferability of prosody across languages and offer empirical guidance for future expressive speech-to-speech translation systems.
发表机构
- Center for Language and Speech Processing (CLSP), Johns Hopkins University（约翰斯·霍普金斯大学语言与语音处理中心）
机构由 AI 辅助整理，请以论文原文为准。
