基于ASR转录文本的语音情感识别：词错误率与融合技术的综合研究
Speech Emotion Recognition with ASR Transcripts: A Comprehensive Study on Word Error Rate and Fusion Techniques
AI总结：
本研究针对现有语音情感识别依赖人工转录文本的问题，在多语料库上测试不同词错误率的ASR转录文本及多种融合技术的SER性能，并提出整合纠错与模态门控融合的抗错框架，提升了真实场景下的SER效果。
摘要：
文本数据通常被用作提升语音情感识别（SER）性能与可靠性的主要输入。然而，多数研究依赖人工转录文本，这阻碍了实用SER系统的发展，在实验室研究与以自动语音识别（ASR）为文本来源的真实场景之间造成了差距。因此，本研究在三个知名语料库（IEMOCAP、CMU-MOSI和MSP-Podcast）上，使用来自11个模型的不同词错误率（WER）的ASR转录文本，对SER性能进行基准测试。我们的评估涵盖纯文本和双模态SER，采用6种融合技术，旨在通过全面分析揭示当前SER研究面临的新发现与挑战。此外，我们提出了一个整合ASR纠错与模态门控融合的统一ASR抗错框架，与性能最优的ASR转录文本相比，实现了更低的WER与更优的SER结果。这些发现为ASR辅助的SER研究尤其是真实场景应用提供了洞见。
英文摘要：
Text data is commonly utilized as a primary input to enhance Speech Emotion Recognition (SER) performance and reliability. However, the reliance on human-transcribed text in most studies impedes the development of practical SER systems, creating a gap between in-lab research and real-world scenarios where Automatic Speech Recognition (ASR) serves as the text source. Hence, this study benchmarks SER performance using ASR transcripts with varying Word Error Rates (WERs) from eleven models on three well-known corpora: IEMOCAP, CMU-MOSI, and MSP-Podcast. Our evaluation includes both text-only and bimodal SER with six fusion techniques, aiming for a comprehensive analysis that uncovers novel findings and challenges faced by current SER research. Additionally, we propose a unified ASR error-robust framework integrating ASR error correction and modality-gated fusion, achieving lower WER and higher SER results compared to the best-performing ASR transcript. These findings provide insights into SER with ASR assistance, especially for real-world applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
