语音情感识别（SER）系统通常假设语音情感与词汇语义是一致的。然而，在现实世界互动中，语音-语义冲突是常见但被忽视的现象，其中由音调传达的情感与 spoken words 的字面意义相矛盾。我们展示，最先进的 SER 模型，包括基于 ASR 的、自监督学习（SSL）方法和音频语言模型（ALMs），在这样的冲突下会因语义偏见或纠缠的语音-语义表示而性能下降。为了解决这个问题，我们提出了融合语音-语义（FAS）框架，该框架明确分离语音和语义路径，并通过一个轻量级、基于查询的注意力模块连接它们。为了实现系统的评估，我们引入了冲突在语音-语义情感（CASE），这是第一个由清晰且可解释的语音-语义冲突主导的、在各种场景中均存在的数据集。广泛的实验表明，FAS 在域内和零样本设置中均优于现有方法。值得注意的是，在 CASE 基准测试中，传统 SER 模型大幅失败，而 FAS 在 59.38% 准确率上设立了新的 SOTA。我们的代码和数据集可在 https://github.com/24DavidHuang/FAS 上获得。
英文摘要：
Speech Emotion Recognition (SER) systems often assume congruence between vocal emotion and lexical semantics. However, in real-world interactions, acoustic-semantic conflict is common yet overlooked, where the emotion conveyed by tone contradicts the literal meaning of spoken words. We show that state-of-the-art SER models, including ASR-based, self-supervised learning (SSL) approaches and Audio Language Models (ALMs), suffer performance degradation under such conflicts due to semantic bias or entangled acoustic-semantic representations. To address this, we propose the Fusion Acoustic-Semantic (FAS) framework, which explicitly disentangles acoustic and semantic pathways and bridges them through a lightweight, query-based attention module. To enable systematic evaluation, we introduce the Conflict in Acoustic-Semantic Emotion (CASE), the first dataset dominated by clear and interpretable acoustic-semantic conflicts in varied scenarios. Extensive experiments demonstrate that FAS consistently outperforms existing methods in both in-domain and zero-shot settings. Notably, on the CASE benchmark, conventional SER models fail dramatically, while FAS sets a new SOTA with 59.38% accuracy. Our code and datasets is available at https://github.com/24DavidHuang/FAS.
Towards LLM-Empowered Fine-Grained Speech Descriptors for Explainable Emotion Recognition
The Chinese University of Hong Kong（香港中文大学）；Chinese Academy of Sciences（中国科学院）；National Research Council Canada（加拿大国家研究理事会）；Institute of Software（软件研究所）
VoxEmo: Benchmarking Speech Emotion Recognition with Speech LLMs
Department of Computer Science, University of Sheffield, United Kingdom（英国谢菲尔德大学计算机科学系）；Signal Analysis and Interpretation Laboratory (SAIL), Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, CA 90089, USA（美国南加州大学电气与计算机工程系信号分析与解释实验室（SAIL））
VoiceWeaver: Staged Learning of Structured Controls for Expressive Speech and Sound-Event Generation
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
