ACR-Net:通过对比声学-语义解耦缓解语义主导问题
ACR-Net: Mitigating Semantic Dominance via Contrastive Acoustic-Semantic Decoupling
AI总结:
该研究针对SER模型和Audio LLMs的语义主导问题,构建ASPIRE基准与SOP、LDD指标,提出ACR-Net模型,在两类场景中均实现更优声学准确率。
摘要:
语音情感识别(SER)模型和音频大语言模型(Audio LLMs)在语音语调与文本语义矛盾时会失效。在一致数据上的大量训练会形成僵化的语义先验,引发“语义主导”问题,这是跨模态冲突中一种严重的文本偏差。我们推出了包含四类冲突类型的对抗性音频-文本对基准ASPIRE,以及两个指标:语义过度自信惩罚(Semantic Overconfidence Penalty,SOP)和潜在解耦程度(Latent Decoupling Degree,LDD)。与仅对坍缩特征重新加权的融合方法不同,我们的声学冲突解决网络(Acoustic Conflict Resolution Network,ACR-Net)利用跨模态注意力和对比解耦损失,将矛盾表示解耦为正交潜在空间。实验表明,现有模型在极性冲突下因极端语义过度自信而失效;相反,ACR-Net保留了声学保真度,维持高LDD以将SOP趋近于零,在对抗性场景和标准场景中均实现了优异的声学准确率(Acoustic Accuracy,ACC)。
英文摘要:
Speech Emotion Recognition (SER) models and Audio LLMs fail when vocal tone contradicts textual semantics. Extensive training on congruent data engenders a rigid semantic prior, causing "Semantic Dominance", a severe text bias during cross-modal conflicts. We introduce the ASPIRE benchmark of adversarial audio-text pairs across four conflict types, alongside two metrics: Semantic Overconfidence Penalty (SOP) and Latent Decoupling Degree (LDD). Unlike fusion methods that merely re-weight collapsed features, our Acoustic Conflict Resolution Network (ACR-Net) uses Cross-Modal Attention and Contrastive Decoupling Loss to disentangle contradictory representations into orthogonal latent spaces. Experiments show existing models fail under polarity conflicts due to extreme semantic overconfidence. Conversely, ACR-Net preserves acoustic fidelity, maintaining high LDD to drive SOP near zero, achieving superior Acoustic Accuracy (ACC) in both adversarial and standard scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。