通过偏差纯化实现多模态情感分析去偏
Towards Multimodal Sentiment Analysis Debiasing via Bias Purification
AI总结：
本文提出基于因果推断的MCIS框架，通过构建因果图并设想反事实场景来纯化多模态情感分析中的标签偏差和上下文偏差，从而在多个基准上实现无偏决策。
摘要：
多模态情感分析（MSA）旨在通过整合来自视觉、语言和音频等不同模态的情感相关线索来理解人类意图。不幸的是，当前的MSA任务不可避免地受到非计划性数据集偏差的影响，尤其是多模态话语级标签偏差和词级上下文偏差。这些有害偏差可能误导模型关注统计捷径和虚假相关性，从而导致严重的性能瓶颈。为缓解这些问题，我们提出了一种基于因果性而非传统似然的多模态反事实推断情感（MCIS）分析框架。具体而言，我们首先构建一个因果图，以从已训练的普通模型中发掘有害偏差。在推断阶段，给定一个事实性多模态输入，MCIS设想两个反事实场景以纯化和缓解这些偏差。然后，MCIS可以通过比较事实结果与反事实结果，从有偏观测中做出无偏决策。我们在多个标准MSA基准上进行了广泛实验。定性和定量结果均表明了所提框架的有效性。
英文摘要：
Multimodal Sentiment Analysis (MSA) aims to understand human intentions by integrating emotion-related clues from diverse modalities, such as visual, language, and audio. Unfortunately, the current MSA task invariably suffers from unplanned dataset biases, particularly multimodal utterance-level label bias and word-level context bias. These harmful biases potentially mislead models to focus on statistical shortcuts and spurious correlations, causing severe performance bottlenecks. To alleviate these issues, we present a Multimodal Counterfactual Inference Sentiment (MCIS) analysis framework based on causality rather than conventional likelihood. Concretely, we first formulate a causal graph to discover harmful biases from already-trained vanilla models. In the inference phase, given a factual multimodal input, MCIS imagines two counterfactual scenarios to purify and mitigate these biases. Then, MCIS can make unbiased decisions from biased observations by comparing factual and counterfactual outcomes. We conduct extensive experiments on several standard MSA benchmarks. Qualitative and quantitative results show the effectiveness of the proposed framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
