通过因果信息最小化实现多模态模型去偏
Debiasing Multimodal Models via Causal Information Minimization
AI总结：
本文提出通过因果信息最小化学习混淆因子表示以消除多模态模型偏差，在多个数据集上提升分布外性能且不损害分布内性能。
摘要：
现有的大多数多模态模型去偏方法，包括因果干预和因果推断方法，都利用近似启发式来表示偏差，例如训练早期阶段的浅层特征或用于VQA等多模态任务的单模态特征等，这些表示可能并不准确。本文研究了多模态数据因果图中由混淆因子引起的偏差，并探讨了一种利用因果驱动的信息最小化来学习混淆因子表示的新方法。鲁棒的预测特征包含多样化的信息，有助于模型泛化到分布外数据。因此，最小化从预训练的有偏模型获得的特征的信息内容，有助于学习能够捕捉底层数据分布的最简单预测特征。我们将这些特征视为混淆因子表示，并通过因果理论驱动的方法使用它们来消除模型中的偏差。我们发现，学习到的混淆因子表示确实捕捉到了数据集偏差，并且所提出的去偏方法在多个多模态数据集上提高了分布外（OOD）性能，同时没有牺牲分布内性能。此外，我们引入了一种新的度量来量化模型预测中虚假特征的充分性，进一步证明了所提方法的有效性。我们的代码可在以下地址获取：https://github.com/Vaidehi99/CausalInfoMin
英文摘要：
Most existing debiasing methods for multimodal models, including causal intervention and inference methods, utilize approximate heuristics to represent the biases, such as shallow features from early stages of training or unimodal features for multimodal tasks like VQA, etc., which may not be accurate. In this paper, we study bias arising from confounders in a causal graph for multimodal data and examine a novel approach that leverages causally-motivated information minimization to learn the confounder representations. Robust predictive features contain diverse information that helps a model generalize to out-of-distribution data. Hence, minimizing the information content of features obtained from a pretrained biased model helps learn the simplest predictive features that capture the underlying data distribution. We treat these features as confounder representations and use them via methods motivated by causal theory to remove bias from models. We find that the learned confounder representations indeed capture dataset biases, and the proposed debiasing methods improve out-of-distribution (OOD) performance on multiple multimodal datasets without sacrificing in-distribution performance. Additionally, we introduce a novel metric to quantify the sufficiency of spurious features in models' predictions that further demonstrates the effectiveness of our proposed methods. Our code is available at: https://github.com/Vaidehi99/CausalInfoMin
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
