用于鲁棒视觉问答的反事实推理
Counterfactual Reasoning for Robust Visual Question Answering
查看机构详情
- University of Science, Ho Chi Minh City（胡志明市理科大学）
- Vietnam National University, Ho Chi Minh City（越南国立大学胡志明市分校）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
提出反事实对比学习框架，通过三阶段课程、增强批对比损失及AC和GD正则化，提升VQA模型OOD鲁棒性，在VQA-CP v2和VQA v2上分别达61.64%和62.80%准确率。
中文摘要 AI 辅助
现代视觉问答（VQA）模型常常利用训练数据中的虚假相关性，导致由于语言偏差而在分布外（OOD）泛化方面表现不佳。尽管反事实学习已显示出潜力，但现有方法仍有改进空间，以更好地引导注意力集中于因果证据并增强特征判别性。为解决这一问题，我们提出了一种新颖的训练框架，以增强VQA的反事实对比学习。我们的框架引入了三个关键贡献：（1）用于稳定多目标优化的三阶段课程，（2）增强的批对比损失以实现更具判别性的特征学习，以及（3）两个新颖的正则化器：答案对比（AC）损失以细化预测空间，以及梯度差异（GD）损失以强制因果视觉接地。我们的模型在偏置敏感的VQA-CP v2基准上达到了61.64%的竞争性准确率，同时在标准VQA v2数据集上保持了62.80%的准确率，产生了1.16%的小泛化差距。这展示了OOD鲁棒性与分布内性能之间的强平衡。
英文摘要
Modern Visual Question Answering (VQA) models often exploit spurious correlations in training data, leading to poor out-of-distribution (OOD) generalization due to language bias. Although counterfactual learning has shown promise, existing methods can be improved to better guide attention toward causal evidence and strengthen feature discrimination. To address this, we propose a novel training framework that enhances counterfactual contrastive learning for VQA. Our framework introduces three key contributions: (1) a three-stage curriculum for stable multi-objective optimization, (2) an enhanced Batch-Contrastive loss for more discriminative feature learning, and (3) two novel regularizers, Answer-Contrastive (AC) loss to refine the prediction space and Gradient-Discrepancy (GD) loss to enforce causal visual grounding. Our model achieves a competitive accuracy of 61.64% on the bias-sensitive VQA-CP v2 benchmark while maintaining 62.80% on the standard VQA v2 dataset, yielding a small generalization gap of 1.16%. This demonstrates a strong balance between OOD robustness and in-distribution performance.
