从提示到标记：将因果监督内化到视觉-语言模型中进行多图像因果推理
From Prompts to Tokens: Internalizing Causal Supervision in Vision-Language Model for Multi-Image Causal Reasoning
AI总结：
提出BridgeVLM，通过从多图像输入诱导因果图并转换为因果标记，注入LLM解码器进行因果消息传递，显著提升多图像因果推理性能。
摘要：
视觉因果推理对于理解和干预物理世界至关重要，需要从视觉输入中识别因果变量并推理干预效果。尽管最近取得了进展，大型视觉-语言模型（VLM）在此类任务上仍然脆弱，尤其是对于多图像输入上的干预和反事实查询。大多数现有探索通过文本提示注入因果知识，使因果机制外在于模型执行，限制了推理过程中的可靠控制。为了解决这个问题，我们提出了BridgeVLM，它通过从多图像输入中诱导因果图并将其转换为结构化的因果标记，由注入到LLM解码器中的RAMP层执行因果消息传递，从而内化视觉因果推理。我们进一步引入了一个统一的训练接口M3S，用于不同粒度（局部/全局级别）的细粒度因果监督。BridgeVLM在CausalVLBench的干预任务上达到了54.4%的准确率（而提示级监督为33.2%），在Causal3D上将结果从43.6%提升到49.0%，并在CausalVLBench上显著改善了因果结构学习（$F_1$：33.4% → 75.1%）。
英文摘要：
Visual causal reasoning is essential for understanding and intervening in the physical world, requiring identification of causal variables from visual inputs and reasoning over intervention effects. Despite recent progress, large vision--language models (VLMs) remain brittle at such tasks, especially for interventional and counterfactual queries over multi-image inputs. Most existing explorations inject causal knowledge via textual prompts, leaving causal mechanisms external to model execution and limiting reliable control during inference. To address this problem, we propose BridgeVLM, which internalizes visual causal reasoning by inducing a causal graph from multi-image inputs and converting it into structured Causal Tokens executed by RAMP layers injected into the LLM decoder for causal message passing. We further introduce a unified training interface M3S for fine-grained causal supervision from different granularities (local/global level). BridgeVLM achieves 54.4% accuracy on intervention tasks on CausalVLBench (vs. 33.2% with prompt-level supervision), improves results on Causal3D from 43.6% to 49.0%, and substantially improves causal structure learning on CausalVLBench ($F_1$: 33.4% $\rightarrow$ 75.1%).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
