CausalVLBench：大型视觉-语言模型中的视觉因果推理基准测试
CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Models
AI总结：
本文提出CausalVLBench基准，包含因果结构推断、干预目标预测和反事实预测三个任务，评估开源LVLMs在视觉因果推理上的能力，揭示其优缺点并指引改进方向。
摘要：
大型语言模型（LLMs）在各种语言任务中展现出显著能力，尤其是其涌现的上下文学习能力。将LLMs扩展以融入视觉输入，大型视觉-语言模型（LVLMs）在识别和视觉问答（VQA）等任务中表现出色。尽管人们对LLMs在因果发现和反事实推理等因果推理任务中的实用性兴趣日益增加，但展示LVLMs在视觉因果推理任务上能力的工作相对较少。我们借此机会正式引入一个全面的因果推理基准，用于LVLMs的多模态上下文学习。我们的CausalVLBench包含三个代表性任务：因果结构推断、干预目标预测和反事实预测。我们在三个因果表示学习数据集上评估了最先进的开源LVLMs在我们因果推理任务上的能力，并展示了它们的基本优势和劣势。我们希望我们的基准能够阐明现有视觉-语言模型的缺陷，并激发改进LVLMs视觉因果推理能力的新方向和范式。
英文摘要：
Large language models (LLMs) have shown remarkable ability in various language tasks, especially with their emergent in-context learning capability. Extending LLMs to incorporate visual inputs, large vision-language models (LVLMs) have shown impressive performance in tasks such as recognition and visual question answering (VQA). Despite increasing interest in the utility of LLMs in causal reasoning tasks such as causal discovery and counterfactual reasoning, there has been relatively little work showcasing the abilities of LVLMs on visual causal reasoning tasks. We take this opportunity to formally introduce a comprehensive causal reasoning benchmark for multi-modal in-context learning from LVLMs. Our CausalVLBench encompasses three representative tasks: causal structure inference, intervention target prediction, and counterfactual prediction. We evaluate the ability of state-of-the-art open-source LVLMs on our causal reasoning tasks across three causal representation learning datasets and demonstrate their fundamental strengths and weaknesses. We hope that our benchmark elucidates the drawbacks of existing vision-language models and motivates new directions and paradigms in improving the visual causal reasoning abilities of LVLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
