CF-VLM：反事实视觉-语言微调
CF-VLM:CounterFactual Vision-Language Fine-tuning
AI总结：
针对视觉-语言模型在细粒度判别和因果推理上的局限，提出反事实视觉-语言微调（CF-VLM）框架，通过三个互补训练目标增强因果推理能力，在组合推理和泛化基准上超越现有方法，并缓解视觉幻觉。
摘要：
近期视觉-语言模型（VLM）的进展极大地提升了跨模态语义理解能力，但在细粒度判别和深度因果推理任务中仍存在显著局限。现有VLM往往依赖表面的统计相关性，缺乏捕捉视觉与文本内容之间潜在因果逻辑的能力。为解决这一问题，我们提出了反事实视觉-语言微调（CF-VLM），一种通过针对性使用反事实样本来增强VLM因果推理能力的新框架。CF-VLM引入了三个互补的训练目标：维持基础的跨模态对齐，强化事实场景表示相对于连贯反事实的独特性和稳定性，以及提升模型对最小但关键因果编辑的敏感性。大量实验表明，CF-VLM在组合推理和泛化基准上持续优于强基线和最先进方法。此外，它在缓解视觉幻觉方面展现出潜力，表明事实一致性得到改善。我们的CF-VLM为在高风险、真实世界场景中部署需要可靠推理和可解释性的VLM提供了坚实基础。
英文摘要：
Recent advances in vision-language models (VLMs) have greatly improved cross-modal semantic understanding, yet significant limitations remain in fine-grained discrimination and deep causal reasoning tasks. Existing VLMs often rely on superficial statistical correlations, lacking the ability to capture the underlying causal logic between visual and textual content. To address this, we propose CounterFactual Vision-Language Fine-tuning (CF-VLM), a novel framework that enhances the causal reasoning capabilities of VLMs through the targeted use of counterfactual samples. CF-VLM introduces three complementary training objectives: maintaining foundational cross-modal alignment, reinforcing the uniqueness and stability of factual scene representations against coherent counterfactuals, and sharpening the model's sensitivity to minimal but critical causal edits. Extensive experiments demonstrate that CF-VLM consistently outperforms strong baselines and state-of-the-art methods on compositional reasoning and generalization benchmarks. Furthermore, it shows promise in mitigating visual hallucinations, indicating improved factual consistency. Our CF-VLM provides a robust foundation for deploying VLMs in high-stakes, real-world scenarios requiring reliable reasoning and interpretability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
