RobustVisRAG: 基于因果性的视觉增强检索增强生成方法在视觉退化下的鲁棒性
RobustVisRAG: Causality-Aware Vision-Based Retrieval-Augmented Generation under Visual Degradations
AI总结：
本文提出RobustVisRAG，通过双路径框架分离语义与退化因素，提升在视觉退化下的检索生成性能，实验显示在真实退化条件下性能提升显著。
摘要：
基于视觉的语言模型（VLMs）联合检索相关视觉文档并生成基于多模态证据的 grounded 答案。然而，现有 VisRAG 模型在视觉输入遭受模糊、噪声、低光或阴影等退化时性能下降，其中语义和退化因素在预训练视觉编码器中交织，导致检索和生成阶段的错误。为解决此问题，我们引入 RobustVisRAG，一种因果引导的双路径框架，提升 VisRAG 的鲁棒性同时保持效率和零样本泛化能力。RobustVisRAG 使用非因果路径通过单向注意力捕捉退化信号，因果路径则通过这些信号学习纯净的语义。结合提出的非因果退化建模和因果语义对齐目标，框架强制语义与退化因素的清晰分离，从而在挑战性视觉条件下实现稳定的检索和生成。为评估在现实条件下的鲁棒性，我们引入了 Distortion-VisRAG 数据集，包含七个领域的大量合成和真实退化文档，涵盖12种合成和5种真实退化类型。实验结果表明，RobustVisRAG 在真实退化条件下检索、生成和端到端性能分别提升7.35%、6.35%和12.40%，同时在清洁输入上保持可比的准确性。
英文摘要：
Vision-based Retrieval-Augmented Generation (VisRAG) leverages vision-language models (VLMs) to jointly retrieve relevant visual documents and generate grounded answers based on multimodal evidence. However, existing VisRAG models degrade in performance when visual inputs suffer from distortions such as blur, noise, low light, or shadow, where semantic and degradation factors become entangled within pretrained visual encoders, leading to errors in both retrieval and generation stages. To address this limitation, we introduce RobustVisRAG, a causality-guided dual-path framework that improves VisRAG robustness while preserving efficiency and zero-shot generalization. RobustVisRAG uses a non-causal path to capture degradation signals through unidirectional attention and a causal path to learn purified semantics guided by these signals. Together with the proposed Non-Causal Distortion Modeling and Causal Semantic Alignment objectives, the framework enforces a clear separation between semantics and degradations, enabling stable retrieval and generation under challenging visual conditions. To evaluate robustness under realistic conditions, we introduce the Distortion-VisRAG dataset, a large-scale benchmark containing both synthetic and real-world degraded documents across seven domains, with 12 synthetic and 5 real distortion types that comprehensively reflect practical visual degradations. Experimental results show that RobustVisRAG improves retrieval, generation, and end-to-end performance by 7.35%, 6.35%, and 12.40%, respectively, on real-world degradations, while maintaining comparable accuracy on clean inputs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
