CC-VQA: 一种考虑冲突和相关性的方法，用于缓解基于知识的视觉问答中的知识冲突
CC-VQA: Conflict- and Correlation-Aware Method for Mitigating Knowledge Conflict in Knowledge-Based Visual Question Answering
AI总结：
CC-VQA提出了一种无训练、考虑冲突和相关性的方法，通过视觉中心的上下文冲突推理和相关性引导的编码解码，提升基于知识的视觉问答的准确率。
摘要：
基于知识的视觉问答（KB-VQA）在处理知识密集型任务中展现出显著潜力。然而，由于预训练中静态模型知识的限制，视觉语言模型（VLMs）中的静态参数化知识与动态检索信息之间会出现冲突。输出要么忽略检索上下文，要么与参数化知识整合不一致，这对KB-VQA提出了重大挑战。当前的知识冲突缓解方法主要借鉴语言类方法，通过工程提示策略或上下文感知解码机制解决上下文层面的冲突。然而，这些方法忽略了视觉信息在冲突中的关键作用，并且存在冗余的检索上下文，这会损害准确的冲突识别和有效的缓解。为了解决这些局限性，我们提出了CC-VQA：一种新的无训练、考虑冲突和相关性的方法用于KB-VQA。我们的方法包含两个核心组成部分：（1）视觉中心的上下文冲突推理，该方法在内部和外部知识上下文中执行视觉-语义冲突分析；（2）相关性引导的编码和解码，通过位置编码压缩低相关性陈述，并使用相关性加权冲突评分进行自适应解码。在E-VQA、InfoSeek和OK-VQA基准上的广泛评估表明，CC-VQA实现了最先进的性能，与现有方法相比，绝对准确率提高了3.3%至6.4%。代码可在https://github.com/cqu-student/CC-VQA上获得。
英文摘要：
Knowledge-based visual question answering (KB-VQA) demonstrates significant potential for handling knowledge-intensive tasks. However, conflicts arise between static parametric knowledge in vision language models (VLMs) and dynamically retrieved information due to the static model knowledge from pre-training. The outputs either ignore retrieved contexts or exhibit inconsistent integration with parametric knowledge, posing substantial challenges for KB-VQA. Current knowledge conflict mitigation methods primarily adapted from language-based approaches, focusing on context-level conflicts through engineered prompting strategies or context-aware decoding mechanisms. However, these methods neglect the critical role of visual information in conflicts and suffer from redundant retrieved contexts, which impair accurate conflict identification and effective mitigation. To address these limitations, we propose \textbf{CC-VQA}: a novel training-free, conflict- and correlation-aware method for KB-VQA. Our method comprises two core components: (1) Vision-Centric Contextual Conflict Reasoning, which performs visual-semantic conflict analysis across internal and external knowledge contexts; and (2) Correlation-Guided Encoding and Decoding, featuring positional encoding compression for low-correlation statements and adaptive decoding using correlation-weighted conflict scoring. Extensive evaluations on E-VQA, InfoSeek, and OK-VQA benchmarks demonstrate that CC-VQA achieves state-of-the-art performance, yielding absolute accuracy improvements of 3.3\% to 6.4\% compared to existing methods. Code is available at https://github.com/cqu-student/CC-VQA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
