面向可解释视觉问答的检索增强自然语言推理
Retrieval-Augmented Natural Language Reasoning for Explainable Visual Question Answering
AI总结：
本文提出ReRe模型，通过检索增强自然语言推理，在不依赖复杂网络和额外数据集的情况下提升视觉问答准确率与解释质量。
摘要：
带自然语言解释的视觉问答（VQA-NLE）任务因其对基于推理的推断的高要求而具有挑战性。近期的VQA-NLE研究侧重于增强模型网络以提升模型的推理能力，但这种方法资源消耗大且不稳定。在本工作中，我们提出了一种新的VQA-NLE模型ReRe（检索增强自然语言推理），利用来自记忆的检索信息来辅助生成准确的答案和有说服力的解释，而无需依赖复杂网络和额外数据集。ReRe是一种编码器-解码器架构模型，使用预训练的CLIP视觉编码器和预训练的GPT-2语言模型作为解码器。在GPT-2中加入了交叉注意力层以处理检索特征。ReRe在VQA准确率和解释得分上超越了先前方法，并在NLE中表现出更具说服力和可靠性的改进。
英文摘要：
Visual Question Answering with Natural Language Explanation (VQA-NLE) task is challenging due to its high demand for reasoning-based inference. Recent VQA-NLE studies focus on enhancing model networks to amplify the model's reasoning capability but this approach is resource-consuming and unstable. In this work, we introduce a new VQA-NLE model, ReRe (Retrieval-augmented natural language Reasoning), using leverage retrieval information from the memory to aid in generating accurate answers and persuasive explanations without relying on complex networks and extra datasets. ReRe is an encoder-decoder architecture model using a pre-trained clip vision encoder and a pre-trained GPT-2 language model as a decoder. Cross-attention layers are added in the GPT-2 for processing retrieval features. ReRe outperforms previous methods in VQA accuracy and explanation score and shows improvement in NLE with more persuasive, reliability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
