Despite considerable recent progress in Visual Question Answering (VQA) models, inconsistent or contradictory answers continue to cast doubt on their true reasoning capabilities. However, most proposed methods use indirect strategies or strong assumptions on pairs of questions and answers to enforce model consistency. Instead, we propose a novel strategy intended to improve model performance by directly reducing logical inconsistencies. To do this, we introduce a new consistency loss term that can be used by a wide range of the VQA models and which relies on knowing the logical relation between pairs of questions and answers. While such information is typically not available in VQA datasets, we propose to infer these logical relations using a dedicated language model and use these in our proposed consistency loss function. We conduct extensive experiments on the VQA Introspect and DME datasets and show that our method brings improvements to state-of-the-art VQA models, while being robust across different architectures and settings.
VQA-LOL: Visual Question Answering under the Lens of Logic
Arizona State University（亚利桑那州立大学）
逻辑连接词及其对自然语言句子意义的影响是理解的一个基本方面。在本文中，我们调查是否视觉问答（VQA）系统训练回答一个问题的图像，能够回答多个这样的问题的逻辑组成。当置于此\textit{Lens of Logic}下时，最先进的VQA模型难以正确回答这些逻辑组成的问题。我们构建了一个增强的VQA数据集作为基准，问题包含逻辑成分和语言转换（否定，析取，连词和反义词）。我们提出了我们的{逻辑透镜（LOL）}模型，该模型使用问题注意和逻辑注意来理解问题中的逻辑连接词，并提出了一种新的Fréchet相容性损失，它确保了组成问题和组合问题的答案与推断的逻辑操作一致。我们的模型在学习逻辑成分的同时保持VQA的性能有了很大的提高。我们建议这项工作是通过在视觉理解中嵌入逻辑连接词来实现鲁棒性。
GeoReason: Aligning Thinking And Answering In Remote Sensing Vision-Language Models Via Logical Consistency Reinforcement Learning
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航空信息研究所）；Key Laboratory of Target Cognition and Application Technology, Chinese Academy of Sciences（中国科学院目标认知与应用技术重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models
Zhejiang University（浙江大学）；The University of Sydney（悉尼大学）；ManyCore；State Key Laboratory of Blockchain and Security, Zhejiang University（浙江大学区块链与安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院）
FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts
Mercer Mettl（默瑟梅特尔）；IGDTUW（英迪拉·甘地德里技术大学（女子））；Google Research（谷歌研究院）；Indian Institute of Technology Guwahati（古瓦哈蒂印度理工学院）；University of Pennsylvania（宾夕法尼亚大学）
Beyond Geometry: Benchmarking and Consistency Reasoning for 3D Logical Anomaly Detection
School of Artificial Intelligence and Robotics, Hunan University（湖南大学人工智能与机器人学院）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（湖南大学机器人视觉感知与控制技术国家工程研究中心）
