Visual Commonsense Reasoning (VCR) remains a significant yet challenging research problem in the realm of visual reasoning. A VCR model generally aims at answering a textual question regarding an image, followed by the rationale prediction for the preceding answering process. Though these two processes are sequential and intertwined, existing methods always consider them as two independent matching-based instances. They, therefore, ignore the pivotal relationship between the two processes, leading to sub-optimal model performance. This paper presents a novel visual attention alignment method to efficaciously handle these two processes in a unified framework. To achieve this, we first design a re-attention module for aggregating the vision attention map produced in each process. Thereafter, the resultant two sets of attention maps are carefully aligned to guide the two processes to make decisions based on the same image regions. We apply this method to both conventional attention and the recent Transformer models and carry out extensive experiments on the VCR benchmark dataset. The results demonstrate that with the attention alignment module, our method achieves a considerable improvement over the baseline methods, evidently revealing the feasibility of the coupling of the two processes as well as the effectiveness of the proposed method.
Cognitive Visual Commonsense Reasoning Using Dynamic Working Memory
Leibniz University of Hannover（汉诺威莱布尼茨大学）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；China Europe International Business School（中欧国际工商学院）；Auburn University（奥本大学）；Guangdong Pharmaceutical University（广东药科大学）；University of Southern Queensland（南昆士兰大学）
Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs
Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；Paul G. Allen School of Computer Science & Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）
School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件学院）
ReasonCLIP-58M: Visually Grounded Commonsense Reasoning Supervision for CLIP
Khalifa University（卡利法大学）；University of Western Australia（西澳大学）；The Chinese University of Hong Kong（香港中文大学）；University of Melbourne（墨尔本大学）；University of Central Florida（佛罗里达中央大学）
