Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering
School of computer science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；South China Normal University（华南师范大学）；Northeastern University（东北大学）；Anhui University（安徽大学）
Recently, to comprehensively improve Vision Language Models (VLMs) for Visual Question Answering (VQA), several methods have been proposed to further reinforce the inference capabilities of VLMs to independently tackle VQA tasks rather than some methods that only utilize VLMs as aids to Large Language Models (LLMs). However, these methods ignore the rich common-sense knowledge inside the given VQA image sampled from the real world. Thus, they cannot fully use the powerful VLM for the given VQA question to achieve optimal performance. Attempt to overcome this limitation and inspired by the human top-down reasoning process, i.e., systematically exploring relevant issues to derive a comprehensive answer, this work introduces a novel, explainable multi-agent collaboration framework by leveraging the expansive knowledge of Large Language Models (LLMs) to enhance the capabilities of VLMs themselves. Specifically, our framework comprises three agents, i.e., Responder, Seeker, and Integrator, to collaboratively answer the given VQA question by seeking its relevant issues and generating the final answer in such a top-down reasoning process. The VLM-based Responder agent generates the answer candidates for the question and responds to other relevant issues. The Seeker agent, primarily based on LLM, identifies relevant issues related to the question to inform the Responder agent and constructs a Multi-View Knowledge Base (MVKB) for the given visual scene by leveraging the build-in world knowledge of LLM. The Integrator agent combines knowledge from the Seeker agent and the Responder agent to produce the final VQA answer. Extensive and comprehensive evaluations on diverse VQA datasets with a variety of VLMs demonstrate the superior performance and interpretability of our framework over the baseline method in the zero-shot setting without extra training cost.
VinDr-CXR-VQA: A Visual Question Answering Dataset for Explainable Chest X-Ray Analysis with Multi-Task Learning
College of Engineering and Computer Science（工程与计算机科学学院）；VinUniversity（Vin大学）；VinUni-Illinois Smart Health Center（VinUni-伊利诺伊智能健康中心）；Vietnam National Cancer Hospital（越南国家癌症医院）；The Computer Vision and Medical AI Lab（计算机视觉与医学人工智能实验室）
Towards Self-Explainable Document Visual Question Answering with Chain-of-Explanation Predictions
Department of Informatics, University of Oslo（奥斯陆大学信息学院）；Integreat – Norwegian Centre for Knowledge-driven Machine Learning（挪威知识驱动机器学习中心）；TRUST – The Norwegian Centre for Trustworthy AI（挪威可信人工智能中心）
MedXplain-VQA: Multi-Component Explainable Medical Visual Question Answering
Faculty Of Information Technology VNU University of Engineering（信息科技学院越南工程大学）；IT-BT Convergence Technology Division Vietnam-Korea Institute of Science（IT-BT融合技术部门越南-韩国科学技术院）；TADI Global Lab TADI Global Company Limited（TADI全球实验室TADI全球有限公司）；Faculty of Finance Banking Academy of Vietnam（金融学院越南银行学院）
Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions
University of Science and Technology of China（中国科学技术大学）；Microsoft Research, Beijing, China（微软研究院（北京，中国））；University of Rochester, Rochester, NY（罗切斯特大学（罗切斯特，纽约州））
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Australian National University（澳大利亚国立大学）；JD AI Research（京东人工智能研究院）；Microsoft Research（微软研究院）；University of Adelaide（阿德莱德大学）；Macquarie University（麦考瑞大学）
VietMEAgent: Culturally-Aware Few-Shot Multimodal Explanation for Vietnamese Visual Question Answering
Faculty Of Information Technology, VNU University of Engineering and Technology（信息科技学院，越南工程与技术大学）；IT-BT Convergence Technology Division, Vietnam-Korea Institute of Science and Technology（IT-BT融合技术部，越南-韩国科学技术院）；TADI Global Lab, TADI Global Company Limited（TADI全球实验室，TADI全球公司）；Faculty of Finance, Banking Academy of Vietnam（金融学院，越南银行学院）
Medico 2025: Visual Question Answering for Gastrointestinal Imaging
SimulaMet - Simula Metropolitan Center for Digital Engineering, Oslo, Norway（SimulaMet - Simula Metropolitan Center for Digital Engineering，挪威奥斯陆）；Simula Research Laboratory, Oslo, Norway（Simula研究实验室，挪威奥斯陆）；OsloMet - Oslo Metropolitan University, Oslo, Norway（OsloMet - 奥斯陆 Metropolitan 大学，挪威奥斯陆）
MuKEA: Multimodal Knowledge Extraction and Accumulation for Knowledge-based Visual Question Answering
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Université de Montréal（蒙特利尔大学）；University of Adelaide（阿德莱德大学）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）
GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosis
The Hong Kong Polytechnic University（香港理工大学）；National University of Singapore（新加坡国立大学）；Sichuan University（四川大学）；West China Hospital of Sichuan University（四川大学华西医院）；Institute of High Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究所）
Counterfactual Samples Synthesizing and Training for Robust Visual Question Answering
The Hong Kong University of Science and Technology（香港科技大学）；Zhejiang University（浙江大学）；Columbia University（哥伦比亚大学）；Nanyang Technological University（南洋理工大学）
ChainReaction: Causal Chain-Guided Reasoning for Modular and Explainable Causal-Why Video Question Answering
Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore（新加坡高性能计算研究所，科技研究局）；Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore（科技研究局前沿人工智能研究中心，新加坡）；College of Computing and Data Science, Nanyang Technological University, Singapore（新加坡南洋理工大学计算与数据科学学院）
DualVGR: A Dual-Visual Graph Reasoning Unit for Video Question Answering
Nanjing University of Posts and Telecommunications（南京邮电大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（电子与计算机工程系，香港科学与技术大学）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（计算机科学与工程系，香港科学与技术大学）
MemoryCard: Topic-Aware Multi-Modal Clue Compression for Long-Video Question Answering
School of Computer Science and Engineering, Northeastern University（东北大学计算机科学与工程学院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；Digital China Group（数字中国集团）
