Artificial Intelligence (AI) and its applications have sparked extraordinary interest in recent years. This achievement can be ascribed in part to advances in AI subfields including Machine Learning (ML), Computer Vision (CV), and Natural Language Processing (NLP). Deep learning, a sub-field of machine learning that employs artificial neural network concepts, has enabled the most rapid growth in these domains. The integration of vision and language has sparked a lot of attention as a result of this. The tasks have been created in such a way that they properly exemplify the concepts of deep learning. In this review paper, we provide a thorough and an extensive review of the state of the arts approaches, key models design principles and discuss existing datasets, methods, their problem formulation and evaluation measures for VQA and Visual reasoning tasks to understand vision and language representation learning. We also present some potential future paths in this field of research, with the hope that our study may generate new ideas and novel approaches to handle existing difficulties and develop new applications.
Knowledge Condensation and Reasoning for Knowledge-based VQA
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Kuaishou(快手)
M$^3$-VQA: A Benchmark for Multimodal, Multi-Entity, Multi-Hop Visual Question Answering
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Tri-VQA: Triangular Reasoning Medical Visual Question Answering for Multi-Attribute Analysis
School of Computing;Artificial Intelligence Southwest Jiaotong University Chengdu, China;Artificial Intelligence Southwest Jiaotong University Chengdu, China z\ c\;Institute of Innovative Research Tokyo Institute of Technology Yokohama, Japan
MedXplain-VQA: Multi-Component Explainable Medical Visual Question Answering
Faculty Of Information Technology VNU University of Engineering(信息科技学院越南工程大学);IT-BT Convergence Technology Division Vietnam-Korea Institute of Science(IT-BT融合技术部门越南-韩国科学技术院);TADI Global Lab TADI Global Company Limited(TADI全球实验室TADI全球有限公司);Faculty of Finance Banking Academy of Vietnam(金融学院越南银行学院)
Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering
School of computer science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院);South China Normal University(华南师范大学);Northeastern University(东北大学);Anhui University(安徽大学)
MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoning
University of Pennsylvania(宾夕法尼亚大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);UC Santa Cruz(加州大学圣克鲁兹分校);Harvard University(哈佛大学);UC Berkeley(加州大学伯克利分校);Emory University(埃默里大学);UC San Francisco(旧金山加州大学)
Adaptive Reinforcement for Open-ended Medical Reasoning via Semantic-Guided Reward Collapse Mitigation
College of Intelligent Robotics and Advanced Manufacturing, Fudan University(复旦大学智能机器人与先进制造学院);Fysics Intelligence Technologies Co., Ltd. (Fysics AI)(菲斯克智能科技有限公司(菲斯克AI));Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning
The University of Texas MD Anderson Cancer Center(德克萨斯大学MD安德森癌症中心);Cohere Labs(Cohere实验室);Eisai Inc.(艾伯维公司);Indian Institute of Science, Bangalore(班加罗尔印度科学研究院)
HCNQA: Enhancing 3D VQA with Hierarchical Concentration Narrowing Supervision
Southern University of Science and Technology, Shenzhen, China(南方科技大学,深圳,中国);College of William and Mary, Williamsburg, VA 23185, USA(威廉玛丽学院,威廉斯堡,VA 23185,美国);Peng Cheng Laboratory, Shenzhen, China(鹏城实验室,深圳,中国)
MC-CoT: A Modular Collaborative CoT Framework for Zero-shot Medical-VQA with LLM and MLLM Integration
Huazhong University of Science and Technology(华中科技大学);Eastern Institute of Technology(宁波工程学院);Purple Mountain Laboratories(紫金山实验室);Alibaba Inc.(阿里巴巴集团);Fudan University(复旦大学)
Kvasir-VQA-x1: A Multimodal Dataset for Medical Reasoning and Robust MedVQA in Gastrointestinal Endoscopy
Simula Metropolitan Center for Digital Engineering (SimulaMet)(Simula Metropolitan Center for Digital Engineering);Oslo Metropolitan University (OsloMet)(Oslo Metropolitan University);Simula Research Laboratory
R^3-VQA: "Read the Room" by Video Social Reasoning
School of Intelligence Science and Technology, Peking University(北京理工大学智能科学与技术学院);College of Artificial Intelligence, Xi’an Jiaotong University(西安交通大学人工智能学院);State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence(一般人工智能国家重点实验室);Yuanpei College, Peking University(北京大学元培学院);Tsinghua University(清华大学);University of California, Los Angeles(加州大学洛杉矶分校);MIT-IBM Watson AI Lab(MIT-IBM沃森人工智能实验室)