Achieving artificial visual reasoning - the ability to answer image-related questions which require a multi-step, high-level process - is an important step towards artificial general intelligence. This multi-modal task requires learning a question-dependent, structured reasoning process over images from language. Standard deep learning approaches tend to exploit biases in the data rather than learn this underlying structure, while leading methods learn to visually reason successfully but are hand-crafted for reasoning. We show that a general-purpose, Conditional Batch Normalization approach achieves state-of-the-art results on the CLEVR Visual Reasoning benchmark with a 2.4% error rate. We outperform the next best end-to-end method (4.5%) and even methods that use extra supervision (3.1%). We probe our model to shed light on how it reasons, showing it has learned a question-dependent, multi-step process. Previous work has operated under the assumption that visual reasoning calls for a specialized architecture, but we show that a general architecture with proper conditioning can learn to visually reason effectively.
UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学);National University of Singapore(新加坡国立大学);Zhongguancun Academy(中关村学院)
Mixture-of-Visual-Thoughts: Exploring Context-Adaptive Reasoning Mode Selection for General Visual Reasoning
Fudan University(复旦大学);Alibaba Group Holding Limited(阿里巴巴集团控股有限公司);Future Living Lab of Alibaba(阿里巴巴未来生活实验室);University of Southern California(南加州大学);Shanghai Innovation Institute(上海创新研究院)
MoLE: Mixture of Latent Experts for Complementary Visual Reasoning
Tongji University(同济大学);Hong Kong Polytechnic University(香港理工大学);Alibaba Group(阿里巴巴集团);University of Electronic Science and Technology of China(电子科技大学)
Zero-Shot Visual Reasoning by Vision-Language Models: Benchmarking and Analysis
ASUS Intelligent Cloud Services (AICS)(华硕智能云服务(AICS));ASUS Global Pte. Ltd.(华硕全球私人有限公司);Centre for Frontier AI Research (CFAR)(前沿人工智能研究中心(CFAR));Institute of High Performance Computing (IHPC)(高性能计算研究所(IHPC));Agency for Science, Technology and Research (A*STAR)(新加坡科学、技术与研究局(A*STAR))
CLiViS: Unleashing Cognitive Map through Linguistic-Visual Synergy for Embodied Visual Reasoning
School of Computer Science and Technology, East China Normal University(东华大学计算机科学与技术学院);King Abdullah University of Science and Technology(科廷大学);Fudan University(复旦大学)
Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models
School of Engineering, Westlake University(西湖大学工程学院);City University of Hong Kong(香港城市大学);Research Center for Industries of the Future, Westlake University(西湖大学未来产业研究中心);Johns Hopkins University(约翰斯·霍普金斯大学)
Understanding the computational demands underlying visual reasoning
Artificial and Natural Intelligence Toulouse Institute(图卢兹人工智能与自然智能研究所);Université de Toulouse(图卢兹大学);Centre de Recherche Cerveau & Cognition (CerCo), CNRS(法国国家科学研究中心大脑与认知研究中心)
Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所);Research Center for Brain-inspired Intelligence, CASIA(中国科学院自动化研究所类脑智能研究中心);University of Chinese Academy of Sciences(中国科学院大学);Center for Excellence in Brain Science and Intelligence Technology, CAS(中国科学院脑科学与智能技术卓越创新中心)
MedLVR: Latent Visual Reasoning for Reliable Medical Visual Question Answering
Department of Radiation Oncology and Winship Cancer Institute, Emory University School of Medicine(埃默里大学医学院放射肿瘤学系与温希普癌症研究所);Department of Biostatistics and Bioinformatics, Emory University(埃默里大学生物统计学与生物信息学系)
ReasonCLIP-58M: Visually Grounded Commonsense Reasoning Supervision for CLIP
Khalifa University(卡利法大学);University of Western Australia(西澳大学);The Chinese University of Hong Kong(香港中文大学);University of Melbourne(墨尔本大学);University of Central Florida(佛罗里达中央大学)
Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Models
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Wuhan AI Research(武汉人工智能研究院)
IVR-R1: Refining Trajectories through Iterative Visual-Grounded Reasoning in Reinforcement Learning
Hangzhou International Innovation Institute, Beihang University(北京航空航天大学杭州国际创新研究院);School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院);Kuaishou Technology(快手科技);Shenzhen Institute of Advanced Integration Technology, Shenzhen(深圳先进集成技术研究院)
Towards Unsupervised Visual Reasoning: Do Off-The-Shelf Features Know How to Reason?
Warsaw University of Technology(华沙工业大学);LIGM, Ecole des Ponts, Univ Gustave Eiffel(古斯塔夫·埃菲尔大学桥路学院LIGM实验室);Tooploox(Tooploox公司);IDEAS NCBR(IDEAS国家研究与发展中心);Jagiellonian University of Cracow(克拉科夫雅盖隆大学)
VisRAG2.0: Mitigating Visual Hallucinations via Evidence-Guided Multi-Image Reasoning in Visual Retrieval-Augmented Generation
School of Software and Microelectronics, Peking University, China(北京大学软件与微电子学院);School of Computer Science and Engineering, Northeastern University, China(东北大学计算机科学与工程学院);Department of Computer Science and Technology, Institute for AI, Tsinghua University, China(清华大学人工智能研究院计算机科学与技术系)
Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs
Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Paul G. Allen School of Computer Science & Engineering, University of Washington(华盛顿大学保罗·G·艾伦计算机科学与工程学院)
Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoning
School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室);Tencent(腾讯);Center for Language AI Research, Tohoku University(东北大学语言人工智能研究中心);RIKEN Center for Advanced Intelligence Project(日本理化学研究所先进人工智能项目中心)
LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning
School of AI, Shanghai Jiao Tong University(上海交通大学人工智能学院);Kling Team, Kuaishou Technology(快手科技 Kling 团队);Peking University(北京大学);HKUST(香港科技大学);CASIA(中国科学院自动化研究所);Nanjing University(南京大学);Renmin University of China(中国人民大学);Tsinghua University(清华大学)
本文提出LatentOmni框架,通过统一的音频-视觉潜在空间进行多模态推理,利用特征级监督和Omni-Sync Position Embedding保持时间一致性,从而在多个音频-视觉推理基准测试中取得最佳性能。