Humans learn to solve tasks of increasing complexity by building on top of previously acquired knowledge. Typically, there exists a natural progression in the tasks that we learn - most do not require completely independent solutions, but can be broken down into simpler subtasks. We propose to represent a solver for each task as a neural module that calls existing modules (solvers for simpler tasks) in a functional program-like manner. Lower modules are a black box to the calling module, and communicate only via a query and an output. Thus, a module for a new task learns to query existing modules and composes their outputs in order to produce its own output. Our model effectively combines previous skill-sets, does not suffer from forgetting, and is fully differentiable. We test our model in learning a set of visual reasoning tasks, and demonstrate improved performances in all tasks by learning progressively. By evaluating the reasoning process using human judges, we show that our model is more interpretable than an attention-based baseline.
X-PCR: A Benchmark for Cross-modality Progressive Clinical Reasoning in Ophthalmic Diagnosis
School of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机科学与软件工程学院);Tsinghua University(清华大学);University of Nottingham(诺丁汉大学);School of AI, Shenzhen University(深圳大学人工智能学院);Wenzhou Medical University(温州医科大学);Guangdong Provincial Key Laboratory of Intelligent Information Processing, Shenzhen University(广东省智能信息处理重点实验室)
Progressive Reasoning with Primitive Correction for Compositional Zero-Shot Learning
School of Computer Science and Technology, Beijing Jiaotong University(北京交通大学计算机科学与技术学院);Key Laboratory of Big Data & Artificial Intelligence in Transportation (Ministry of Education)(交通运输大数据与人工智能教育部重点实验室)
GeoSearcher: Anchor-Guided Progressive Reasoning for Remote Sensing Visual Grounding with Process Supervision
Key Laboratory of Target Cognition and Application Technology (TCAT), Chinese Academy of Sciences(中国科学院目标认知与应用技术重点实验室);School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences(中国科学院大学电子电气与通信工程学院)
Conan: Progressive Learning to Reason Like a Detective over Multi-Scale Visual Evidence
State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机学院,北京大学);WeChat AI, Tencent Inc., China(微信AI,腾讯公司,中国)
Check It Again: Progressive Visual Question Answering via Visual Entailment
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Progressive Multimodal Search and Reasoning for Knowledge-Intensive Visual Question Answering
Interdisciplinary Program in Artificial Intelligence(人工智能交叉学科项目);Department of Intelligence and Information(智能与信息系);Samsung Advanced Institute of Technology(三星先进技术研究院)
3D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selection
Institute of Artificial Intelligence, Beihang University(北京航空航天大学人工智能研究院);Hangzhou Innovation Institute, Beihang University(北京航空航天大学杭州创新研究院);Meituan Inc.(美团公司)
PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models
Tsinghua University(清华大学);OpenGVLab(开放通用实验室);Shanghai AI Laboratory(上海人工智能实验室);SenseTime Research(商汤科技研究院);Nanjing University(南京大学);The Chinese University of Hong Kong(香港中文大学)
STAR-Pro: Stage-Wise Token Adaptive Reduction with Progressive Refinement for Efficient Large Vision-Language Models
Peking University(北京大学);Nanyang Technological University, Singapore(南洋理工大学);University of Electronic Science and Technology of China(电子科技大学);University of Michigan, Ann Arbor(密歇根大学安娜堡分校);De Artificial Intelligence Lab(De人工智能实验室)
Astrea: A MOE-based Visual Understanding Model with Progressive Alignment
Zhejiang University(浙江大学);Southern University of Science and Technology(南方科技大学);Beijing University of Technology(北京工业大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Hong Kong Polytechnic University(香港理工大学);Qingdao University(青岛大学)
ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering
OPPO AI Center, OPPO Inc. China(OPPO AI中心,OPPO公司);The Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Nanyang Technological University, Singapore(新加坡南洋理工大学)
UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学);National University of Singapore(新加坡国立大学);Zhongguancun Academy(中关村学院)
Mixture-of-Visual-Thoughts: Exploring Context-Adaptive Reasoning Mode Selection for General Visual Reasoning
Fudan University(复旦大学);Alibaba Group Holding Limited(阿里巴巴集团控股有限公司);Future Living Lab of Alibaba(阿里巴巴未来生活实验室);University of Southern California(南加州大学);Shanghai Innovation Institute(上海创新研究院)
PhyBlock: A Progressive Benchmark for Physical Understanding and Planning via 3D Block Assembly
Mohamed bin Zayed University of Artificial Intelligence(莫扎德·本·扎耶德人工智能大学);Tsinghua Shenzhen International Graduate School, Tsinghua University, China(清华大学深圳国际研究生院);Shenzhen Campus of Sun Yat-sen University(孙中山大学深圳校区);Spatialtemporal AI(时空人工智能);Shanghai Jiao Tong University(上海交通大学)
LLaVA-UHD v3: Progressive Visual Compression for Efficient Native-Resolution Encoding in MLLMs
Tsinghua University(清华大学);University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院航天信息研究所);School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences(中国科学院大学先进交叉学科学院)
Understanding the computational demands underlying visual reasoning
Artificial and Natural Intelligence Toulouse Institute(图卢兹人工智能与自然智能研究所);Université de Toulouse(图卢兹大学);Centre de Recherche Cerveau & Cognition (CerCo), CNRS(法国国家科学研究中心大脑与认知研究中心)
MedLVR: Latent Visual Reasoning for Reliable Medical Visual Question Answering
Department of Radiation Oncology and Winship Cancer Institute, Emory University School of Medicine(埃默里大学医学院放射肿瘤学系与温希普癌症研究所);Department of Biostatistics and Bioinformatics, Emory University(埃默里大学生物统计学与生物信息学系)