Vision-Language Models (VLMs) have achieved remarkable progress in integrating visual perception with language understanding. However, effective multimodal reasoning requires both accurate perception and robust reasoning, and weakness in either limits the performance of VLMs. Prior efforts to enhance reasoning often depend on high-quality chain-of-thought (CoT) data, obtained via labor-intensive human annotations, costly proprietary models, or self-training methods that overlook perception. To address these limitations, we propose a simple yet effective self-training framework called See-Think-Learn (STL). At its core, STL introduces a structured reasoning template that encourages the model to see before thinking, first extracting visual attributes in textual form, then using them to guide reasoning. The framework jointly improves perception and reasoning by having the model generate and learn from its own structured rationales in a self-training loop. Furthermore, we augment the training data with negative rationales, i.e. explanations that justify why certain answer choices are incorrect, to enhance the model's ability to distinguish between correct and misleading responses. This fosters more discriminative and robust learning. Experiments across diverse domains show that STL consistently outperforms baselines trained directly only on answers or self-generated reasoning, while qualitative analysis confirms the high quality of its rationales. STL thus provides a cost-effective solution to enhance multimodal reasoning ability of VLMs.
Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning
Wuhan University（武汉大学）；Xiaomi Inc（小米公司）；Wuhan University of Technology（武汉理工大学）；Nanyang Technological University（南洋理工大学）；The Hong Kong University of Science and Technology（香港科技大学）
MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；UMass Amherst（马萨诸塞大学阿默斯特分校）；Tsinghua University（清华大学）
Large pre-trained vision and language models have demonstrated remarkable capacities for various tasks. However, solving the knowledge-based visual reasoning tasks remains challenging, which requires a model to comprehensively understand image content, connect the external world knowledge, and perform step-by-step reasoning to answer the questions correctly. To this end, we propose a novel framework named Interactive Prompting Visual Reasoner (IPVR) for few-shot knowledge-based visual reasoning. IPVR contains three stages, see, think and confirm. The see stage scans the image and grounds the v…
Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Alibaba Group（阿里巴巴集团）；DAMO Academy, Alibaba Group（阿里巴巴集团大模型学院）；Nanjing Institute of Software Technology（南京软件技术研究所）；Nanjing University of Posts and Telecommunications（南京邮电大学）；Hohai University（河海大学）
University of Nottingham Ningbo China（宁波诺丁汉大学）；Nanyang Technological University（南洋理工大学）
Raven's Progressive Matrices (RPMs) are frequently used in evaluating human's visual reasoning ability. Researchers have made considerable efforts in developing systems to automatically solve the RPM problem, often through a black-box end-to-end convolutional neural network for both visual recognition and logical reasoning tasks. Based on the two intrinsic natures of RPM problem, visual recognition and logical reasoning, we propose a Two-stage Rule-Induction Visual Reasoner (TRIVR), which consists of a perception module and a reasoning module, to tackle the challenges of real-world visual reco…
Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm
Institute of Trustworthy Embodied AI, Fudan University（复旦大学可信具身智能研究所）；Shanghai Innovation Institute（上海创新研究院）；Shanghai Key Laboratory of Multimodal Embodied AI（上海市多模态具身人工智能重点实验室）；College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院）；The Chinese University of Hong Kong（香港中文大学）；Central South University（中南大学）；Fudan University（复旦大学）
Thinking Without Images: Internalizing Visual Manipulation with On-Policy Self-Distillation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（北京大学计算机学院多媒体信息处理国家重点实验室）；Central South University（中南大学）；University of Science and Technology of China（中国科学技术大学）；Peking University（北京大学）；Huawei Technologies（华为技术有限公司）
