Spatial reasoning is a cornerstone capability for intelligent systems to perceive and interact with the physical world. However, multimodal large language models (MLLMs) frequently suffer from hallucinations and imprecision when parsing complex geometric layouts. As data-driven scaling struggles to internalize structured geometric priors and spatial constraints, integrating mature, specialized vision models presents a compelling alternative. Despite its promise, applying this paradigm to spatial reasoning is hindered by two key challenges: The difficulty of invoking heterogeneous, parameter-rich tools, as well as the challenge of understanding and effectively leveraging their diverse low-level outputs (e.g., segmentation masks, depth maps) in high-level reasoning. To address these challenges, we propose LAST, a unified framework for tool-augmented spatial reasoning. LAST features an extensible interactive sandbox, termed LAST-Box, which abstracts heterogeneous tool invocations into atomic instructions and reusable spatial skills, returning multimodal hints (e.g., annotated images and textual descriptions) that can be directly consumed by LLMs. We further design a three-stage progressive training strategy that guides models from understanding tool outputs to proficient and adaptive tool invocation. Experiments on four datasets show that LAST-7B achieves around 20\% performance gains over its backbone and outperforms strong proprietary closed-source LLMs, substantially enhancing reasoning on complex spatial tasks.
Teaching the Way, Not the Answer: Privileged Tutoring Distillation for Multimodal Policy Optimization
Tianjin University(天津大学);Beijing Institute of Technology(北京理工大学);Singapore Management University(新加坡国立大学);University of Chinese Academy of Sciences(中国科学院大学);Xiaomi Inc(小米公司)
CheXthought: A global multimodal dataset of clinical chain-of-thought reasoning and visual attention for chest X-ray interpretation
Center for Artificial Intelligence in Medicine and Imaging(医学与影像人工智能中心);Department of Radiology(放射科);Department of Pediatrics(儿科);Department of Radiology, Weill Cornell Medicine(韦氏 Cornell 医学院放射科);Department of Radiology, Brigham and Women’s Hospital(布里奇妇产科医院放射科);University of California, Berkeley(加州大学伯克利分校);University of California, San Francisco(加州大学旧金山分校);Department of Biomedical Data Science(生物医学数据科学部)
How to Bridge the Gap between Modalities: Survey on Multimodal Large Language Model
College of Computer, National University of Defense Technology(国防科技大学计算机学院);School of Computer and Information Engineering, Hefei University of Technology(合肥工业大学计算机与信息工程学院)
IoU-PD: IoU-Aware Privileged Distillation for Visual Grounding with Multimodal Large Language Models
University of Chinese Academy of Sciences(中国科学院大学);State Key Laboratory of Communication Content Cognition(通信内容认知技术国家重点实验室);Peng Cheng Laboratory(鹏城实验室)
Multimodal Deep Learning for Scientific Imaging Interpretation
Science and Technology Division, Corning Incorporated(康宁公司科技事业部);Robert Frederick Smith School of Chemical and Biomolecular Engineering, Cornell University(康奈尔大学罗伯特·弗雷德里克·史密斯化学与生物分子工程学院)
本文提出多模态深度学习框架 GlassLLaVA,结合论文图文数据与 GPT-4,实现玻璃材料 SEM 图像的类人解释、特征识别和缺陷检测,并建立可对照研究答案的评估指标。
Multimodal Chain of Continuous Thought for Latent-Space Reasoning in Vision-Language Models
Harvard Medical School, Harvard University(哈佛医学院、哈佛大学);Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital(阿提尼乌拉A.马丁努斯生物医学成像中心、麻省总医院);Knovel Engineering Lab, Singapore(Knovel工程实验室、新加坡)
LLMs Meet Multimodal Generation and Editing: A Survey
The Hong Kong University of Science and Technology(香港科技大学);The Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学);Tencent AI Lab(腾讯AI实验室)
Radiology's Last Exam (RadLE): Benchmarking Frontier Multimodal AI Against Human Experts and a Taxonomy of Visual Reasoning Errors in Radiology
Centre for Responsible Autonomous Systems in Healthcare (CRASH) Lab, Koita Centre for Digital Health(负责任的自主医疗系统中心(CRASH)实验室、Koita数字健康中心);Ashoka University(阿什oka大学);Independent Researcher(独立研究者);Koita Centre for Digital Health(Koita数字健康中心)
REALM: RAG-Driven Enhancement of Multimodal Electronic Health Records Analysis via Large Language Models
Institute of Artificial Intelligence, Beihang University(北京航空航天大学人工智能研究院);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);School of Software, Beihang University(北京航空航天大学软件学院);China Mobile Research Institute(中国移动研究院)
Autonomous Workflow for Multimodal Fine-Grained Training Assistants Towards Mixed Reality
Centrum Wiskunde & Informatica(数学与计算机科学中心);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);University College London(伦敦大学学院);Shandong University(山东大学);TU Delft(代尔夫特理工大学)
Unlocking the Potential of Difficulty Prior in RL-based Multimodal Reasoning
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);NLPR&MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Peking University(北京大学);Zhongguancun Academy(中关村学院)
Octopus: History-Free Gradient Orthogonalization for Continual Learning in Multimodal Large Language Models
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University(人工智能大模型关键实验室,人工智能研究院,上海交通大学);vivo Mobile Communication Co., Ltd.(vivo移动通信有限公司)