视觉大模型 / VLM

SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planning

Yuecheng Liu, Dafeng Chi, Shiguang Wu, Zhanguang Zhang, Yaochen Hu, Lingfeng Zhang, Yingxue Zhang, Shuang Wu, Tongtong Cao, Guowei Huang, Helong Huang, Guangjian Tian, Weichao Qiu, Xingyue Quan, Jianye Hao, Yuzheng Zhuang

专题命中 视觉推理 ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments Under Review

JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images

Zhecan Wang, Junzhang Liu, Chia-Wei Tang, Hani Alomari, Anushka Sivakumar, Rui Sun, Wenhao Li, Md. Atabuzzaman, Hammad Ayyubi, Haoxuan You, Alvi Ishmam, Kai-Wei Chang, Shih-Fu Chang, Chris Thomas

专题命中 视觉推理 ：visual reasoning(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Socratic Questioning: Learn to Self-guide Multimodal Reasoning in the Wild

Wanpeng Hu, Haodi Liu, Lin Chen, Feng Zhou, Changming Xiao, Qi Yang, Changshui Zhang

专题命中 视觉推理 ：visual reasoning(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

PhotoBot: Reference-Guided Interactive Photography via Natural Language

Oliver Limoyo, Jimmy Li, Dmitriy Rivkin, Jonathan Kelly, Gregory Dudek

专题命中 视觉推理 ：VLM(abstract)；visual language model(abstract)；分类 cs.CV、cs.AI

Comments In Proceedings of the IEEE/RSJ International Conference on Intelligent Robotics and Systems (IROS'24), Abu Dhabi, UAE, Oct. 14-18, 2024

Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning

Qi Sun, Pengfei Hong, Tej Deep Pala, Vernon Toh, U-Xuan Tan, Deepanway Ghosal, Soujanya Poria

专题命中 视觉推理 ：visual language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments https://github.com/declare-lab/Emma-X, https://huggingface.co/declare-lab/Emma-X

NL-Eye: Abductive NLI for Images

Mor Ventura, Michael Toker, Nitay Calderon, Zorik Gekhman, Yonatan Bitton, Roi Reichart

专题命中 视觉推理 ：VLM(abstract)；visual language model(abstract)；分类 cs.CV、cs.AI

How Far Are We from Intelligent Visual Deductive Reasoning?

Yizhe Zhang, He Bai, Ruixiang Zhang, Jiatao Gu, Shuangfei Zhai, Josh Susskind, Navdeep Jaitly

专题命中 视觉推理 ：vision-language model(abstract)；visual reasoning(abstract)；分类 cs.CV、cs.AI

Comments COLM 2024. https://github.com/apple/ml-rpm-bench

CadVLM: Bridging Language and Vision in the Generation of Parametric CAD Sketches

Sifan Wu, Amir Khasahmadi, Mor Katz, Pradeep Kumar Jayaraman, Yewen Pu, Karl Willis, Bang Liu

专题命中 视觉推理 ：vision language model(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

NEUSIS: A Compositional Neuro-Symbolic Framework for Autonomous Perception, Reasoning, and Planning in Complex UAV Search Missions

Zhixi Cai, Cristian Rojas Cardenas, Kevin Leo, Chenyuan Zhang, Kal Backman, Hanbing Li, Boying Li, Mahsa Ghorbanali, Stavya Datta, Lizhen Qu, Julian Gutierrez Santiago, Alexey Ignatiev, Yuan-Fang Li, Mor Vered, Peter J Stuckey, Maria Garcia de la Banda, Hamid Rezatofighi

专题命中 视觉推理 ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulation

Jin Wang, Rui Dai, Weijie Wang, Luca Rossini, Francesco Ruscelli, Nikos Tsagarakis

专题命中 视觉推理 ：VLM(abstract)；visual language model(abstract)；分类 cs.AI、cs.LG

Comments Project page: https://hy-motion.github.io/

X-Former: Unifying Contrastive and Reconstruction Learning for MLLMs

Sirnam Swetha, Jinyu Yang, Tal Neiman, Mamshad Nayeem Rizve, Son Tran, Benjamin Yao, Trishul Chilimbi, Mubarak Shah

专题命中 视觉推理 ：visual reasoning(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted at ECCV2024

Whiteboard-of-Thought: Thinking Step-by-Step Across Modalities

Sachit Menon, Richard Zemel, Carl Vondrick

专题命中 视觉推理 ：visual reasoning(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Project website: whiteboard.cs.columbia.edu/

RS-GPT4V: A Unified Multimodal Instruction-Following Dataset for Remote Sensing Image Understanding

Linrui Xu, Ling Zhao, Wang Guo, Qiujun Li, Kewang Long, Kaiqi Zou, Yuhan Wang, Haifeng Li

专题命中 视觉推理 ：visual reasoning(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 6 figures, 4 tables

Conceptual Learning via Embedding Approximations for Reinforcing Interpretability and Transparency

Maor Dikter, Tsachi Blau, Chaim Baskin

专题命中 视觉推理 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation data

Chenhui Zhang, Sherrie Wang

专题命中 视觉推理 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 62 pages; work in progress

A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertise

Chaoyou Fu, Renrui Zhang, Zihan Wang, Yubo Huang, Zhengye Zhang, Longtian Qiu, Gaoxiang Ye, Yunhang Shen, Mengdan Zhang, Peixian Chen, Sirui Zhao, Shaohui Lin, Deqiang Jiang, Di Yin, Peng Gao, Ke Li, Hongsheng Li, Xing Sun

专题命中 视觉推理 ：visual reasoning(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Total 120 pages. See our project at https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models

Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyond

Liang Chen, Yichi Zhang, Shuhuai Ren, Haozhe Zhao, Zefan Cai, Yuchi Wang, Peiyi Wang, Tianyu Liu, Baobao Chang

专题命中 视觉推理 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments FMDM@NeurIPS2023, Code and data: https://github.com/pkunlp-icler/PCA-EVAL/

ViLLA: Fine-Grained Vision-Language Representation Learning from Real-World Data

Maya Varma, Jean-Benoit Delbrouck, Sarah Hooper, Akshay Chaudhari, Curtis Langlotz

专题命中 视觉推理 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments ICCV 2023

Preconditioned Visual Language Inference with Weak Supervision

Ehsan Qasemi, Amani R. Maina-Kilaas, Devadutta Dash, Khalid Alsaggaf, Muhao Chen

专题命中 视觉推理 ：VLM(abstract)；visual language model(abstract)；分类 cs.CV、cs.AI

TAB-VCR: Tags and Attributes based Visual Commonsense Reasoning Baselines

Jingxiang Lin, Unnat Jain, Alexander G. Schwing

专题命中 视觉推理 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments Accepted to NeurIPS 2019. Project page: https://deanplayerljx.github.io/tabvcr

Cambrian-P: Pose-Grounded Video Understanding

Cambrian-P: 基于姿态的视频理解

Jihan Yang, Zifan Zhao, Xichen Pan, Shusheng Yang, Junyi Zhang, Bingyi Kang, Hu Xu, Shang-Wen Li, Saining Xie

机构 * New York University（纽约大学） ； UC Berkeley（加州大学伯克利分校） ； Meta FAIR 纠错

专题命中 视觉推理 ：MLLM(abstract,abstract_cn)；分类 cs.CV

AI总结 该研究提出Cambrian-P，一种增强的视频多模态大语言模型，通过引入可学习的相机令牌和姿态回归头，利用姿态作为轻量级监督信号，显著提升了空间推理能力，并在多个视频问答基准上实现了SOTA表现。

Comments Project Page: https://cambrian-mllm.github.io/

MindClaw: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention

MindClaw: 用于精确干预的闭环具身心理状态推理

Ruoxuan Zhang, Qiaoqiao Wan, Zhengguang Wang, Chenghao Yu, Hongxia Xie, Jianlong Fu, Wen-Huang Cheng

机构 * Jilin University（吉林大学） ； Microsoft Asia（微软亚洲） ； National Taiwan University（国立台湾大学） 纠错

专题命中 视觉推理 ：VLM(abstract,abstract_cn)；分类 cs.AI

AI总结 提出MindClaw框架，通过闭环具身心理状态推理实现精确干预，结合多源输入、信念记忆、认知触发技能和动作生成，在动态环境中优化干预时机。

Comments Extended version of the CVPR 2026 paper *MindPower: Enabling Theory-of-Mind Reasoning in VLM-based Embodied Agents*. This work is in progress

Vision-Language and Large Language Model Performance in Gastroenterology: GPT, Claude, Llama, Phi, Mistral, Gemma, and Quantized Models

视觉-语言与大语言模型在胃肠病学中的表现：GPT、Claude、Llama、Phi、Mistral、Gemma及量化模型

Seyed Amir Ahmad Safavi-Naini, Shuhaib Ali, Omer Shahab, Zahra Shahhoseini, Thomas Savage, Sara Rafiee, Jamil S Samaan, Reem Al Shabeeb, Farah Ladak, Jamie O Yang, Juan Echavarria, Sumbal Babar, Aasma Shaukat, Samuel Margolis, Nicholas P Tatonetti, Girish Nadkarni, Bara El Kurdi, Ali Soroush

机构 * Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院） ； University of Texas Health（德克萨斯大学健康科学中心） ； Virginia Hospital Center（弗吉尼亚医院中心） ； Shahid Beheshti University of Medical Sciences（沙希德·贝赫什提医科大学） ； Stanford University（斯坦福大学） ； Cedars-Sinai Medical Center（西达赛奈医疗中心） ； Inova Fairfax Medical Campus（伊诺瓦费尔法克斯医疗中心） ； University of California–Los Angeles（加州大学洛杉矶分校） ； NYU Grossman School of Medicine（纽约大学格罗斯曼医学院） ； Columbia University（哥伦比亚大学） 纠错

专题命中 视觉推理 ：VLM(abstract,comments)；vision language model(abstract)；分类 cs.AI

AI总结 本研究评估了大语言模型和视觉-语言模型在胃肠病学中的医学推理性能，比较了不同模型配置、参数及提示工程策略对性能的影响，发现专有模型在准确性上优于开源模型，且图像描述对视觉-语言模型性能有显著影响。

Comments Manuscript Pages: 34, Figures: 7, Tables: 2, Supplementary File Pages: 35, Data Transparency Statement: Code is available at: https://github.com/Sdamirsa/LLM-VLM-in-Gastroenterology . Study data from American College of Gastroenterology (ACG) are restricted and available upon request with ACG permission. Correction: updated abstract considering Llama3.1 results

Journal ref npj Digital Medicine 8, 797 (2025)

NeuroMosaic: Anatomically Grounded Multimodal Large Language Modeling for Molecularly Aware Glioma Reasoning from 3D MRI and Clinical Narratives

NeuroMosaic：基于解剖学的多模态大语言模型，用于从3D MRI和临床叙事中进行分子感知的胶质瘤推理

Yantong Liu, Zheyu Zhang, Runpeng Liu, Mu Xitang, Seong-Yoon Shin, Hyun-Ae Lee

专题命中 视觉推理 ：multimodal large language model(title)

AI总结 该研究针对多模态医疗大语言模型在神经肿瘤学中的结构缺陷，提出NeuroMosaic模型，通过多模块架构实现胶质瘤的准确推理，在多个数据集上取得优异性能，验证了解剖学索引路由机制的有效性。

Grounding latent algorithm routing in transformer reasoning

在变压器推理中为潜在算法路由建立基础

Xiangbo Zhang, Xiaoxu Ma

机构 * Georgia Institute of Technology（佐治亚理工学院） 纠错

专题命中 视觉推理 ：grounding(title)

AI总结 研究变压器能否围绕不同归纳偏差族组织情节级适应，通过潜在算法路由及ROUTEBENCH基准实验，发现从零训练的密集仅解码器变压器能开发类似路由的内部变量，缩小最优路由差距，但未建立通用路由。

Comments Accepted by COLM 2026

Before Thinking, Learn to Decide: Proactive Routing for Efficient Visual Reasoning

在思考之前，先学会决策：面向高效视觉推理的主动路由

Yinan Zhou, Haokun Lin, Yichen Wu, Yuxin Chen, Teng Wang, Caifeng Shan, Zhenan Sun, Chen Ma, Li Zhu, Ying Shan

机构 * Xi’an Jiaotong University（西安交通大学） ； ARC Lab, Tencent IEG（腾讯IEG ARC实验室） ； City University of Hong Kong（香港城市大学） ； Institute of Automation, CAS（中国科学院自动化研究所） ； Harvard University（哈佛大学） ； Nanjing University（南京大学） 纠错

专题命中 视觉推理 ：visual reasoning(title)

AI总结 提出主动路由范式PRP，通过联合评估草稿模型和目标模型的能力，实现早期决策，加速多模态推理而不牺牲性能。

Comments 36 pages, 20 figures

Structured Visualization Design Knowledge for Grounding Generative Reasoning and Situated Feedback

结构化可视化设计知识：用于基础生成推理和情境反馈

Péter Ferenc Gyarmati, Dominik Moritz, Torsten Möller, Laura Koesten

专题命中 视觉推理 ：grounding(title)

AI总结 提出一种结构化方案，将可视化设计知识编码为带语义元数据的自然语言指南，支持专家编写、机器查询，并嵌入向量空间以分析冲突和跨领域原则，弥补符号系统与生成模型之间的鸿沟。

Evaluating and Steering Modality Preferences in Multimodal Large Language Model

评估和引导多模态大语言模型中的模态偏好

Yu Zhang, Jinlong Ma, Yongshuai Hou, Xuefeng Bai, Kehai Chen, Yang Xiang, Jun Yu, Min Zhang

机构 * Harbin Institute of Technology, Shenzhen, China（哈尔滨工业大学（深圳）） ； Peng Cheng Laboratory, Shenzhen, China（鹏城实验室） 纠错

专题命中 视觉推理 ：multimodal large language model(title)

AI总结 本文提出MC²基准测试，通过受控证据冲突场景评估和引导多模态大语言模型的模态偏好，揭示了其可通过指令引导和潜在表示控制，并展示了通过表示工程方法提升多模态任务性能的潜力。

Comments Modality Preference

FinCoT: Grounding Chain-of-Thought in Expert Financial Reasoning

FinCoT：在专家金融推理中 grounding 思维链

Natapong Nitarach, Warit Sirichotedumrong, Panop Pitchayarthorn, Pittawat Taveekitworachai, Potsawee Manakul, Kunat Pipatanakul

机构 * SCB 10X, SCBX Group（SCB 10X集团） 纠错

专题命中 视觉推理 ：grounding(title)

AI总结 FinCoT通过整合专家金融推理蓝图，提升金融领域模型性能并减少推理成本，实现更可解释的推理过程。

Comments Accepted at FinNLP-2025, EMNLP (Oral Presentation)

Grounding Long-Context Reasoning with Contextual Normalization for Retrieval-Augmented Generation

通过上下文归一化增强长上下文推理用于检索增强生成

Jiamin Chen, Yuchen Li, Xinyu Ma, Xinran Chen, Xiaokun Zhang, Shuaiqiang Wang, Chen Ma, Dawei Yin

机构 * City University of Hong Kong（香港城市大学） ； Baidu Inc.（百度公司） 纠错

专题命中 视觉推理 ：grounding(title)

AI总结 通过上下文归一化策略提升RAG在长上下文推理中的鲁棒性和稳定性