视觉大模型 / VLM

Discovering Pathology Rationale and Token Allocation for Efficient Multimodal Pathology Reasoning

Zhe Xu, Cheng Jin, Yihui Wang, Ziyi Liu, Hao Chen

机构 * Department of Computer Science Engineering（计算机科学与工程系） ； HKUST（香港科技大学） 纠错

专题命中 视觉推理 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Plane Geometry Problem Solving with Multi-modal Reasoning: A Survey

Seunghyuk Cho, Zhenyue Qin, Yang Liu, Youngbin Choi, Seungbeom Lee, Dongwoo Kim

机构 * Graduate School of Artificial Intelligence, POSTECH（POSTECH人工智能研究生院） ； Department of Computer Science and Engineering, POSTECH（POSTECH计算机科学与工程系） ； Australian National University（澳大利亚国立大学） 纠错

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments 18 pages

X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domains

Qianchu Liu, Sheng Zhang, Guanghui Qin, Timothy Ossowski, Yu Gu, Ying Jin, Sid Kiblawi, Sam Preston, Mu Wei, Paul Vozila, Tristan Naumann, Hoifung Poon

机构 * Microsoft Research（微软研究院） 纠错

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.AI、cs.LG

LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perception

Yuan-Hong Liao, Sven Elflein, Liu He, Laura Leal-Taixé, Yejin Choi, Sanja Fidler, David Acuna

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.LG

Comments 24 pages, 10 figures, in submission. Project page: https://andrewliao11.github.io/LongPerceptualThoughts

Improved Visual-Spatial Reasoning via R1-Zero-Like Training

Zhenyi Liao, Qingsong Xie, Yanhao Zhang, Zijian Kong, Haonan Lu, Zhenyu Yang, Zhijie Deng

机构 * Shanghai Jiao Tong University（上海交通大学） ； OPPO AI Center（OPPO人工智能中心） 纠错

专题命中 视觉推理 ：LLaVA(abstract)；分类 cs.CV、cs.AI

VCR-Bench: A Comprehensive Evaluation Framework for Video Chain-of-Thought Reasoning

Yukun Qi, Yiming Zhao, Yu Zeng, Xikun Bao, Wenxuan Huang, Lin Chen, Zehui Chen, Jie Zhao, Zhongang Qi, Feng Zhao

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI

TGraphX: Tensor-Aware Graph Neural Network for Multi-Dimensional Feature Learning

Arash Sajjadi, Mark Eramian

机构 * University of Saskatchewan（萨斯喀彻温大学） 纠错

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI

Comments Submitted to arXiv. Code repository: https://github.com/arashsajjadi/TGraphX ||| https://git.cs.usask.ca/arash/tgraphx

Bonsai: Interpretable Tree-Adaptive Grounded Reasoning

Kate Sanders, Benjamin Van Durme

机构 * Johns Hopkins University（约翰斯·霍普金斯大学） 纠错

专题命中 视觉推理 ：grounding(abstract)；分类 cs.CV、cs.AI

Comments 9 pages, preprint

GenVP: Generating Visual Puzzles with Contrastive Hierarchical VAEs

Kalliopi Basioti, Pritish Sahu, Qingze Tony Liu, Zihao Xu, Hao Wang, Vladimir Pavlovic

机构 * Rutgers University（罗格斯大学） ； SRI International（SRI国际） 纠错

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICLR 2025

EgoToM: Benchmarking Theory of Mind Reasoning from Egocentric Videos

Yuxuan Li, Vijay Veerabadran, Michael L. Iuzzolino, Brett D. Roads, Asli Celikyilmaz, Karl Ridgeway

机构 * Reality Labs ； FAIR 纠错

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Towards Visual Discrimination and Reasoning of Real-World Physical Dynamics: Physics-Grounded Anomaly Detection

Wenqiao Li, Yao Gu, Xintao Chen, Xiaohao Xu, Ming Hu, Xiaonan Huang, Yingna Wu

机构 * ShanghaiTech University（上海科技大学） ； University of Michigan, Ann Arbor（密歇根大学安娜堡分校） ； Monash University（莫纳什大学） 纠错

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR25

ROCKET-1: Mastering Open-World Interaction with Visual-Temporal Context Prompting

Shaofei Cai, Zihao Wang, Kewei Lian, Zhancun Mu, Xiaojian Ma, Anji Liu, Yitao Liang

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Lost in Time: Clock and Calendar Understanding Challenges in Multimodal LLMs

Rohit Saxena, Aryo Pradipta Gema, Pasquale Minervini

机构 * University of Edinburgh（爱丁堡大学） 纠错

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted at the ICLR 2025 Workshop on Reasoning and Planning for Large Language Models

TxAgent: An AI Agent for Therapeutic Reasoning Across a Universe of Tools

Shanghua Gao, Richard Zhu, Zhenglun Kong, Ayush Noori, Xiaorui Su, Curtis Ginder, Theodoros Tsiligkaridis, Marinka Zitnik

机构 * Harvard Medical School（哈佛医学院） ； MIT Lincoln Laboratory（麻省理工学院林肯实验室） ； Harvard University（哈佛大学） ； Broad Institute of MIT and Harvard（麻省理工学院与哈佛大学博德研究所） ； Harvard Data Science Initiative（哈佛数据科学倡议） 纠错

专题命中 视觉推理 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments Project page: https://zitniklab.hms.harvard.edu/TxAgent TxAgent code: https://github.com/mims-harvard/TxAgent ToolUniverse code: https://github.com/mims-harvard/ToolUniverse

Uncovering Hidden Connections: Iterative Search and Reasoning for Video-grounded Dialog

Haoyu Zhang, Meng Liu, Yisen Feng, Yaowei Wang, Weili Guan, Liqiang Nie

机构 * School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）计算机科学与技术学院） ； Pengcheng Laboratory（鹏城实验室） ； School of Computer Science and Technology, Shandong Jianzhu University（山东建筑大学计算机科学与技术学院） 纠错

专题命中 视觉推理 ：visual question answering(abstract)；分类 cs.CV、cs.AI

R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning

Jiaxing Zhao, Xihan Wei, Liefeng Bo

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.LG

VCA: Video Curious Agent for Long Video Understanding

Zeyuan Yang, Delin Chen, Xueyang Yu, Maohao Shen, Chuang Gan

机构 * University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校） ； Massachusetts Institute of Technology（麻省理工学院） 纠错

专题命中 视觉推理 ：VLM(abstract)；分类 cs.CV、cs.AI

DriveGen: Towards Infinite Diverse Traffic Scenarios with Large Models

Shenyu Zhang, Jiaguo Tian, Zhengbang Zhu, Shan Huang, Jucheng Yang, Weinan Zhang

专题命中 视觉推理 ：visual language model(abstract)；分类 cs.AI、cs.LG

Comments 8 pages, 3 figures

SPEX: Scaling Feature Interaction Explanations for LLMs

Justin Singh Kang, Landon Butler, Abhineet Agarwal, Yigit Efe Erginbas, Ramtin Pedarsani, Kannan Ramchandran, Bin Yu

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.AI、cs.LG

Language Models Can See Better: Visual Contrastive Decoding For LLM Multimodal Reasoning

Yuqi Pang, Bowen Yang, Haoqin Tu, Yun Cao, Zeyu Zhang

机构 * Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所） ； School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院） ； University of California, Santa Cruz（加州大学圣克鲁兹分校） 纠错

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICASSP 2025

I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models

Zhenxing Mi, Kuan-Chieh Wang, Guocheng Qian, Hanrong Ye, Runtao Liu, Sergey Tulyakov, Kfir Aberman, Dan Xu

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments Project page: https://mizhenxing.github.io/ThinkDiff, 19 pages, 14 figures

OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving

Shuo Xing, Chengyuan Qian, Yuping Wang, Hongyuan Hua, Kexin Tian, Yang Zhou, Zhengzhong Tu

机构 * Texas A&M University（得克萨斯农工大学） ； University of Michigan（密歇根大学） ； University of Toronto（多伦多大学） 纠错

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments The 3rd WACV Workshop on Large Language and Vision Models for Autonomous Driving (LLVM-AD) 2025

Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks

Maan Qraitem, Nazia Tasnim, Piotr Teterwak, Kate Saenko, Bryan A. Plummer

机构 * Boston University（波士顿大学） 纠错

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Imagine while Reasoning in Space: Multimodal Visualization-of-Thought

Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, Furu Wei

机构 * Microsoft Research（微软研究院） ； University of Cambridge（剑桥大学） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） 纠错

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments 11 pages, 6 figures, 4 tables (27 pages, 10 figures, 16 tables including references and appendices)

From Simple to Professional: A Combinatorial Controllable Image Captioning Agent

Xinran Wang, Muxi Diao, Baoteng Li, Haiwen Zhang, Kongming Liang, Zhanyu Ma

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments A technical report. Project: https://github.com/xin-ran-w/CapAgent

Visual Large Language Models for Generalized and Specialized Applications

Yifan Li, Zhixin Lai, Wentao Bao, Zhen Tan, Anh Dao, Kewei Sui, Jiayi Shen, Dong Liu, Huan Liu, Yu Kong

机构 * Michigan State University（密歇根州立大学） ； Cornell University（康奈尔大学） ； Arizona State University（亚利桑那州立大学） ； University of California, Berkeley（加州大学伯克利分校） ； University of Texas at Austin（德克萨斯大学奥斯汀分校） ； Yale University（耶鲁大学） 纠错

专题命中 视觉推理 ：VLM(abstract)；分类 cs.CV、cs.AI

Progressive Multimodal Reasoning via Active Retrieval

Guanting Dong, Chenghao Zhang, Mengjie Deng, Yutao Zhu, Zhicheng Dou, Ji-Rong Wen

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Working in progress

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

Pan Zhang, Xiaoyi Dong, Yuhang Cao, Yuhang Zang, Rui Qian, Xilin Wei, Lin Chen, Yifei Li, Junbo Niu, Shuangrui Ding, Qipeng Guo, Haodong Duan, Xin Chen, Han Lv, Zheng Nie, Min Zhang, Bin Wang, Wenwei Zhang, Xinyue Zhang, Jiaye Ge, Wei Li, Jingwen Li, Zhongying Tu, Conghui He, Xingcheng Zhang, Kai Chen, Yu Qiao, Dahua Lin, Jiaqi Wang

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Github Repo: https://github.com/InternLM/InternLM-XComposer/tree/main/InternLM-XComposer-2.5-OmniLive

Towards Neuro-Symbolic Video Understanding

Minkyu Choi, Harsh Goel, Mohammad Omama, Yunhao Yang, Sahil Shah, Sandeep Chinchali

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by The European Conference on Computer Vision (ECCV) 2024

Recurrent Aggregators in Neural Algorithmic Reasoning

Kaijia Xu, Petar Veličković

专题命中 视觉推理 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments Presented at the Third Learning on Graphs Conference (LoG 2024). 10 pages, 1 figure