视觉大模型 / VLM

RapGuard: Safeguarding Multimodal Large Language Models via Rationale-aware Defensive Prompting

Yilei Jiang, Yingshui Tan, Xiangyu Yue

专题命中 视觉推理 ：multimodal large language model(title,abstract)；MLLM(abstract)

Perception Tokens Enhance Visual Reasoning in Multimodal Language Models

Mahtab Bigverdi, Zelun Luo, Cheng-Yu Hsieh, Ethan Shen, Dongping Chen, Linda G. Shapiro, Ranjay Krishna

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Learning to Reason Iteratively and Parallelly for Complex Visual Reasoning Scenarios

Shantanu Jaiswal, Debaditya Roy, Basura Fernando, Cheston Tan

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments NeurIPS 2024 camera ready; source code to be released at: https://github.com/shantanuj/IPRM_Iterative_and_Parallel_Reasoning_Mechanism

Leveraging Vision-Language Models for Manufacturing Feature Recognition in CAD Designs

Muhammad Tayyab Khan, Lequn Chen, Ye Han Ng, Wenhe Feng, Nicholas Yew Jin Tan, Seung Ki Moon

专题命中 视觉推理 ：vision-language model(title,abstract)；LLaVA(abstract)

Comments Paper has been submitted to The ASME Journal of Computing and Information Science in Engineering (JCISE)

Can Multimodal Large Language Model Think Analogically?

Diandian Guo, Cong Cao, Fangfang Yuan, Dakui Wang, Wei Ma, Yanbing Liu, Jianhui Fu

专题命中 视觉推理 ：multimodal large language model(title,abstract)；MLLM(abstract)

Reasoning Grasping via Multimodal Large Language Model

Shiyu Jin, Jinxuan Xu, Yutian Lei, Liangjun Zhang

专题命中 视觉推理 ：multimodal large language model(title,abstract)；LLaVA(abstract)

Comments CoRL 2024

I Know About "Up"! Enhancing Spatial Reasoning in Visual Language Models Through 3D Reconstruction

Zaiqiao Meng, Hao Zhou, Yifang Chen

专题命中 视觉推理 ：visual language model(title,abstract)；visual reasoning(abstract)

A Unified View of Abstract Visual Reasoning Problems

Mikołaj Małkiński, Jacek Mańdziuk

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

A3VLM: Actionable Articulation-Aware Vision Language Model

Siyuan Huang, Haonan Chang, Yuhan Liu, Yimeng Zhu, Hao Dong, Peng Gao, Abdeslam Boularias, Hongsheng Li

专题命中 视觉推理 ：vision language model(title,abstract)；visual reasoning(abstract)

MMCTAgent: Multi-modal Critical Thinking Agent Framework for Complex Visual Reasoning

Somnath Kumar, Yash Gadhia, Tanuja Ganu, Akshay Nambi

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Vision-Language Model-based Physical Reasoning for Robot Liquid Perception

Wenqiang Lai, Yuan Gao, Tin Lun Lam

专题命中 视觉推理 ：vision-language model(title,abstract)；grounding(abstract)

Comments 8 pages, 6 figures, submitted to IROS 2024

Prismer: A Vision-Language Model with Multi-Task Experts

Shikun Liu, Linxi Fan, Edward Johns, Zhiding Yu, Chaowei Xiao, Anima Anandkumar

专题命中 视觉推理 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Published at TMLR 2024. Project Page: https://shikun.io/projects/prismer Code: https://github.com/NVlabs/prismer

EuclidNet: Deep Visual Reasoning for Constructible Problems in Geometry

Man Fai Wong, Xintong Qi, Chee Wei Tan

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by 2nd MATH-AI Workshop at NeurIPS'22

Journal ref Adv. Artif. Intell. Mach. Learn.(2023), 3(1):839-852

Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions

Huaizu Jiang, Xiaojian Ma, Weili Nie, Zhiding Yu, Yuke Zhu, Song-Chun Zhu, Anima Anandkumar

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR 2022 (oral); First two authors contributed equally; Code: https://github.com/NVlabs/Bongard-HOI

A Review of Emerging Research Directions in Abstract Visual Reasoning

Mikołaj Małkiński, Jacek Mańdziuk

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Journal ref Information Fusion, Volume 91, March 2023, Pages 713-736

See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoning

Zhenfang Chen, Qinhong Zhou, Yikang Shen, Yining Hong, Hao Zhang, Chuang Gan

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments The first two authors contributed equally to this work

Abstract Visual Reasoning with Tangram Shapes

Anya Ji, Noriyuki Kojima, Noah Rush, Alane Suhr, Wai Keen Vong, Robert D. Hawkins, Yoav Artzi

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments EMNLP 2022 long paper

Measuring CLEVRness: Blackbox testing of Visual Reasoning Models

Spyridon Mouselinos, Henryk Michalewski, Mateusz Malinowski

专题命中 视觉推理 ：visual reasoning(title)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICLR 2022

Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language

Mingyu Ding, Zhenfang Chen, Tao Du, Ping Luo, Joshua B. Tenenbaum, Chuang Gan

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments NeurIPS 2021. Project page: http://vrdp.csail.mit.edu/

Attention over learned object embeddings enables complex visual reasoning

David Ding, Felix Hill, Adam Santoro, Malcolm Reynolds, Matt Botvinick

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 22 pages, 5 figures

Interpretable Visual Reasoning via Induced Symbolic Space

Zhonghao Wang, Kai Wang, Mo Yu, Jinjun Xiong, Wen-mei Hwu, Mark Hasegawa-Johnson, Humphrey Shi

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICCV 2021

Transformation Driven Visual Reasoning

Xin Hong, Yanyan Lan, Liang Pang, Jiafeng Guo, Xueqi Cheng

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to CVPR 2021. Resources including the TRANCE dataset and the code can be found at our homepage https://hongxin2019.github.io/TVR

SHOP-VRB: A Visual Reasoning Benchmark for Object Perception

Michal Nazarczuk, Krystian Mikolajczyk

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments International Conference on Robotics and Automation (ICRA) 2020

RAVEN: A Dataset for Relational and Analogical Visual rEasoNing

Chi Zhang, Feng Gao, Baoxiong Jia, Yixin Zhu, Song-Chun Zhu

专题命中 视觉推理 ：visual reasoning(title)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR 2019 paper. Supplementary: http://wellyzhang.github.io/attach/cvpr19zhang_supp.pdf Project: http://wellyzhang.github.io/project/raven.html

A Dataset and Architecture for Visual Reasoning with a Working Memory

Guangyu Robert Yang, Igor Ganichev, Xiao-Jing Wang, Jonathon Shlens, David Sussillo

专题命中 视觉推理 ：visual reasoning(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Rationale-Guided Learning for Multimodal Emotion Recognition

基于理由引导学习的多模态情感识别

Sujung Oh, Jung Uk Kim, Sangmin Lee

专题命中 视觉推理 ：MLLM(summary_cn,abstract)；分类 cs.AI

AI总结 针对多模态情感识别忽略人类因果推理的问题，提出理由引导学习框架，结合双加工理论与MLLM生成的理由训练模型，在IEMOCAP和MELD基准取得最优性能，且推理无额外开销。

Comments ICASSP 2026

Spatial Memory Agent: Experience-Grounded Procedure Memory for Spatial Intelligence

空间记忆智能体：用于空间智能的基于经验的过程记忆

Haokai Zhang, Yuhang Ding, Yunshu Zhou, Xinze Du, Shengtao Zhang, Zhiyue Zhao, Yuling Xi, Hao Chen

机构 * Zhejiang University（浙江大学） ； Shanghai Jiao Tong University（上海交通大学） ； Shanghai Innovation Institute（上海创新研究院） 纠错

专题命中 视觉推理 ：VLM(summary_cn,abstract)；分类 cs.AI

AI总结 该研究提出SMA框架，让冻结VLM智能体无需外部空间工具，通过无参数更新自进化提升空间推理，在多基准和模型上表现最优，提供了空间自进化的实用路径。

Comments Under Review

MMArch: Benchmarking Multimodal Reasoning Grounded in Architectural Evidence

MMArch：基于建筑证据的多模态推理基准测试

Chenxu Du, Kang An, Tengyue Wang, Zhongyu Yang, Xinqi Yang, Yuanchi Zhu, Hebao Zhu, Ziliang Wang, Faqiang Qian, Yunli Yang, Qibing Ren

专题命中 视觉推理 ：MLLM(summary_cn,abstract_cn)；multimodal large language model(abstract)；分类 cs.AI

AI总结 研究针对多模态大语言模型（MLLM）在工程多模态推理基准上的表现，构建MMArch基准，发现现有MLLM与人类专家存在差距，为相关研究提供评估基准。

EffectLearner: World-Aware Object-Effect Reasoning for Real-World Video Object Removal

EffectLearner：面向真实世界视频物体移除的世界感知物体-效应推理

Feier Wu, Wanke Xia, Xu He, Zilang Zhou, Si Chen, Dongxia Liu, Liyang Chen, Qimeng Wu, Zhengbo Zhang, Wenming Yang, Zhiyong Wu

专题命中 视觉推理 ：VLM(summary_cn,abstract)；分类 cs.CV

AI总结 该研究提出EffectLearner框架，结合VLM物体-效应推理器与DiT视频擦除器，构建专属数据集EffectWorld并采用渐进式训练，在视频物体移除任务上实现更优性能。

Comments Project: https://morleyolsen.github.io/EffectLearner/

Beyond the Single Camera: Agentic Multi-View Reasoning in Sports Video Understanding

超越单摄像头：体育视频理解中的智能多视角推理

Kerui Chen, Jinglu Wang, Xiaoyi Zhang, Yan Lu

机构 * Zhejiang University（浙江大学） ； Microsoft Research Asia（微软亚洲研究院） 纠错

专题命中 视觉推理 ：MLLM(summary_cn,abstract)；multimodal large language model(abstract)；分类 cs.CV

AI总结 针对体育视频多视角理解缺乏评估基准及MLLMs难以利用多视角信息的问题，引入SportMV - Bench基准，分析瓶颈所在，并提出SportMV - Agent框架，通过迭代循环实现主动视角选择等，相比最强MLLM基线有显著提升。