视觉大模型 / VLM

Learning Reasoning Paths over Semantic Graphs for Video-grounded Dialogues

Hung Le, Nancy F. Chen, Steven C. H. Hoi

专题命中 视觉推理 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at ICLR (International Conference on Learning Representations) 2021

VIPHY: Probing "Visible" Physical Commonsense Knowledge

Shikhar Singh, Ehsan Qasemi, Muhao Chen

专题命中 视觉推理 ：vision-language model(abstract)；visual reasoning(abstract)

Comments In Progress (under review)

IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning

Pan Lu, Liang Qiu, Jiaqi Chen, Tony Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, Song-Chun Zhu

专题命中 视觉推理 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Corrected typos. Accepted to NeurIPS 2021, 27 pages, 18 figures. Data and code are available at https://iconqa.github.io

RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning

Xiaojian Ma, Weili Nie, Zhiding Yu, Huaizu Jiang, Chaowei Xiao, Yuke Zhu, Song-Chun Zhu, Anima Anandkumar

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICLR 2022; Code: https://github.com/NVlabs/RelViT

Combining Commonsense Reasoning and Knowledge Acquisition to Guide Deep Learning in Robotics

Mohan Sridharan, Tiago Mota

专题命中 视觉推理 ：grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 37 pages, 17 figures, 5 tables

PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning

Yining Hong, Li Yi, Joshua B. Tenenbaum, Antonio Torralba, Chuang Gan

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments NeurIPS 2021. Project page: http://ptr.csail.mit.edu/

Scene Graph Generation with Geometric Context

Vishal Kumar, Albert Mundu, Satish Kumar Singh

专题命中 视觉推理 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Paper accepted at 6th IAPR International Conference on Computer Vision & Image Processing (CVIP2021), IIT Ropar, India

ProTo: Program-Guided Transformer for Program-Guided Tasks

Zelin Zhao, Karan Samel, Binghong Chen, Le Song

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted in NeurIPS 2021

Five Points to Check when Comparing Visual Perception in Humans and Machines

Christina M. Funke, Judy Borowski, Karolina Stosio, Wieland Brendel, Thomas S. A. Wallis, Matthias Bethge

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments V3: minor changes like in published JOV version (https://doi.org/10.1167/jov.21.3.16) V2: New title; added general section (checklist); manuscript restructured such that each case study is one chapter; adversarial examples in first study replaced by different analysis

Journal ref Journal of Vision 21, no. 3 (2021): 16-16

ACRE: Abstract Causal REasoning Beyond Covariation

Chi Zhang, Baoxiong Jia, Mark Edmonds, Song-Chun Zhu, Yixin Zhu

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR 2021 paper. Supplementary: http://wellyzhang.github.io/attach/cvpr21zhang_acre_supp.pdf Project: http://wellyzhang.github.io/project/acre.html

Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning

Weili Nie, Zhiding Yu, Lei Mao, Ankit B. Patel, Yuke Zhu, Animashree Anandkumar

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 22 pages, NeurIPS 2020

CLEVR Parser: A Graph Parser Library for Geometric Learning on Language Grounded Image Scenes

Raeid Saqur, Ameet Deshpande

专题命中 视觉推理 ：visual reasoning(abstract)；grounding(abstract)

Comments Accepted at NLP-OSS, EMNLP 2020 (2nd Workshop for Natural Language Processing Open Source Software)

Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning

Qing Li, Siyuan Huang, Yining Hong, Yixin Chen, Ying Nian Wu, Song-Chun Zhu

专题命中 视觉推理 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICML 2020. Project page: https://liqing-ustc.github.io/NGS

Cross-Modality Relevance for Reasoning on Language and Vision

Chen Zheng, Quan Guo, Parisa Kordjamshidi

专题命中 视觉推理 ：visual reasoning(abstract)；visual question answering(abstract)

Comments Accepted by ACL 2020

CLEVRER: CoLlision Events for Video REpresentation and Reasoning

Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, Joshua B. Tenenbaum

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments The first two authors contributed equally to this work. Accepted as Oral Spotlight as ICLR 2020. Project page: http://clevrer.csail.mit.edu/

Transfer Learning in Visual and Relational Reasoning

T. S. Jayram, Vincent Marois, Tomasz Kornuta, Vincent Albouy, Emre Sevgen, Ahmet S. Ozcan

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 18 pages; more baseline comparisons; additional clarifications

Enforcing Reasoning in Visual Commonsense Reasoning

Hammad A. Ayyubi, Md. Mehrab Tanjim, David J. Kriegman

专题命中 视觉推理 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Learning by Abstraction: The Neural State Machine

Drew A. Hudson, Christopher D. Manning

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Published as a conference paper at NeurIPS 2019 (spotlight)

CLEVR-Dialog: A Diagnostic Dataset for Multi-Round Reasoning in Visual Dialog

Satwik Kottur, José M. F. Moura, Devi Parikh, Dhruv Batra, Marcus Rohrbach

专题命中 视觉推理 ：grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 13 pages, 11 figures, 3 tables, accepted as a short paper at NAACL 2019

Mapping Natural Language Commands to Web Elements

Panupong Pasupat, Tian-Shun Jiang, Evan Zheran Liu, Kelvin Guu, Percy Liang

专题命中 视觉推理 ：visual reasoning(abstract)；grounding(abstract)

Comments EMNLP 2018

Same-different problems strain convolutional neural networks

Matthew Ricci, Junkyung Kim, Thomas Serre

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 6 Pages, 4 Figures

Chitrakshara: A Large Multilingual Multimodal Dataset for Indian languages

Chitrakshara：一种用于印度语言的大型多语言多模态数据集

Shaharukh Khan, Ali Faraz, Abhinav Ravi, Mohd Nauman, Mohd Sarfraz, Akshat Patidar, Raja Kolla, Chandra Khatri, Shubham Agarwal

机构 * Krutrim AI 纠错

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI；vision language model(comments)

AI总结 本文提出Chitrakshara数据集，涵盖11种印度语言，旨在提升多模态模型对印度语言的表示能力，通过大规模预训练和图像文本对的构建，促进更文化包容的视觉语言模型发展。

Comments Accepted at "CVPR 2025: Workshop Vision Language Models For All"

RynnEC: Bringing MLLMs into Embodied World

Ronghao Dang, Yuqian Yuan, Yunxuan Mao, Kehan Li, Jiangpin Liu, Zhikai Wang, Xin Li, Fan Wang, Deli Zhao

机构 * DAMO Academy, Alibaba Group（达摩院，阿里巴巴集团） ； Hupan Lab（虎盘实验室） ； Zhejiang University（浙江大学） 纠错

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI；MLLM(comments)

Comments The technical report of RynnEC, an embodied cognition MLLM

When Text Misleads: Inconsistent-Aware Reasoning for Audio-Grounded Dialogue

当文本误导时：面向音频接地对话的不一致感知推理

Yen-Ju Lu, Yuzhe Wang, Yaohan Guan, Xiluo He, Jiarui Hai, Mingrui Liang, Kaavya Chaparala, Thomas Thebaud, Laureano Moro-Velazquez, Najim Dehak, Jesus Villalba

机构 * Center for Language and Speech Processing, Johns Hopkins University（约翰霍普金斯大学语言与语音处理中心） 纠错

专题命中 视觉推理 ：grounding(abstract)；分类 cs.AI、cs.LG

AI总结 本研究针对口语对话理解中基于转录本的捷径问题，构建了含501个问题的受控基准ContraTalk，提出Audio Twin智能体式推理框架，可提升冲突问答案例的准确率并减少文本偏向陷阱选择。

Comments 24 pages, 4 figures

Co-Evolving Structured Knowledge and Reasoning in Language Models

语言模型中结构化知识与推理的协同演化

Ryan Thomas Noonan, Linxi Zhao, Menghan Xu, Akanksha Sarkar, Mihir Mishra, Dongyoung Go, Kilian Q. Weinberger, Yoav Artzi, Jennifer J. Sun

机构 * Cornell University（康奈尔大学） 纠错

专题命中 视觉推理 ：grounding(abstract)；分类 cs.AI、cs.LG

AI总结 针对检索增强方法的局限，提出协同演化框架KBevo，联合学习构建结构化知识库与推理，提升了知识结构质量、答案可达性及推理与可控性。

Comments COLM2026. Code available at this https URL (https://github.com/kilian-group/KBevo)

TAU-Agent: An Agentic Retrieval-Augmented Framework for Traffic Anomaly Understanding

TAU-Agent：用于交通异常理解的智能体式检索增强框架

Yuqiang Lin, Yan Shi, Sam Lockyer, Harish Tayyar Madabushi, Adrian Evans, Wenbin Li, Yinhai Wang, Nic Zhang

机构 * University of Bath（巴斯大学） ； University of Washington（华盛顿大学） 纠错

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结 针对交通异常理解需求，提出TAU-Agent智能体式检索增强框架，调度两类视觉工具获取证据，结合微调视觉语言模型推理，在AI City 2026挑战赛多赛道取得对应排名成绩。

V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning

V-Rubrics：基于评分规则的强化学习实现视觉忠实性

Shulin Tian, Minglun Li, Yuhao Dong, Hao Ding, Jiarui Yao, Haiwen Diao, Jingkang Yang, Hongyuan Zhu, Ziwei Liu

机构 * S-Lab, Nanyang Technological University（新加坡南洋理工大学S-Lab） ； UIUC（伊利诺伊大学厄巴纳-香槟分校） ； A*STAR（新加坡科技研究局） 纠错

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结 该研究针对视觉-语言模型回答缺乏视觉依据的问题，提出基于评分规则的强化学习方法V-Rubrics，通过分解响应为原子命题并从三方面评分，训练的模型在相关推理基准上性能优于基线。

Comments Proj page: https://shulin16.github.io/v-rubrics/

VGI-Bench: Probing Visual Intelligence in Video Generation Models

VGI-BENCH：探究视频生成模型的视觉智能

Xuan He, Cong Wei, Yuhao Cheng, Linrui Ma, Yuxuan Zhang, Zuojun Li, Yuhao Wen, Jize Jiang, Zeyi Liu, Yuren Hao, Songcheng Cai, Keming Wu, Penghui Du, Kai Zou, Rui Yang, Chenkai Sun, Ke Yang, Ping Nie, Kelsey R Allen, Chenglong Wang, Michel Galley, Jianfeng Gao, ChengXiang Zhai

机构 * University of Illinois Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校） ； Tsinghua University（清华大学） ； University of Waterloo（滑铁卢大学） ； Massachusetts Institute of Technology（麻省理工学院） ； University of British Columbia（不列颠哥伦比亚大学） ； Vector Institute（矢量研究所） ； Microsoft Research（微软研究院） ； Etude AI 纠错

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI

AI总结 本研究推出VGI-bench基准，含27项任务810个实例，评估视频生成模型视觉推理能力，发现最强模型Seedance~2.0仅达51.0%，将推动下一代视频生成模型发展。

Infer Human's Intentions Before Following Natural Language Instructions

在遵循自然语言指令前推断人类的意图

Yanming Wan, Yue Wu, Yiping Wang, Jiayuan Mao, Natasha Jaques

专题命中 视觉推理 ：grounding(abstract)；分类 cs.AI、cs.LG

AI总结 该研究针对人类指令的歧义问题，提出FISER框架，通过显式推断人类意图改进协作具身任务的指令遵循，在HandMeThat基准上达到最优性能。

Thinking Beyond Videos: Unifying Video Reasoning and Deep Research for Open-World Video Agents

超越视频：统一视频推理与深度研究的开放世界视频智能体

Wenqi Liu, Shijie Ma, Yunxiao Wang, Meng Liu, Qile Su, Han Liu, Bohan Hou, Zeyu Wang, Xuanyu Zheng, Changyi Liu, Tianke Zhang, Haonan Fan, Kaiyu Jiang, Yingxin Li, Jiankang Chen, Xu Wang, Hongyi Fu, Jianxiong Wang, Bin Wen, Tingting Gao, Han Li, Jianhua Yin, Yinwei Wei, Xuemeng Song

机构 * Shandong University（山东大学） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） ； Beihang University（北京航空航天大学） ； City University of Hong Kong（香港城市大学） ； Nanyang Technological University（南洋理工大学） ； Kuaishou Technology（快手科技） ； Southern University of Science and Technology（南方科技大学） 纠错

专题命中 视觉推理 ：grounding(abstract)；分类 cs.CV、cs.AI

AI总结 本研究提出统一视频推理与深度研究的VideoRover框架，构建相关数据集与基准，其8B-RL模型在无工具直接回答场景性能接近专有模型，优于同工具套件的更大开源模型。