视觉大模型 / VLM

Do Explanations make VQA Models more Predictable to a Human?

Arjun Chandrasekaran, Viraj Prabhu, Deshraj Yadav, Prithvijit Chattopadhyay, Devi Parikh

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments EMNLP 2018. 16 pages, 11 figures. Content overlaps with "It Takes Two to Tango: Towards Theory of AI's Mind" (arXiv:1704.00717)

The Wisdom of MaSSeS: Majority, Subjectivity, and Semantic Similarity in the Evaluation of VQA

Shailza Jolly, Sandro Pezzelle, Tassilo Klein, Andreas Dengel, Moin Nabi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 7 figures

Neural Compositional Denotational Semantics for Question Answering

Nitish Gupta, Mike Lewis

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments EMNLP 2018

Tree Memory Networks for Modelling Long-term Temporal Dependencies

Tharindu Fernando, Simon Denman, Aaron McFadyen, Sridha Sridharan, Clinton Fookes

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Journal ref Neurocomputing, Volume 304, 23 August 2018, Pages 64-81

Multimodal Explanations: Justifying Decisions and Pointing to the Evidence

Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, Marcus Rohrbach

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments arXiv admin note: text overlap with arXiv:1612.04757

Game of Sketches: Deep Recurrent Models of Pictionary-style Word Guessing

Ravi Kiran Sarvadevabhatla, Shiv Surya, Trisha Mittal, Venkatesh Babu Radhakrishnan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments To be presented at AAAI-2018. Code, pre-trained models and dataset at github.com/val-iisc/sketchguess

Modulating early visual processing by language

Harm de Vries, Florian Strub, Jérémie Mary, Hugo Larochelle, Olivier Pietquin, Aaron Courville

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Advances in Neural Information Processing Systems 30 (NIPS 2017)

Learning by Asking Questions

Ishan Misra, Ross Girshick, Rob Fergus, Martial Hebert, Abhinav Gupta, Laurens van der Maaten

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

It Takes Two to Tango: Towards Theory of AI's Mind

Arjun Chandrasekaran, Deshraj Yadav, Prithvijit Chattopadhyay, Viraj Prabhu, Devi Parikh

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

VQS: Linking Segmentations to Questions and Answers for Supervised Attention in VQA and Question-Focused Semantic Segmentation

Chuang Gan, Yandong Li, Haoxiang Li, Chen Sun, Boqing Gong

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments To appear on ICCV 2017

Attentive Explanations: Justifying Decisions and Pointing to the Evidence

Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Bernt Schiele, Trevor Darrell, Marcus Rohrbach

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Neural Module Networks

Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Dan Klein

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Corrects an error in the evaluation of the NMN-only ablation experiment

Video Question Answering via Attribute-Augmented Attention Network Learning

Yunan Ye, Zhou Zhao, Yimeng Li, Long Chen, Jun Xiao, Yueting Zhuang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted for SIGIR 2017

ShapeWorld - A new test methodology for multimodal language understanding

Alexander Kuhnle, Ann Copestake

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Grad-CAM: Why did you say that?

Ramprasaath R Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, Dhruv Batra

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Presented at NIPS 2016 Workshop on Interpretable Machine Learning in Complex Systems. This is an extended abstract version of arXiv:1610.02391 (CVPR format)

Question Relevance in VQA: Identifying Non-Visual And False-Premise Questions

Arijit Ray, Gordon Christie, Mohit Bansal, Dhruv Batra, Devi Parikh

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Conference on Empirical Methods in Natural Language Processing (EMNLP) 2016

Visual Question: Predicting If a Crowd Will Agree on the Answer

Danna Gurari, Kristen Grauman

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Yin and Yang: Balancing and Answering Binary Visual Questions

Peng Zhang, Yash Goyal, Douglas Summers-Stay, Dhruv Batra, Devi Parikh

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

A Restricted Visual Turing Test for Deep Scene and Event Understanding

Hang Qi, Tianfu Wu, Mun-Wai Lee, Song-Chun Zhu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models

Fan Bai, Yuxin Du, Tiejun Huang, Max Q. -H. Meng, Bo Zhao

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV；MLLM(comments)

Comments MLLM, 3D medical image analysis

Graph Relation Transformer: Incorporating pairwise object features into the Transformer architecture

Michael Yang, Aditya Anantharaman, Zachary Kitowski, Derik Clive Robert

专题命中 视觉问答 ：visual question answering(abstract,comments)；分类 cs.CV

Comments Presented as poster in CVPR 2021 Visual Question Answering Workshop

PreResQ-R1: Response-Preference Disentangled Ranking-and-Scoring Reinforcement Optimization for Robust Visual Quality Assessment

PreResQ-R1：用于鲁棒视觉质量评估的响应偏好解耦排序与评分强化优化

Zehui Feng, Weichuan Wang, Xiaohan Chen, Ting Han

机构 * Shanghai Jiao Tong University（上海交通大学） ； Zhejiang University（浙江大学） 纠错

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV

AI总结 本文提出PreResQ-R1框架，通过解耦响应偏好的强化学习统一评分与排序目标，结合GRPO优化，在IQA和VQA基准上取得最优结果，推理轨迹更贴合人类感知。

Comments 27 pages, 16 figures, Accepted as EMNLP 2026 conference paper

EgoErrorVQA: Assess Egocentric Comprehension Capabilities through Procedural Errors for Ego-Agentic AI

EgoErrorVQA：通过程序错误评估自我主体人工智能的以自我为中心的理解能力

Junlong Li, Junxi Li, Jianjun Gao, Chen Cai, Lap-Pui Chau, Yi Wang

机构 * The Hong Kong Polytechnic University（香港理工大学） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 视觉问答 ：visual language model(abstract)；分类 cs.CV

AI总结 针对现有视觉智能体和VLMs忽略以自我为中心视角评估程序理解能力的问题，本文提出EgoErrorVQA任务，开发基于A2A协议的评估智能体，引入Ego-ADR框架提升模型对程序错误的理解，取得良好效果。

Compositional Chain-of-Relations for Faithful Knowledge Graph Question Answering with Large Language Models

面向大型语言模型的忠实知识图谱问答的组合式关系链

Chenhui Liu, Jianpeng Zhou, Jiahai Wang

机构 * Sun Yat-sen University（中山大学） 纠错

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI

AI总结 本文针对现有基于LLM的KGQA方法存在的实体剪枝不可靠、约束处理无基础的局限，提出组合式关系链（CCoR）框架，以关系为中心探索实现更忠实的多跳KGQA，在多个基准上性能优于基线。

On the Role of Citations in Preference Data

引用在偏好数据中的作用

Yu Hou, Hal Daumé, Rachel Rudinger, William Walden

机构 * University of Maryland（马里兰大学） ； Johns Hopkins University（约翰斯·霍普金斯大学） 纠错

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI

AI总结 本文在科学问答场景下，通过混合效应模型研究引用对人类与四个开源LLM偏好的影响，发现人类偏好多样且数量少的引用，LLM也有相关偏好，还探讨了对偏好数据收集的启示。

SurgTEMP: Temporal-Aware Surgical Video Question Answering with Text-guided Visual Memory for Laparoscopic Cholecystectomy

SurgTEMP：基于文本引导的视觉记忆的时序感知手术视频问答系统用于腹腔镜胆囊切除术

Shi Li, Vinkle Srivastav, Nicolas Chanel, Saurav Sharma, Nabani Banik, Lorenzo Arboit, Kun Yuan, Pietro Mascagni, Nicolas Padoy

机构 * University of Strasbourg（斯特拉斯堡大学） ； CNRS（法国国家科学研究中心） ； INSERM（法国国家健康与医学研究院） ； ICube, UMR7357（ICube实验室，UMR7357） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

AI总结 SurgTEMP通过文本引导的视觉记忆和时序感知模块，提升手术视频问答的性能，其核心方法是构建层次化的视觉记忆并采用手术能力进步训练方案，从而在复杂手术场景中实现更准确的问答与评估。

Comments 29 pages, 14 figures, 9 tables

Masking Is Not Enough: Generative Restoration for Multimodal De-Identification in Medical AI

仅掩码不足：面向医疗AI的多模态去标识化生成式修复

Shiva Shrestha, Zongxing Xie, Chen Zhao, Liran Ma, Zhipeng Cai, Honghui Xu

机构 * Kennesaw State University（肯尼索州立大学） ； Miami University（迈阿密大学） ； Georgia State University（佐治亚州立大学） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

AI总结 针对医疗图像-文本数据的PHI泄漏问题，提出端到端多模态净化框架ClinX，结合图像侧生成式修复与文本侧渐进式去标识化，在MedVQA中验证其优于仅OCR掩码的性能。

A Collaborative Multi-Modality Interaction for VLA-based End-to-End Autonomous Driving

面向基于视觉-语言-动作（VLA）的端到端自动驾驶的协同多模态交互

Jingtao Sun, Xiaohai He, Yike Zhang, Dong Huang, Yaonan Wang, Ajmal Mian, Mike Zheng Shou

机构 * National University of Singapore (NUS)（新加坡国立大学） ； Hunan University（湖南大学） ； The University of Western Australia (UWA)（西澳大学） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

AI总结 本文针对现有VLA自动驾驶模型决策不可靠、多模态交互不足的问题，提出含三类核心组件的多模态交互与多轨迹规划系统，实验显示其在安全推理与场景感知上优于现有系统。

Enhancing Localized Reasoning for Long Video Understanding via Efficient Segment-to-Video Supervision

通过高效的段级到视频级监督增强长视频理解的局部推理能力

Beibei Zhang, Chao Xu, Jun Lan, Zongyi Li, Lai Wei, Huijia Zhu, Tongwei Ren

机构 * Ant Group（蚂蚁集团） 纠错

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV

AI总结 针对长视频理解中MLLMs易受干扰噪声影响的问题，提出S2V方法，通过段级VQA训练模型，提升了LVU性能与训练推理效率。

CogCanvas: A Benchmark for Evaluating Multi-Subject Reference-Based Image Generation

CogCanvas: 用于评估多主体参考图像生成的基准

Long-Bao Nguyen, Quang-Khai Le, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia Le

机构 * University of Science, Ho Chi Minh City, Vietnam（胡志明市理科大学） ； University of Dayton, Ohio, United States（代顿大学） ； Vietnam National University, Ho Chi Minh City, Vietnam（越南国家大学胡志明市分校） 纠错

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV

AI总结 提出CogCanvas基准，包含1952张参考图像和1361个组合提示，评估多身份、对象绑定和背景场景的生成，引入BG-Sim和Attr-VQA指标，发现现有模型在超过3个主体时性能严重下降。