视觉大模型 / VLM

Pyramid Coder: Hierarchical Code Generator for Compositional Visual Question Answering

Ruoyue Shen, Nakamasa Inoue, Koichi Shinoda

专题命中 视觉问答 ：visual question answering(title,abstract)；visual reasoning(abstract)；分类 cs.CV、cs.AI

Comments Accepted to the IEEE International Conference on Image Processing (IEEE ICIP) 2024

FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts

Shubhankar Singh, Purvi Chaurasia, Yerram Varun, Pranshu Pandya, Vatsal Gupta, Vivek Gupta, Dan Roth

专题命中 视觉问答 ：visual question answering(title,abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments Accepted in ACL 2024 (Findings), 21 pages, 7 figures, 9 Tables

Evaluating Zero-Shot GPT-4V Performance on 3D Visual Question Answering Benchmarks

Simranjit Singh, Georgios Pavlakos, Dimitrios Stamoulis

专题命中 视觉问答 ：visual question answering(title,abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments Accepted at 1st Workshop on Multimodalities for 3D Scenes CVPR 2024

VSA4VQA: Scaling a Vector Symbolic Architecture to Visual Question Answering on Natural Images

Anna Penzkofer, Lei Shi, Andreas Bulling

专题命中 视觉问答 ：visual question answering(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI

Comments To be published in the Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci'24)

Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy

Simon Ging, María A. Bravo, Thomas Brox

专题命中 视觉问答 ：vision-language model(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Accepted as Spotlight Paper for ICLR 2024. The first two authors contributed equally to this work

When are Lemons Purple? The Concept Association Bias of Vision-Language Models

Yutaro Yamada, Yingtian Tang, Yoyo Zhang, Ilker Yildirim

专题命中 视觉问答 ：vision-language model(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Comments EMNLP 2023 main

Vision-Language Models in Remote Sensing: Current Progress and Future Trends

Xiang Li, Congcong Wen, Yuan Hu, Zhenghang Yuan, Xiao Xiang Zhu

专题命中 视觉问答 ：vision-language model(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted by IEEE Geoscience and Remote Sensing Magazine

LCV2: An Efficient Pretraining-Free Framework for Grounded Visual Question Answering

Yuhan Chen, Lumei Su, Lihua Chen, Zhiwei Lin

专题命中 视觉问答 ：visual question answering(title,abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments 21 pages,9 figures

Robust Visual Question Answering: Datasets, Methods, and Future Challenges

Jie Ma, Pinghui Wang, Dechen Kong, Zewei Wang, Jun Liu, Hongbin Pei, Junzhou Zhao

专题命中 视觉问答 ：visual question answering(title,abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments Accepted by IEEE TPAMI

How to Configure Good In-Context Sequence for Visual Question Answering

Li Li, Jiawei Peng, Huiyi Chen, Chongyang Gao, Xu Yang

专题命中 视觉问答 ：visual question answering(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 8 pages, 6 figures

Toloka Visual Question Answering Benchmark

Dmitry Ustalov, Nikita Pavlichenko, Sergey Koshelev, Daniil Likhobaba, Alisa Smirnova

专题命中 视觉问答 ：visual question answering(title,abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments 16 pages; see https://toloka.ai/challenges/wsdm2023/ for more details

LOIS: Looking Out of Instance Semantics for Visual Question Answering

Siyu Zhang, Yeming Chen, Yaoru Sun, Fang Wang, Haibo Shi, Haoran Wang

专题命中 视觉问答 ：visual question answering(title,abstract)；visual reasoning(abstract)；分类 cs.CV、cs.AI

Multimodal Prompt Retrieval for Generative Visual Question Answering

Timothy Ossowski, Junjie Hu

专题命中 视觉问答 ：visual question answering(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI

Chain of Thought Prompt Tuning in Vision Language Models

Jiaxin Ge, Hongyin Luo, Siyuan Qian, Yulu Gan, Jie Fu, Shanghang Zhang

专题命中 视觉问答 ：vision language model(title)；vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

SimVLM: Simple Visual Language Model Pretraining with Weak Supervision

Zirui Wang, Jiahui Yu, Adams Wei Yu, Zihang Dai, Yulia Tsvetkov, Yuan Cao

专题命中 视觉问答 ：visual language model(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Published at ICLR 2022

Visual Question Answering based on Formal Logic

Muralikrishnna G. Sethuraman, Ali Payani, Faramarz Fekri, J. Clayton Kerce

专题命中 视觉问答 ：visual question answering(title,abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Separating Skills and Concepts for Novel Visual Question Answering

Spencer Whitehead, Hui Wu, Heng Ji, Rogerio Feris, Kate Saenko

专题命中 视觉问答 ：visual question answering(title,abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments Paper at CVPR 2021. 14 pages, 7 figures

Adapting Visual Question Answering Models for Enhancing Multimodal Community Q&A Platforms

Avikalp Srivastava, Hsin Wen Liu, Sumio Fujita

专题命中 视觉问答 ：visual question answering(title,abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments Submitted for review at CIKM 2019

Second Place Solution of WSDM2023 Toloka Visual Question Answering Challenge

Xiangyu Wu, Zhouyang Chi, Yang Yang, Jianfeng Lu

专题命中 视觉问答 ：visual question answering(title,abstract)；grounding(abstract)；分类 cs.CV

Comments Second Place of WSDM2023 Toloka Visual Question Answering Challenge

AWM: Answerable Working Memory for Long-Document VQA Agents

AWM：面向长文档VLM智能体的可回答工作记忆

Dongzhuoran Zhou, Yuqicheng Zhu, Yule Liu, Zhen Yang, Rui Lu, Yuxiao Dong, Jie Tang, Evgeny Kharlamov

机构 * University of Oslo（奥斯陆大学） ； Bosch Center for AI（博世人工智能中心） ； University of Stuttgart（斯图加特大学） ； Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； Tsinghua University（清华大学） 纠错

专题命中 视觉问答 ：VLM(title_cn,abstract)；visual question answering(abstract)

AI总结 针对长文档VQA智能体的记忆质量盲区，提出AWM方法，通过融入仅记忆可回答性的GRPO奖励机制，提升了最终答案准确率并降低记忆缺失正确的比例。

Comments EMNLP 2026 Findings. 16 pages, 4 figures, 9 tables

Debiasing Text-to-Image Evaluation via Implicit Cultural Alignment Reward Modeling

通过隐式文化对齐奖励建模消除文本到图像评估中的偏差

Bo-An Chang, Yu-Chih Chen

机构 * National Tsing Hua University（国立清华大学） ； National Yang Ming Chiao Tung University（国立阳明交通大学） 纠错

专题命中 视觉问答 ：MLLM(abstract,abstract_cn)；visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 研究文本到图像评估中文化真实性问题，提出基于轻量级多模态大语言模型构建的隐式文化对齐奖励模型，集成隐式文化探测器与跳跃连接交叉注意力机制，实验证明该模型准确率高、速度快，能为偏好优化管道提供有效信号。

Comments 16 pages, 2 figures, ECCV 2026 Workshop FAILED

CHaystack: Benchmarking Chinese Document Retrieval and VQA

CHaystack：中文文档检索与视觉问答基准测试

Hanxi Li

专题命中 视觉问答 ：VLM(summary_cn,abstract)；visual question answering(abstract)

AI总结 本文介绍了CHaystack中文文档检索与视觉问答基准测试，涵盖四类文档。提出CDocRAG系统，用VLM相关性过滤器验证文档图像。评估开源模型发现Qwen系列在文本丰富文档表现佳，中文大规模DocumentVQA在文本编码方面有挑战，仍需改进。

HueManity: Probing Fine-Grained Visual Perception in MLLMs

HueManity: 探索 MLLMs 中的细粒度视觉感知

Rynaa Grover, Jayant Sravan Tamarapalli, Sahiti Yerramilli, Nilay Pande

机构 * Google（谷歌） ； Waymo 纠错

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)；grounding(abstract)；multimodal large language model(abstract)

AI总结 HueManity 通过细粒度视觉感知基准测试揭示 MLLMs 在捕捉细粒度视觉细节方面的显著缺陷。

Journal ref ICML 2025 Workshop on Assessing World Models

Beyond Atomic Layouts: Compositional Design Understanding with Vision-Language Models

超越原子布局：基于视觉-语言模型的组合式设计理解

Yiyang Huang, Zhaowen Wang, Simon Jenni, Jing Shi, Yitian Zhang, Yizhou Wang, Yun Fu

机构 * Northeastern University（东北大学） ； Adobe Research（奥多比研究院） 纠错

专题命中 视觉问答 ：vision-language model(title,abstract)；grounding(abstract)；分类 cs.CV

AI总结 本文针对组合式布局理解任务，提出后训练范式MASON，利用约2万布局的CoDeLayout数据集，使Qwen2.5-VL 7B准确率达91.66%，优于基线及全数据微调。

Comments ECCV 2026

GRACE: Grounded Reasoning via Adapter Composition and Evidence-Aware Calibration for Educational Visual Question Answering

GRACE：基于适配器组合与证据感知校准的教育视觉问答的接地推理

Xinjin Li, Yudi Xia, Xi Zhao, Yiliu Xu, Yining Liu, Cheng Lu, Yujian Long, Yu Ma, Jinghan Cao, Liang Fan, Yeyun Xu

机构 * Columbia University（哥伦比亚大学） ； Carnegie Mellon University（卡内基梅隆大学） ； University of California, Berkeley（加州大学伯克利分校） ； Stevens Institute of Technology（史蒂文斯理工学院） ； Georgetown University（乔治城大学） ； San Francisco State University（旧金山州立大学） ； Loughborough University（拉夫堡大学） ； Texas A&M University（德克萨斯农工大学） 纠错

专题命中 视觉问答 ：visual question answering(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

AI总结 针对教育视觉问答的问题-选项捷径问题，提出GRACE框架，利用结构化教育状态实现参数高效多模态适配，在ScienceQA上提升了多项准确率。

Reinforcing Egocentric Spatial Perception in Multimodal Large Language Models via Ego Scene Augmentation

通过自我场景增强在多模态大语言模型中强化自我中心空间感知

Chi Kit Wong, Ye Pan, Yuanhuiyi Lyu, Xu Zheng, Zidong Cao, Lutao Jiang, Zixin Zhang, Huiyu Zhou, Xuming Hu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； Guangxi Zhuang Autonomous Region Information Center（广西壮族自治区信息中心） ； The Hong Kong University of Science and Technology（香港科技大学） 纠错

专题命中 视觉问答 ：multimodal large language model(title,abstract)；visual question answering(abstract)；分类 cs.CV

AI总结 研究如何强化多模态大语言模型的自我中心空间感知，提出自我场景增强框架ESA，利用自我元素图作为中间表示，通过视觉基础模型增强空间感知，在EgoTextVQA基准上取得显著性能提升。

Comments 14 pages, 8 figures. Chi Kit Wong and Ye Pan contributed equally. Code: https://github.com/Chikit-WONG/spatialGraph

Reasoning for Social Audio-Visual Question Answering: Where Do We Stand?

社会视听问答的推理：我们处于什么阶段？

Koen P. de Vries, Xavier Alameda-Pineda, Estefanía Talavera, Stéphane Lathuilière

机构 * Inria（法国国家信息与自动化研究所） ； Univ. Grenoble Alpes（格勒诺布尔大学） ； CNRS（法国国家科学研究中心） ； University of Twente（特文特大学） 纠错

专题命中 视觉问答 ：visual question answering(title)；multimodal large language model(abstract,abstract_cn)；分类 cs.CV

AI总结 本文针对社会视听问答研究，发现IntentBench存在高噪声，Vanilla SFT基线性能优于现有推理方法，仅用文本模态即可实现与视频相当的性能，并发布了IntentBench-Prime等资源。

Comments Accepted at HCMIW ECCV workshop. Code available here: https://github.com/koenv759/VanillaSFT

LHSDet: High-Resolution AI-Generated Image Detection via Visual Question Answering

LHSDet：基于视觉问答的高分辨率AI生成图像检测方法

Qian Yao, Jun-Jie Huang, Yongjun Wang, Luming Yang

机构 * College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机学院） ； Academy of Military Science（军事科学院） 纠错

专题命中 视觉问答 ：visual question answering(title,abstract)；vision-language model(abstract)；分类 cs.CV

AI总结 针对现有AI生成图像检测方法忽略高分辨率细节、难以应对未知生成模型的问题，提出LHSDet，将检测任务转为视觉问答，采用三分支架构，在各类生成模型上实现高检测准确率与稳健性能。

Improving Large Vision-Language Models' Understanding for Flow Field Data

提升大型视觉-语言模型对流场数据的理解能力

Xiaomei Zhang, Hanyu Zheng, Xiangyu Zhu, Jinghuan Wei, Junhong Zou, Zhen Lei, Zhaoxiang Zhang

机构 * State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院） 纠错

专题命中 视觉问答 ：vision-language model(title,abstract)；visual question answering(abstract)；分类 cs.CV

AI总结 FieldLVLM通过场感知语言生成策略和数据压缩多模态模型调优，提升大型视觉-语言模型对流场数据的理解能力。

Comments Accepted by Machine Intelligence Research

Can Multimodal Large Language Models Understand OCT?

多模态大语言模型能理解光学相干断层扫描（OCT）吗？

Baochen Fu, Wenzhi Deng, Baihao Jin, Yang Li, Zihan Nie, Kailin Jiang, Yuntao Du, Weiye Song

专题命中 视觉问答 ：multimodal large language model(title,abstract)；visual question answering(abstract)；分类 cs.CV

AI总结 研究探讨多模态大语言模型能否理解OCT，引入OCT - Bench基准，包含多维度细粒度任务，基于多个数据集构建大量选择题，评估20个代表性模型，发现当前模型理解OCT能力不足，为评估模型和推动OCT理解提供基础。