视觉大模型 / VLM - arXivDaily 专题

2305.16328 2023-05-29 cs.CL cs.LG 70%

Semantic Composition in Visually Grounded Language Models

Rohan Pandey

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.LG

Comments Carnegie Mellon University Senior Thesis. arXiv admin note: substantial text overlap with arXiv:2212.10549

2303.02635 2023-03-07 cs.CV cs.MM 70%

VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning

Kang Chen, Xiangqian Wu

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

2301.09045 2023-02-21 cs.CV 70%

Champion Solution for the WSDM2023 Toloka VQA Challenge

Shengyi Gao, Zhe Chen, Guo Chen, Wenhai Wang, Tong Lu

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Technical report in WSDM Cup 2023

2208.09374 2022-08-22 cs.CV 70%

VLMAE: Vision-Language Masked Autoencoder

Sunan He, Taian Guo, Tao Dai, Ruizhi Qiao, Chen Wu, Xiujun Shu, Bo Ren

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments 12 pages, 7 figures

2208.03030 2022-08-08 cs.CL cs.CV 70%

ChiQA: A Large Scale Image-based Real-World Question Answering Dataset for Multi-Modal Understanding

Bingning Wang, Feiyang Lv, Ting Yao, Yiming Yuan, Jin Ma, Yu Luo, Haijin Liang

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments CIKM2022 camera ready version

2201.10963 2022-05-31 cs.CV cs.MM 70%

Learning to Compose Diversified Prompts for Image Emotion Classification

Sinuo Deng, Lifang Wu, Ge Shi, Lehao Xing, Meng Jian, Ye Xiang

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments 10 pages, 5 figures

2106.13488 2021-11-10 cs.CV 70%

Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training

Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2021

2108.08965 2021-08-23 cs.CV cs.CL 70%

Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling

Xiaopeng Lu, Zhen Fan, Yansen Wang, Jean Oh, Carolyn P. Rose

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments 9 pages

1911.09655 2019-11-22 cs.CL cs.LG cs.SD eess.AS 70%

Temporal Reasoning via Audio Question Answering

Haytham M. Fayek, Justin Johnson

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.LG

1905.06139 2019-11-05 cs.CL cs.CV 70%

Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations

Fenglin Liu, Yuanxin Liu, Xuancheng Ren, Xiaodong He, Xu Sun

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2019

1902.03751 2019-10-29 cs.CV 70%

Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded

Ramprasaath R. Selvaraju, Stefan Lee, Yilin Shen, Hongxia Jin, Shalini Ghosh, Larry Heck, Dhruv Batra, Devi Parikh

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Published at ICCV'2019

Journal ref The IEEE International Conference on Computer Vision (ICCV) 2019

1902.09368 2019-08-30 cs.CV 70%

Dual Attention Networks for Visual Reference Resolution in Visual Dialog

Gi-Cheon Kang, Jaeseo Lim, Byoung-Tak Zhang

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments EMNLP 2019

1908.06327 2019-08-20 cs.CV cs.CL 70%

Language Features Matter: Effective Language Representations for Vision-Language Tasks

Andrea Burns, Reuben Tan, Kate Saenko, Stan Sclaroff, Bryan A. Plummer

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments ICCV 2019 accepted paper

1908.04107 2019-08-20 cs.CV 70%

Multimodal Unified Attention Networks for Vision-and-Language Interactions

Zhou Yu, Yuhao Cui, Jun Yu, Dacheng Tao, Qi Tian

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments 11 pages, 7 figures

1907.06794 2019-08-20 cs.CV 70%

2nd Place Solution to the GQA Challenge 2019

Shijie Geng, Ji Zhang, Hang Zhang, Ahmed Elgammal, Dimitris N. Metaxas

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

1901.06706 2019-01-23 cs.CV 70%

Visual Entailment: A Novel Task for Fine-Grained Image Understanding

Ning Xie, Farley Lai, Derek Doran, Asim Kadav

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

1812.00500 2018-12-04 cs.CV 70%

Multi-task Learning of Hierarchical Vision-Language Representation

Duy-Kien Nguyen, Takayuki Okatani

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

1709.07992 2018-08-08 cs.CV 70%

Visual Reference Resolution using Attention Memory for Visual Dialog

Paul Hongsuck Seo, Andreas Lehrmann, Bohyung Han, Leonid Sigal

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

2608.25422 2026-08-27 cs.IT math.IT 新提交 67%

Towards Faithful and Efficient Semantic Communication: An Ontological Approach

面向忠实且高效的语义通信：一种本体论方法

Yixiao Feng, Yueting Wang, Yining Wang, Han Han, Bo Zhang

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)

AI总结针对多视图VQA任务，提出ODSC本体驱动语义通信框架，通过共享本体知识库优化场景图，可减少SI数据量、提升问答准确率及多视图VQA准确率。

2608.22688 2026-08-25 cs.IR cs.MM 新提交 67%

FashionKG-RAG: Knowledge Graph-Enhanced Retrieval-Augmented Generation for Fashion Question Answering

FashionKG-RAG：面向时尚问答的知识图谱增强检索增强生成

Yujuan Ding, Linyin Luo, Shijie Wang, Xu Yuan, Yunshan Ma, Yi Bin, Wenqi Fan, Qing Li

专题命中视觉问答：grounding(abstract,abstract_cn)

AI总结针对现有时尚知识图谱的局限性，本文提出全领域知识图谱FashionEcoKG，并开发无需训练的PG-RAG框架，通过双粒度路径重排序模块提升时尚问答的检索与答案准确性，效果优于多种基线方法。

2608.06571 2026-08-10 cs.CR cs.CL 新提交 67%

Model Confidence Under Answer-Preserving Attacks: An Informativeness-Manipulability Frontier

答案保留型攻击下的模型置信度：信息性-可操纵性前沿

Reza Khanmohammadi, Ivan Brugere, Simerjot Kaur, Charese H. Smiley, Kundan Thind, Mohammad M. Ghassemi

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)

AI总结该研究针对答案保留型攻击，发现视觉-语言系统的置信度信号不具备固有鲁棒性，四类防御均无效，协调攻击可大幅降低置信度门控下的接受准确率。

2604.00086 2026-08-04 cs.CV cs.AI cs.CL cs.LG 版本更新 67%

Hierarchical Pre-Training of Vision Encoders with Large Language Model

基于大语言模型的视觉编码器分层预训练

Eugene Lee, Ting-Yu Chang, Jui-Huang Tsai, Jiajie Diao, Chen-Yi Lee

机构 * University of Cincinnati（辛辛那提大学）； National Yang Ming Chiao Tung University（国立阳明交通大学）

专题命中视觉问答：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结本文提出HIVE框架，通过引入视觉编码器与大语言模型间的分层交叉注意力机制，提升视觉语言对齐，改进特征融合与表征学习，实验表明其在图像分类和多模态任务中表现优异。

Comments 17 pages, 14 figures, accepted to Computer Vision and Pattern Recognition Conference (CVPR) Workshops 2026. 5th MMFM Workshop: What is Next in Multimodal Foundation Models?

Journal ref In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7415-7424) 2026

2605.09635 2026-07-24 cs.CL 版本更新 67%

K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs

K12-KGraph：一种对齐课程的图谱用于基准测试和训练教育大语言模型

Hao Liang, Qihan Lin, Zhaoyang Han, Xiaochen Ma, Zhen Hao Wong, Meiyi Qiang, Linzhuang Sun, Wentao Zhang

机构 * Peking University（北京大学）； Institute for Advanced Algorithms Research（先进算法研究所）； OriginHub Technology（OriginHub技术）； Zhongguancun Academy（中关村学院）

专题命中视觉问答：vision-language model(abstract)；grounding(abstract)

AI总结本文提出K12-KGraph，基于中小学教材构建的课程对齐知识图谱，用于构建多选基准测试K12-Bench和训练数据集K12-Train，验证课程结构监督在教育模型训练中的高效性。

2606.16583 2026-06-16 cs.CL 新提交 67%

Uncertainty Is Not a Safety Net for Clinical VQA, but Can It Anticipate Model Failure?

不确定性并非临床VQA的安全网，但它能预测模型失败吗？

Arnisa Fazla, Alberto Testoni, Ameen Abu-Hanna, Barbara Plank, Iacer Calixto

机构 * Amsterdam University Medical Center, University of Amsterdam（阿姆斯特丹大学医学中心）； Amsterdam Public Health（阿姆斯特丹公共卫生）； LMU Munich（慕尼黑大学）； Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）

专题命中视觉问答：vision-language model(abstract)；VLM(abstract_cn)

AI总结研究临床视觉语言模型的不确定性估计是否可靠，发现其质量随模型准确率变化，在模型脆弱时失效，但能预测扰动下的性能崩溃。

Comments 17 pages, 4 figures

2605.25813 2026-05-26 cs.RO 67%

Extending Embodied Question Answering from Perception to Decision

将具身问答从感知扩展到决策

Xicheng Gong, Qiwei Li, Peiran Xu, Yadong Mu

机构 * Peking University（北京大学）； XYZ Embodied AI（XYZ具身AI）

专题命中视觉问答：VLM(abstract,abstract_cn)

AI总结提出大规模具身问答数据集EQA-Decision和基线模型RoboDecision，系统覆盖静态场景构建、空间理解、任务动态推理和即时决策四个维度，以统一框架评估具身环境中的感知、推理和行动级决策。

Comments 11 pages,4 figures

2605.22099 2026-05-22 cs.CL 67%

A Comparative Study of Language Models for Khmer Retrieval-Augmented Question Answering

一种用于柬埔寨检索增强问答的语言模型比较研究

Sereiwathna Ros, Phannet Pov, Ratanaktepi Chhor, Kimleang Ly, Wan-Sup Cho, Saksonita Khoeurn

机构 * Department of Computer Science, Chungbuk National University（Chungbuk National University 计算机科学系）； Department of Big Data, Chungbuk National University（Chungbuk National University 大数据系）； General Department of Information and Communication Technology, Ministry of Post and Telecommunications（邮电部信息和通信技术总局）； Department of Management Information Systems, Chungbuk National University（Chungbuk National University 管理信息系统系）； BigDatalabs Co., Ltd（BigDatalabs 公司）

专题命中视觉问答：grounding(abstract,abstract_cn)

AI总结本文针对低资源非拉丁语种柬埔寨语言，比较了多种语言模型在检索增强问答任务中的性能，发现检索器选择是影响效果的关键因素，生成器在不同指标上表现各异。

Comments 14 pages, 1 figure,

2505.24840 2026-03-27 cs.CV cs.AI cs.CL cs.LG 67%

The LLM Bottleneck: Why Open-Source Vision LLMs Struggle with Hierarchical Visual Recognition

大语言模型的瓶颈：为何开源视觉大语言模型在层级视觉识别上遇到困难

Yuwen Tan, Yuan Qing, Boqing Gong

机构 * Boston University（波士顿大学）

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结本文指出开源大语言模型缺乏对视觉世界的层级知识，导致视觉大语言模型在识别如水母鱼但无法识别脊椎动物时存在瓶颈，通过构建六种分类学和四个图像数据集的百万级多项选择视觉问答任务验证了这一问题。

Comments Accepted to CVPR 2026. Project page and code: https://yuanqing-ai.github.io/llm-hierarchy/

2603.15409 2026-03-17 cs.CL 67%

SEA-Vision: A Multilingual Benchmark for Comprehensive Document and Scene Text Understanding in Southeast Asia

SEA-Vision：面向东南亚的多语言综合文档和场景文本理解基准

Pengfei Yue, Xingran Zhao, Juntao Chen, Peng Hou, Wang Longchao, Jianghang Lin, Shengchuan Zhang, Anxiang Zeng, Liujuan Cao

机构 * Xiamen University, China（厦门大学，中国）； Shopee, China（Shopee，中国）； Tongji University, China（同济大学，中国）

专题命中视觉问答：visual question answering(abstract)；MLLM(abstract)

AI总结 SEA-Vision提出一个多语言基准，用于评估文档解析和文本中心视觉问答，涵盖11种东南亚语言，包含15234页文档和7496对问答对，揭示多语言文档理解的差距。

Comments Accepted By CVPR2026

2305.17066 2026-03-12 cs.AI cs.CL cs.CV cs.LG cs.MA 67%

Mindstorms in Natural Language-Based Societies of Mind

自然语言基础的思维社会中的风暴

Mingchen Zhuge, Haozhe Liu, Francesco Faccio, Dylan R. Ashley, Róbert Csordás, Anand Gopalakrishnan, Abdullah Hamdi, Hasan Abed Al Kader Hammoud, Vincent Herrmann, Kazuki Irie, Louis Kirsch, Bing Li, Guohao Li, Shuming Liu, Jinjie Mai, Piotr Piękos, Aditya Ramesh, Imanol Schlag, Weimin Shi, Aleksandar Stanić, Wenyi Wang, Yuhui Wang, Mengmeng Xu, Deng-Ping Fan, Bernard Ghanem, Jürgen Schmidhuber

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结本文探讨了基于自然语言的思维社会（NLSOMs）的结构和应用，通过多代理系统解决多种AI任务，并提出未来研究方向。

Comments published in Computational Visual Media Journal (CVMJ); 9 pages in main text + 7 pages of references + 38 pages of appendices, 14 figures in main text + 13 in appendices, 7 tables in appendices

Journal ref (2025). Computational Visual Media, 11(1), 29-81

2602.12203 2026-02-13 cs.CL 67%

ExStrucTiny: A Benchmark for Schema-Variable Structured Information Extraction from Document Images

ExStrucTiny：一种用于从文档图像中进行模式变量结构信息提取的基准

Mathieu Sibue, Andres Muñoz Garza, Samuel Mensah, Pranav Shetty, Zhiqiang Ma, Xiaomo Liu, Manuela Veloso

机构 * J.P. Morgan AI Research（摩根大通人工智能研究）

专题命中视觉问答：vision language model(abstract)；visual question answering(abstract)

AI总结 ExStrucTiny是一个新的文档图像结构信息提取基准，通过结合手动和合成样本，涵盖更多多样化的文档类型和提取场景，旨在提升通用模型在结构化信息提取中的性能。

Comments EACL 2026, main conference

1. 视觉问答 3164 篇

Semantic Composition in Visually Grounded Language Models

VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning

Champion Solution for the WSDM2023 Toloka VQA Challenge

VLMAE: Vision-Language Masked Autoencoder

ChiQA: A Large Scale Image-based Real-World Question Answering Dataset for Multi-Modal Understanding

Learning to Compose Diversified Prompts for Image Emotion Classification

Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training

Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling

Temporal Reasoning via Audio Question Answering

Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations

Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded

Dual Attention Networks for Visual Reference Resolution in Visual Dialog

Language Features Matter: Effective Language Representations for Vision-Language Tasks

Multimodal Unified Attention Networks for Vision-and-Language Interactions

2nd Place Solution to the GQA Challenge 2019

Visual Entailment: A Novel Task for Fine-Grained Image Understanding

Multi-task Learning of Hierarchical Vision-Language Representation

Visual Reference Resolution using Attention Memory for Visual Dialog

Towards Faithful and Efficient Semantic Communication: An Ontological Approach

FashionKG-RAG: Knowledge Graph-Enhanced Retrieval-Augmented Generation for Fashion Question Answering

Model Confidence Under Answer-Preserving Attacks: An Informativeness-Manipulability Frontier

Hierarchical Pre-Training of Vision Encoders with Large Language Model

K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs

Uncertainty Is Not a Safety Net for Clinical VQA, but Can It Anticipate Model Failure?

Extending Embodied Question Answering from Perception to Decision

A Comparative Study of Language Models for Khmer Retrieval-Augmented Question Answering

The LLM Bottleneck: Why Open-Source Vision LLMs Struggle with Hierarchical Visual Recognition

SEA-Vision: A Multilingual Benchmark for Comprehensive Document and Scene Text Understanding in Southeast Asia

Mindstorms in Natural Language-Based Societies of Mind

ExStrucTiny: A Benchmark for Schema-Variable Structured Information Extraction from Document Images