视觉大模型 / VLM

Robust Pre-Training of Medical Vision-and-Language Models with Domain-Invariant Multi-Modal Masked Reconstruction

具有领域不变多模态掩码重建的医学视觉-语言模型鲁棒预训练

Melika Filvantorkaman, Mohsen Piri

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 本文提出Robust-MMR框架，通过显式建模鲁棒性提升医学视觉-语言模型在跨领域和扰动下的表现，实现更可靠的医疗应用。

Comments 28 pages, 3 figures

VOILA: Value-of-Information Guided Fidelity Selection for Cost-Aware Multimodal Question Answering

VOILA：基于信息价值的保真度选择框架用于成本感知的多模态问答

Rahul Atul Bhope, K. R. Jayaram, Vinod Muthusamy, Ritesh Kumar, Vatche Isahagian, Nalini Venkatasubramanian

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 VOILA通过基于信息价值的自适应保真度选择，在多模态问答中实现成本优化与高精度的平衡。

From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensing

从像素到 prose：推进遥感多模态语言模型

Xintian Sun, Benji Peng, Charles Zhang, Fei Jin, Qian Niu, Junyu Liu, Keyu Chen, Ming Li, Pohsun Feng, Ziqian Bi, Ming Liu, Xinyuan Song, Yichao Zhang

机构 * Simon Fraser University（西蒙弗雷泽大学） ； University of Minnesota - Twin Cities（明尼苏达大学双城分校） ； Kyoto University（京都大学） ； Georgia Institute of Technology（佐治亚理工学院） ； National Taiwan Normal University（台湾师范大学） ； Purdue University（普渡大学） ； Emory University（埃默里大学） ； The University of Texas at Dallas（德克萨斯大学达拉斯分校） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 本文探讨了多模态语言模型在遥感中的应用，分析了其技术基础、挑战及未来发展方向，强调了其在环境监测和灾害响应中的重要作用。

Comments 10 pages, 1 figure

FOCUS: Efficient Keyframe Selection for Long Video Understanding

FOCUS: 长视频理解中的高效关键帧选择

Zirui Zhu, Hailun Xu, Yang Luo, Yong Liu, Kanchan Sarkar, Zhenheng Yang, Yang You

机构 * National University of Singapore（新加坡国立大学） ； TikTok 纠错

专题命中 视觉问答 ：vision-language model(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 FOCUS通过两阶段探索-利用策略，在严格标记预算下高效选择关键帧，提升长视频理解的准确性。

IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs

Ali Faraz, Akash, Shaharukh Khan, Raja Kolla, Akshat Patidar, Suranjan Goswami, Abhinav Ravi, Chandra Khatri, Shubham Agarwal

机构 * Krutrim AI ； OLA Electric 纠错

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Breakdance Video classification in the age of Generative AI

Sauptik Dhar, Naveen Ramakrishnan, Michelle Munson

机构 * Eluvio AI Labs（Eluvio AI实验室） 纠错

专题命中 视觉问答 ：vision language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 11 pages

TALENT: Table VQA via Augmented Language-Enhanced Natural-text Transcription

Guo Yutong, Wanying Wang, Yue Wu, Zichen Miao, Haoyu Wang

机构 * Johns Hopkins University（约翰霍普金斯大学） ； Purdue University（普渡大学） ； University at Albany（阿尔巴尼大学） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)；visual question answering(abstract)

Mind the (Language) Gap: Towards Probing Numerical and Cross-Lingual Limits of LVLMs

Somraj Gautam, Abhirama Subramanyam Penamakuri, Abhishek Bhandari, Gaurav Harit

机构 * Indian Institute of Technology Jodhpur（印度理工学院朱罗普尔） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Traveling Across Languages: Benchmarking Cross-Lingual Consistency in Multimodal LLMs

Hao Wang, Pinzhi Huang, Jihan Yang, Saining Xie, Daisuke Kawahara

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments The first version of this paper mistakenly included a prompt injection phrase, which was inappropriate and unprofessional. Although we corrected the version on arXiv and withdrew from the conference, my co-authors and university strongly request a full withdrawal. Given the situation, I no longer have the authority to manage this paper, and withdrawing it from arXiv is the most responsible action

Structured Captions Improve Prompt Adherence in Text-to-Image Models (Re-LAION-Caption 19M)

Nicholas Merchant, Haitz Sáez de Ocáriz Borde, Andrei Cristian Popescu, Carlos Garcia Jurado Suarez

专题命中 视觉问答 ：LLaVA(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 7-page main paper + appendix, 18 figures

Dual Diffusion for Unified Image Generation and Understanding

Zijie Li, Henry Li, Yichun Shi, Amir Barati Farimani, Yuval Kluger, Linjie Yang, Peng Wang

机构 * Carnegie Mellon University（卡内基梅隆大学） ； Yale University（耶鲁大学） ； ByteDance（字节跳动） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models

Hulingxiao He, Geng Li, Zijun Geng, Jinglin Xu, Yuxin Peng

机构 * Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所） ； School of Intelligence Science and Technology, University of Science and Technology Beijing（北京科技大学智能科学与技术学院） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Published as a conference paper at ICLR 2025. The model is available at https://huggingface.co/StevenHH2000/Finedefics

FlexCap: Describe Anything in Images in Controllable Detail

Debidatta Dwibedi, Vidhi Jain, Jonathan Tompson, Andrew Zisserman, Yusuf Aytar

机构 * Google Deepmind（谷歌DeepMind） ； Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at NeurIPS 2024

ErgoChat: a Visual Query System for the Ergonomic Risk Assessment of Construction Workers

Chao Fan, Qipei Mei, Xiaonan Wang, Xinming Li

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 32 pages, 8 figures

Right this way: Can VLMs Guide Us to See More to Answer Questions?

Li Liu, Diji Yang, Sijia Zhong, Kalyana Suma Sree Tholeti, Lei Ding, Yi Zhang, Leilani H. Gilpin

专题命中 视觉问答 ：vision language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments NeurIPS 2024

Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE

Xun Zhu, Ying Hu, Fanbin Mo, Miao Li, Ji Wu

专题命中 视觉问答 ：visual question answering(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale

Junying Chen, Chi Gui, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji, Guangjun Yu, Xiang Wan, Benyou Wang

专题命中 视觉问答 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Explore until Confident: Efficient Exploration for Embodied Question Answering

Allen Z. Ren, Jaden Clark, Anushri Dixit, Masha Itkina, Anirudha Majumdar, Dorsa Sadigh

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Robotics: Science and Systems (RSS) 2024

BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions

Wenbo Hu, Yifan Xu, Yi Li, Weiyue Li, Zeyuan Chen, Zhuowen Tu

专题命中 视觉问答 ：vision language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at AAAI Conference on Artificial Intelligence (AAAI-24)

RoboVQA: Multimodal Long-Horizon Reasoning for Robotics

Pierre Sermanet, Tianli Ding, Jeffrey Zhao, Fei Xia, Debidatta Dwibedi, Keerthana Gopalakrishnan, Christine Chan, Gabriel Dulac-Arnold, Sharath Maddineni, Nikhil J Joshi, Pete Florence, Wei Han, Robert Baruch, Yao Lu, Suvir Mirchandani, Peng Xu, Pannag Sanketi, Karol Hausman, Izhak Shafran, Brian Ichter, Yuan Cao

专题命中 视觉问答 ：VLM(abstract)；visual language model(abstract)；visual question answering(abstract)

Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis

Zhu Wang, Sourav Medya, Sathya N. Ravi

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning

Qian Jiang, Changyou Chen, Han Zhao, Liqun Chen, Qing Ping, Son Dinh Tran, Yi Xu, Belinda Zeng, Trishul Chilimbi

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 14 pages, 8 figure, CVPR 2023 accepted

Visually Grounded VQA by Lattice-based Retrieval

Daniel Reich, Felix Putze, Tanja Schultz

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

VisFIS: Visual Feature Importance Supervision with Right-for-the-Right-Reason Objectives

Zhuofan Ying, Peter Hase, Mohit Bansal

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments NeurIPS 2022 (first two authors contributed equally)

SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency

Sameer Dharur, Purva Tendulkar, Dhruv Batra, Devi Parikh, Ramprasaath R. Selvaraju

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to the NeurIPS 2020 workshop on Interpretable Inductive Biases and Physically Structured Learning

X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers

Jaemin Cho, Jiasen Lu, Dustin Schwenk, Hannaneh Hajishirzi, Aniruddha Kembhavi

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments EMNLP 2020

Neural Reasoning, Fast and Slow, for Video Question Answering

Thao Minh Le, Vuong Le, Svetha Venkatesh, Truyen Tran

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Journal ref International Joint Conference on Neural Networks (IJCNN) 2020

From Pixels to Feelings: Aligning MLLMs with Human Cognitive Perception of Images

从像素到感受：将MLLMs对齐于人类对图像的认知感知

Yiming Chen, Junlin Han, Tianyi Bai, Shengbang Tong, Filippos Kokkinos, Philip Torr

机构 * Oxford University（牛津大学） ； HKUST（香港科技大学） ； University College London（伦敦大学学院） ； New York University（纽约大学） 纠错

专题命中 视觉问答 ：MLLM(abstract,comments)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

AI总结 本文提出CogIP-Bench基准，通过后训练提升MLLMs对图像认知属性的对齐能力，并展示其在图像生成中的应用。

Comments Project page with codes/datasets/models: https://follen-cry.github.io/MLLM-Cognition-project-page/

Multi-Task Learning for Visually Grounded Reasoning in Gastrointestinal VQA

Itbaan Safwan, Muhammad Annas Shaikh, Muhammad Haaris, Ramail Khan, Muhammad Atif Tahir

机构 * School of Mathematics and Computer Science, Institute of Business Administration (IBA), Karachi, Pakistan（数学与计算机科学学院，商学院（IBA），巴基斯坦卡里奇） 纠错

专题命中 视觉问答 ：visual question answering(abstract,comments)；grounding(abstract)；分类 cs.CV、cs.LG

Comments This is a working paper submitted for Medico 2025: Visual Question Answering (with multimodal explanations) for Gastrointestinal Imaging at MediaEval 2025. 5 pages, 3 figures and 1 table

CrossView: Can Vision-Language Models Reason Across Cameras?

CrossView：视觉-语言模型能否跨相机进行推理？

Sahil Shah, S P Sharan, Harsh Goel, Manvik Pasula, Adithya Hebbalae, Minkyu Choi, Sandeep P. Chinchali

机构 * The University of Texas at Austin（德克萨斯大学奥斯汀分校） 纠错

专题命中 视觉问答 ：vision-language model(title)；分类 cs.CV

AI总结 本文提出CrossView多相机视频问答基准，评估发现GPT-5.2等模型跨相机推理准确率低，开源模型表现更差，该基准可用于测试模型联合处理多视角的能力。

Comments ECCV 2026