视觉大模型 / VLM

VideoQA in the Era of LLMs: An Empirical Study

Junbin Xiao, Nanxin Huang, Hangyu Qin, Dongyang Li, Yicong Li, Fengbin Zhu, Zhulin Tao, Jianxing Yu, Liang Lin, Tat-Seng Chua, Angela Yao

机构 * National University of Singapore（新加坡国立大学） ； Communication University of China（中国传媒大学） ； Sun Yat-Sen University（中山大学） 纠错

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV、cs.AI

Comments IJCV'25

Dynamic Double Space Tower

Weikai Sun, Shijie Song, Han Wang

机构 * Huawei Cloud Computing（华为云计算） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Kvasir-VQA-x1: A Multimodal Dataset for Medical Reasoning and Robust MedVQA in Gastrointestinal Endoscopy

Sushant Gautam, Michael A. Riegler, Pål Halvorsen

机构 * Simula Metropolitan Center for Digital Engineering (SimulaMet)（Simula Metropolitan Center for Digital Engineering） ； Oslo Metropolitan University (OsloMet)（Oslo Metropolitan University） ； Simula Research Laboratory 纠错

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Outside Knowledge Conversational Video (OKCV) Dataset -- Dialoguing over Videos

Benjamin Reichman, Constantin Patsch, Jack Truxal, Atishay Jain, Larry Heck

机构 * Georgia Institute of Technology（佐治亚理工学院） ； Technical University of Munich（慕尼黑技术大学） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

From Symbolic to Neural and Back: Exploring Knowledge Graph-Large Language Model Synergies

Blaž Škrlj, Boshko Koloski, Senja Pollak, Nada Lavrač

机构 * Jožef Stefan Institute（乔泽夫·斯塔芬研究所） 纠错

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments To-appear as a book chapter

EgoVLM: Policy Optimization for Egocentric Video Understanding

Ashwin Vinod, Shrey Pandit, Aditya Vavre, Linshen Liu

机构 * The University of Texas at Austin（德克萨斯大学奥斯汀分校） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Our Code can be found at https://github.com/adityavavre/VidEgoVLM

Fire360: A Benchmark for Robust Perception and Episodic Memory in Degraded 360-Degree Firefighting Videos

Aditi Tiwari, Farzaneh Masoud, Dac Trong Nguyen, Jill Kraft, Heng Ji, Klara Nahrstedt

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； Illinois Fire Service Institute（伊利诺伊州消防服务研究所） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 20 pages, 9 figures, 6 tables

NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQA

Marlon Tobaben, Mohamed Ali Souibgui, Rubèn Tito, Khanh Nguyen, Raouf Kerkouche, Kangsoo Jung, Joonas Jälkö, Lei Kang, Andrey Barsky, Vincent Poulain d'Andecy, Aurélie Joseph, Aashiq Muhamed, Kevin Kuo, Virginia Smith, Yusuke Yamasaki, Takumi Fukami, Kenta Niwa, Iifan Tyou, Hiro Ishii, Rio Yokota, Ragul N, Rintu Kutum, Josep Llados, Ernest Valveny, Antti Honkela, Mario Fritz, Dimosthenis Karatzas

机构 * University of Helsinki（赫尔辛基大学） ； Computer Vision Center, Universitat Autònoma de Barcelona（巴塞罗那自治大学计算机视觉中心） ； CISPA Helmholtz Center for Information Security（信息安全赫尔姆霍茨中心） ； INRIA（法国国家信息与自动化技术研究所） ； Yooz ； Carnegie Mellon University（卡内基梅隆大学） ； NTT（日本NTT公司） ； Institute of Science Tokyo（东京科学研究所） ； Department of Computer Science（计算机科学系） ； Mphasis AI & Applied Tech Lab at Ashoka, Ashoka University（阿什oka大学人工智能与应用技术实验室） ； Koita Centre for Digital Health - Ashoka (KCDH-A)（阿什oka数字健康中心（KCDH-A）） ； Trivedi School of Biosciences, Ashoka University（阿什oka大学Trivedi生物科学学院） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments 33 pages, 7 figures; published in TMLR 06/2025 https://openreview.net/forum?id=3HKNwejEEq

Journal ref Transactions on Machine Learning Research, ISSN 2835-8856, 2025

Progressive Prompt Detailing for Improved Alignment in Text-to-Image Generative Models

Ketan Suhaas Saichandran, Xavier Thomas, Prakhar Kaushik, Deepti Ghadiyaram

机构 * Boston University（波士顿大学） ； Johns Hopkins University（约翰霍普金斯大学） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted at CVPR 2025 workshops (AI4CC (oral) & GMCV (poster))

HLV-1K: A Large-scale Hour-Long Video Benchmark for Time-Specific Long Video Understanding

Heqing Zou, Tianze Luo, Guiyang Xie, Victor Xiao Jie Zhang, Fengmao Lv, Guangcong Wang, Junyang Chen, Zhuochen Wang, Hansheng Zhang, Huaijian Zhang

机构 * ByteDance（字节跳动） ； Nanyang Technological University（南洋理工大学） ； Southwest Jiaotong University（西南交通大学） ； Great Bay University（大湾大学） ； Shenzhen University（深圳大学） 纠错

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICME 2025

Overview of the NLPCC 2025 Shared Task 4: Multi-modal, Multilingual, and Multi-hop Medical Instructional Video Question Answering Challenge

Bin Li, Shenxi Liu, Yixuan Weng, Yue Du, Yuhang Tian, Shoujun Zhou

机构 * Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院） ； School of Computer Science and Technology, Beijing Institute of Technology（计算机科学与技术学院，北京理工大学） ； School of Engineering, Westlake University（工程学院，西湖大学） 纠错

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 5 figures, 4 tables

R^3-VQA: "Read the Room" by Video Social Reasoning

Lixing Niu, Jiapeng Li, Xingping Yu, Shu Wang, Ruining Feng, Bo Wu, Ping Wei, Yisen Wang, Lifeng Fan

机构 * School of Intelligence Science and Technology, Peking University（北京理工大学智能科学与技术学院） ； College of Artificial Intelligence, Xi’an Jiaotong University（西安交通大学人工智能学院） ； State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence（一般人工智能国家重点实验室） ； Yuanpei College, Peking University（北京大学元培学院） ； Tsinghua University（清华大学） ； University of California, Los Angeles（加州大学洛杉矶分校） ； MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI

MEG: Medical Knowledge-Augmented Large Language Models for Question Answering

Laura Cabello, Carmen Martin-Turrero, Uchenna Akujuobi, Anders Søgaard, Carlos Bobed

机构 * University of Copenhagen（哥本哈根大学） ； Sony AI（索尼人工智能） ； University of Zaragoza（阿拉维萨大学） 纠错

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI、cs.LG

Compositional 4D Dynamic Scenes Understanding with Physics Priors for Video Question Answering

Xingrui Wang, Wufei Ma, Angtian Wang, Shuo Chen, Adam Kortylewski, Alan Yuille

机构 * Johns Hopkins University（约翰霍普金斯大学） ； Tsinghua University（清华大学） ； Max Planck Institute for Informatics（马克斯·普朗克研究所（信息学）） ； University of Freiburg（弗赖堡大学） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments ICLR 2025 accepted paper. Project url: https://xingruiwang.github.io/projects/DynSuperCLEVR/

ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding

Yijia Xiao, Edward Sun, Yiqiao Jin, Qifan Wang, Wei Wang

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.AI、cs.LG

Comments Spotlight, Machine Learning for Genomics Explorations @ ICLR 2025

Online Video Understanding: OVBench and VideoChat-Online

Zhenpeng Huang, Xinhao Li, Jiaqi Li, Jing Wang, Xiangyu Zeng, Cheng Liang, Tao Wu, Xi Chen, Liang Li, Limin Wang

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments CVPR 2025 Camera Ready Version. Project Page: https://videochat-online.github.io

Bridging the Semantic Gaps: Improving Medical VQA Consistency with LLM-Augmented Question Sets

Yongpei Ma, Pengyu Wang, Adam Dunn, Usman Naseem, Jinman Kim

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments The first two listed authors contributed equally to this work

Towards A Generalizable Pathology Foundation Model via Unified Knowledge Distillation

Jiabo Ma, Zhengrui Guo, Fengtao Zhou, Yihui Wang, Yingxue Xu, Jinbang Li, Fang Yan, Yu Cai, Zhengjie Zhu, Cheng Jin, Yi Lin, Xinrui Jiang, Chenglong Zhao, Danyi Li, Anjia Han, Zhenhui Li, Ronald Cheong Kin Chan, Jiguang Wang, Peng Fei, Kwang-Ting Cheng, Shaoting Zhang, Li Liang, Hao Chen

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments update

Detecting Multimodal Situations with Insufficient Context and Abstaining from Baseless Predictions

Junzhang Liu, Zhecan Wang, Hammad Ayyubi, Haoxuan You, Chris Thomas, Rui Sun, Shih-Fu Chang, Kai-Wei Chang

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI

M-LLM Based Video Frame Selection for Efficient Video Understanding

Kai Hu, Feng Gao, Xiaohan Nie, Peng Zhou, Son Tran, Tal Neiman, Lingyun Wang, Mubarak Shah, Raffay Hamid, Bing Yin, Trishul Chilimbi

专题命中 视觉问答 ：visual reasoning(abstract)；分类 cs.CV、cs.AI

MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models

Wenbo Hu, Jia-Chen Gu, Zi-Yi Dou, Mohsen Fayyaz, Pan Lu, Kai-Wei Chang, Nanyun Peng

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments ICLR 2025

SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding

Ying Chen, Guoan Wang, Yuanfeng Ji, Yanjun Li, Jin Ye, Tianbin Li, Ming Hu, Rongshan Yu, Yu Qiao, Junjun He

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR2025

ECBench: Can Multi-modal Foundation Models Understand the Egocentric World? A Holistic Embodied Cognition Benchmark

Ronghao Dang, Yuqian Yuan, Wenqi Zhang, Yifei Xin, Boqiang Zhang, Long Li, Liuyi Wang, Qinyang Zeng, Xin Li, Lidong Bing

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Can Large Language Models Unveil the Mysteries? An Exploration of Their Ability to Unlock Information in Complex Scenarios

Chao Wang, Luning Zhang, Zheng Wang, Yang Zhou

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 11pages

Attention Mechanism based Cognition-level Scene Understanding

Xuejiao Tang, Wenbin Zhang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Published in Information

Enhancing Vietnamese VQA through Curriculum Learning on Raw and Augmented Text Representations

Khoi Anh Nguyen, Linh Yen Vu, Thang Dinh Duong, Thuan Nguyen Duong, Huy Thanh Nguyen, Vinh Quang Dinh

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments 10 pages, 3 figures, AAAI-25 Workshop on Document Understanding and Intelligence

BioD2C: A Dual-level Semantic Consistency Constraint Framework for Biomedical VQA

Zhengyang Ji, Shang Gao, Li Liu, Yifan Jia, Yutao Yue

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

CIC: A Framework for Culturally-Aware Image Captioning

Youngsik Yun, Jihie Kim

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted by IJCAI 2024

Abduction of Domain Relationships from Data for VQA

Al Mehdi Saadat Chowdhury, Paulo Shakarian, Gerardo I. Simari

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI、cs.LG

Comments In Proceedings ICLP 2024, arXiv:2502.08453

Journal ref EPTCS 416, 2025, pp. 168-174

Visual Graph Question Answering with ASP and LLMs for Language Parsing

Jakob Johannes Bauer, Thomas Eiter, Nelson Higuera Ruiz, Johannes Oetsch

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments In Proceedings ICLP 2024, arXiv:2502.08453. This work was partially funded from the Bosch Center for AI

Journal ref EPTCS 416, 2025, pp. 15-28