视觉大模型 / VLM

Multimodal Large Language Models for Medicine: A Comprehensive Survey

Jiarui Ye, Hao Tang

机构 * School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院） ； School of Computer Science, Peking University（北京大学计算机科学学院） 纠错

专题命中 其他VLM ：multimodal large language model(title)；分类 cs.LG

Queryable Prototype Multiple Instance Learning with Vision-Language Models for Incremental Whole Slide Image Classification

Jiaxiang Gou, Luping Ji, Pei Liu, Mao Ye

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

Comments Accepted by AAAI 2025

BOLT: Boost Large Vision-Language Model Without Training for Long-form Video Understanding

Shuming Liu, Chen Zhao, Tianqi Xu, Bernard Ghanem

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

Comments Accepted to CVPR 2025

Multimodal Large Language Models for Image, Text, and Speech Data Augmentation: A Survey

Ranjan Sapkota, Shaina Raza, Maged Shoman, Achyut Paudel, Manoj Karkee

专题命中 其他VLM ：multimodal large language model(title)；分类 cs.CV

Comments 52 pages

Diversity Covariance-Aware Prompt Learning for Vision-Language Models

Songlin Dong, Zhengdong Zhou, Chenhao Ding, Xinyuan Gao, Alex Kot, Yihong Gong

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

Understanding and Evaluating Hallucinations in 3D Visual Language Models

Ruiying Peng, Kaiyuan Li, Weichen Zhang, Chen Gao, Xinlei Chen, Yong Li

专题命中 其他VLM ：visual language model(title)；分类 cs.CV

Vision-Language Models for Automated Chest X-ray Interpretation: Leveraging ViT and GPT-2

Md. Rakibul Islam, Md. Zahid Hossain, Mustofa Ahmed, Most. Sharmin Sultana Samu

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

Comments Preprint, manuscript under-review

FILP-3D: Enhancing 3D Few-shot Class-incremental Learning with Pre-trained Vision-Language Models

Wan Xu, Tianyu Huang, Tianyu Qu, Guanglei Yang, Yiwen Guo, Wangmeng Zuo

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

CosmoCLIP: Generalizing Large Vision-Language Models for Astronomical Imaging

Raza Imam, Mohammed Talha Alam, Umaima Rahman, Mohsen Guizani, Fakhri Karray

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

Comments Accepted at SPAICE Conference, ECSAT, UK, 2024

HiPrompt: Tuning-free Higher-Resolution Generation with Hierarchical MLLM Prompts

Xinyu Liu, Yingqing He, Lanqing Guo, Xiang Li, Bu Jin, Peng Li, Yan Li, Chi-Min Chan, Qifeng Chen, Wei Xue, Wenhan Luo, Qifeng Liu, Yike Guo

专题命中 其他VLM ：MLLM(title)；分类 cs.CV

Comments https://liuxinyv.github.io/HiPrompt/

What Makes Good Few-shot Examples for Vision-Language Models?

Zhaojun Guo, Jinghui Lu, Xuejing Liu, Rui Zhao, ZhenXing Qian, Fei Tan

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

Comments 8 pages, 4 figures

Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyond

Tianxin Wei, Bowen Jin, Ruirui Li, Hansi Zeng, Zhengyang Wang, Jianhui Sun, Qingyu Yin, Hanqing Lu, Suhang Wang, Jingrui He, Xianfeng Tang

专题命中 其他VLM ：vision-language model(title)；分类 cs.AI

Comments ICLR 2024

A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models

Julio Silva-Rodríguez, Sina Hajimiri, Ismail Ben Ayed, Jose Dolz

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

Comments CVPR 2024. Code: https://github.com/jusiro/CLAP

CLIPSwarm: Generating Drone Shows from Text Prompts with Vision-Language Models

Pablo Pueyo, Eduardo Montijano, Ana C. Murillo, Mac Schwager

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

ArGue: Attribute-Guided Prompt Tuning for Vision-Language Models

Xinyu Tian, Shu Zou, Zhaoyuan Yang, Jing Zhang

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

Comments Accepted to CVPR2024

Gemini vs GPT-4V: A Preliminary Comparison and Combination of Vision-Language Models Through Qualitative Cases

Zhangyang Qi, Ye Fang, Mengchen Zhang, Zeyi Sun, Tong Wu, Ziwei Liu, Dahua Lin, Jiaqi Wang, Hengshuang Zhao

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

Comments Project Page: https://github.com/Qi-Zhangyang/Gemini-vs-GPT4V. arXiv admin note: substantial text overlap with arXiv:2309.17421

Learning Hierarchical Prompt with Structured Linguistic Knowledge for Vision-Language Models

Yubin Wang, Xinyang Jiang, De Cheng, Dongsheng Li, Cairong Zhao

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

Comments AAAI2024

Multimodal Large Language Models: A Survey

Jiayang Wu, Wensheng Gan, Zefeng Chen, Shicheng Wan, Philip S. Yu

专题命中 其他VLM ：multimodal large language model(title)；分类 cs.AI

Comments IEEE BigData 2023. 10 pages

Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models

Juncheng Li, Minghe Gao, Longhui Wei, Siliang Tang, Wenqiao Zhang, Mengze Li, Wei Ji, Qi Tian, Tat-Seng Chua, Yueting Zhuang

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

Comments Accepted by ICCV 2023

A Review of Vision-Language Models and their Performance on the Hateful Memes Challenge

Bryan Zhao, Andrew Zhang, Blake Watson, Gillian Kearney, Isaac Dale

专题命中 其他VLM ：vision-language model(title)；分类 cs.AI

Towards the Human Global Context: Does the Vision-Language Model Really Judge Like a Human Being?

Sangmyeong Woh, Jaemin Lee, Ho Joong Kim, Jinsuk Lee

专题命中 其他VLM ：vision-language model(title)；分类 cs.CV

Illuminating Visual Identity in Universal Multimodal Embeddings

揭示通用多模态嵌入中的视觉身份

Jiawei Cao, Junyi Feng, Jiashen Hua, Ziheng Huang, Bing Deng, Kaijie Wu, Chaochen Gu, Jieping Ye

机构 * Shanghai Jiao Tong University（上海交通大学） ； Alibaba Group（阿里巴巴集团） 纠错

专题命中 其他VLM ：multimodal large language model(abstract,abstract_cn)；分类 cs.CV、cs.AI

AI总结 该研究针对通用多模态嵌入缺乏视觉身份判别能力的问题，提出视觉身份判别统一形式化，构建MVEB基准，设计身份感知采样的学习框架，提升了UMEs的身份判别能力。

Comments Accepted to CVPR 2026

Is Oracle Pruning the True Oracle?

Oracle剪枝真的是真正的Oracle吗？

Sicheng Feng, Keda Tao, Huan Wang

机构 * Westlake University（西湖大学） ； Nankai University（南开大学） ； ENCODE Lab, Westlake University（西湖大学ENCODE实验室） 纠错

专题命中 其他VLM ：MLLM(abstract,abstract_cn)；分类 cs.CV、cs.LG

AI总结 本文通过大规模实验（37K模型）发现，对于中等规模以上的深度学习模型，Oracle剪枝选择的权重在重训练后性能与重训练前几乎无关，质疑了Oracle剪枝作为剪枝方法基础的有效性。

Comments TMLR, Webpage: https://fscdc.github.io/Oracle-Pruning-Sanity-Check/

ReefNet: A Large-Scale Dataset and Benchmark for Fine-Grained Coral Reef Recognition

ReefNet：一个大规模数据集和基准，用于细粒度珊瑚礁识别

Abdulwahab Felemban, Yahia Battach, Faizan Farooq Khan, Yuqian Fu, Xuhui Liu, Yesmeen M. Khattab, Yousef A. Radwan, Xiang Li, Fabio Marchese, Sara Beery, Burton H. Jones, Francesca Benzoni, Mohamed Elhoseiny

机构 * King Abdullah University of Science and Technology (KAUST)（国王阿卜杜勒·阿齐兹科技大学） ； Massachusetts Institute of Technology (MIT)（麻省理工学院） 纠错

专题命中 其他VLM ：vision-language model(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

AI总结 本文提出ReefNet，一个大规模珊瑚礁图像数据集，用于细粒度识别。通过专家验证和过滤，构建了高置信度基准子集，揭示了多模态模型在生物多样性监测中的挑战。

GazeQwen: Lightweight Gaze-Conditioned LLM Modulation for Streaming Video Understanding

GazeQwen: 轻量级 gaze-条件 LLM 调制用于流视频理解

Trong Thang Pham, Hien Nguyen, Ngan Le

机构 * University of Arkansas（阿肯色大学） ； University of Houston（休斯顿大学） 纠错

专题命中 其他VLM ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

AI总结 GazeQwen 通过隐藏状态调制使开源 MLLM 获得 gaze 意识，采用紧凑的 gaze resampler 和可选 LoRA 调整，在 StreamGaze 基准上达到 63.9% 准确率，优于其他模型。

Tiny Inference-Time Scaling with Latent Verifiers

微小推理时间缩放与潜在验证器

Davide Bucciarelli, Evelyn Turri, Lorenzo Baraldi, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

机构 * University of Modena and Reggio Emilia（摩德纳和雷吉奥艾米利亚大学） ； University of Pisa（比萨大学） 纠错

专题命中 其他VLM ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

AI总结 本文提出VHS验证器，直接在扩散变换器单步生成器的中间隐藏表示上操作，减少验证成本并提升性能，实现更高效的推理时间缩放。

Comments Findings of CVPR 2026 - Code at: https://aimagelab.github.io/VHS/

MCIE: Multimodal LLM-Driven Complex Instruction Image Editing with Spatial Guidance

MCIE: 多模态大语言模型驱动的复杂指令图像编辑与空间引导

Xuehai Bai, Xiaoling Gu, Akide Liu, Hangjie Yuan, YiFan Zhang, Jack Ma

专题命中 其他VLM ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

AI总结 MCIE通过空间引导和背景一致性模块，提升复杂指令图像编辑的指令合规性，实现23.96%的性能提升。

Comments Accepted by AAAI2026

Advances and Innovations in the Multi-Agent Robotic System (MARS) Challenge

多智能体机器人系统（MARS）挑战的进展与创新

Li Kang, Heng Zhou, Xiufeng Song, Rui Li, Bruno N. Y. Chen, Ziye Wang, Ximeng Meng, Stone Tao, Yiran Qin, Xiaohong Liu, Ruimao Zhang, Lei Bai, Yilun Du, Hao Su, Philip Torr, Zhenfei Yin, Ruihao Gong, Yejun Zeng, Fengjun Zhong, Shenghao Jin, Jinyang Guo, Xianglong Liu, Xiaojun Jia, Tianqi Shan, Wenqi Ren, Simeng Qin, Jialing Yang, Xiaoyu Ma, Tianxing Chen, Zixuan Li, Zijian Cai, Yan Qin, Yusen Qin, Qiangyu Chen, Kaixuan Wang, Zhaoming Han, Yao Mu, Ping Luo, Yuanqi Yao, Haoming Song, Jan-Nico Zaech, Fabien Despinoy, Danda Pani Paudel, Luc Van Gool

机构 * SJTU（上海交通大学） ； Oxford（牛津大学） ； USTC（中国科学技术大学） ； Shanghai AI Lab（上海人工智能实验室） ； CMU（卡内基梅隆大学） ； HKU（香港大学） ； Tongji（同济大学） ； UC San Diego（南加州大学） ； CUHK-SZ（香港中文大学（深圳）） ； SYSU（华南理工大学） ； Harvard（哈佛大学） 纠错

专题命中 其他VLM ：vision-language model(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

AI总结 MARS挑战通过多智能体具身规划与控制任务，推动多智能体协作AI系统的发展。

Comments MARS Challenge @ NeurIPS 2025 Workshop on Space in Vision, Language, and Embodied AI. Challenge page: https://mars-eai.github.io/MARS-Challenge-Webpage/

NeuroABench: A Multimodal Evaluation Benchmark for Neurosurgical Anatomy Identification

NeuroABench: 一种多模态评估基准，用于神经外科解剖识别

Ziyang Song, Zelin Zang, Xiaofan Ye, Boqiang Xu, Long Bai, Jinlin Wu, Hongliang Ren, Hongbin Liu, Jiebo Luo, Zhen Lei

机构 * Hong Kong Institute of Science and Innovation（香港科学与创新研究院） ； The University of Hong Kong-Shenzhen Hospital（香港大学深圳医院） ； Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系） 纠错

专题命中 其他VLM ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

AI总结 NeuroABench是首个用于评估神经外科领域解剖理解的多模态基准，通过实验揭示了MLLMs在解剖识别任务中的局限性，并展示了人类表现与模型之间的差距。

Comments Accepted by IEEE ICIA 2025

Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMs

Dingkun Zhang, Shuhan Qi, Xinyu Xiao, Kehai Chen, Xuan Wang

机构 * Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）） ； Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies（广东省新型安全智能技术重点实验室） 纠错

专题命中 其他VLM ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

Comments EMNLP 2025 Main Conference