视觉大模型 / VLM

Vision Transformers Don't Need Trained Registers

Nick Jiang, Amil Dravid, Alexei Efros, Yossi Gandelsman

机构 * UC Berkeley（伯克利大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Project page and code: https://avdravid.github.io/test-time-registers. Accepted to NeurIPS '25 (spotlight)

I Spy With My Model's Eye: Visual Search as a Behavioural Test for MLLMs

John Burden, Jonathan Prunty, Ben Slater, Matthieu Tehenan, Greg Davis, Lucy Cheke

机构 * Leverhulme Centre for the Future of Intelligence, University of Cambridge（未来智能研究中心、剑桥大学） ； Department of Engineering, University of Cambridge（工程系、剑桥大学） ； Department of Psychology, University of Cambridge（心理学系、剑桥大学） ； Department of Computer Science, University of Cambridge（计算机科学系、剑桥大学） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Preprint

Training-Free Label Space Alignment for Universal Domain Adaptation

Dujin Lee, Sojung An, Jungmyung Wi, Kuniaki Saito, Donghyun Kim

机构 * Department of Artificial Intelligence, Korea University（人工智能系，韩国大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 22 pages, 12 figures

CovMatch: Cross-Covariance Guided Multimodal Dataset Distillation with Trainable Text Encoder

Yongmin Lee, Hye Won Chung

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments NeurIPS 2025

ImageGem: In-the-wild Generative Image Interaction Dataset for Generative Model Personalization

Yuanhe Guo, Linxi Xie, Zhuoran Chen, Kangrui Yu, Ryan Po, Guandao Yang, Gordon Wetztein, Hongyi Wen

机构 * NYU（纽约大学） ； Stanford（斯坦福大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

GradES: Significantly Faster Training in Transformers with Gradient-Based Early Stopping

Qifu Wen, Xi Zeng, Zihan Zhou, Shuaijun Liu, Mehdi Hosseinzadeh, Ningxin Su, Reza Rawassizadeh

机构 * Department of Computer Science, Boston University Metropolitan College（波士顿大学计算机科学系） ； Information Hub, The Hong Kong University of Science and Technology, Guangzhou（香港科技大学广州信息中心） ； School of Engineering and Technology, Duy Tan University, Da Nang, Vietnam（杜益大学工程科技学院，岘港，越南） ； Department of AI, School of Computer Science and Engineering, Galgotias University, Greater Noida, India（加洛吉亚大学人工智能系，诺伊达，印度） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments 20 pages, 5 figures

MIO: A Foundation Model on Multimodal Tokens

Zekun Wang, King Zhu, Chunpu Xu, Wangchunshu Zhou, Jiaheng Liu, Yibo Zhang, Jiashuo Wang, Ning Shi, Siyu Li, Yizhi Li, Haoran Que, Zhaoxiang Zhang, Yuanxing Zhang, Ge Zhang, Ke Xu, Jie Fu, Wenhao Huang

机构 * Beihang University（北航） ； M-A-P ； The Hong Kong Polytechnic University（香港理工大学） ； AIWaves ； University of Alberta（阿尔伯塔大学） ； University of Waterloo（滑铁卢大学） ； University of Manchester（曼彻斯特大学） ； Chinese Academy of Sciences（中国科学院） ； Peking University（北京大学） ； Shanghai AI Lab（上海AI实验室） ； Nanjing University（南京大学） ； Kuaishou Technology（快手科技） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.AI、cs.LG

Comments EMNLP 2025 (Oral). Codes and models are available in https://github.com/MIO-Team/MIO

Journal ref EMNLP 2025

Modular Embedding Recomposition for Incremental Learning

Aniello Panariello, Emanuele Frascaroli, Pietro Buzzega, Lorenzo Bonicelli, Angelo Porrello, Simone Calderara

机构 * AImageLab University of Modena and Reggio Emilia Italy（AImageLab 罗马那瓦尔大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to the 36th British Machine Vision Conference (BMVC 2025), Sheffield, UK

SeeingSounds: Learning Audio-to-Visual Alignment via Text

Simone Carnemolla, Matteo Pennisi, Chiara Russo, Simone Palazzo, Daniela Giordano, Concetto Spampinato

机构 * University of Catania（卡塔尼亚大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments accepted to ACM Multimedia Asia 2025

MC#: Mixture Compressor for Mixture-of-Experts Large Models

Wei Huang, Yue Liao, Yukang Chen, Jianhui Liu, Haoru Tan, Si Liu, Shiming Zhang, Shuicheng Yan, Xiaojuan Qi

机构 * Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电子与电气工程系） ； School of Computing, National University of Singapore（新加坡国立大学计算机学院） ； NVIDIA Research（NVIDIA研究） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments 15 pages, 13 figures

PatentVision: A multimodal method for drafting patent applications

Ruo Yang, Sai Krishna Reddy Mudhiganti, Manali Sharma

机构 * Samsung Semiconductor, Inc.（三星半导体公司） 纠错

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.AI、cs.LG

D-CoDe: Scaling Image-Pretrained VLMs to Video via Dynamic Compression and Question Decomposition

Yiyang Huang, Yizhou Wang, Yun Fu

机构 * Northeastern University（东北大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments This paper has been accepted to EMNLP 2025

V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuning

Hang Hua, Yolo Yunlong Tang, Chenliang Xu, Jiebo Luo

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to AAAI 2025

Stacked Regression using Off-the-shelf, Stimulus-tuned and Fine-tuned Neural Networks for Predicting fMRI Brain Responses to Movies (Algonauts 2025 Report)

Robert Scholz, Kunal Bagga, Christine Ahrends, Carlo Alberto Barbano

机构 * Université Paris Cité（巴黎Cité大学） ； University of Oxford（牛津大学） ； University of Turin（都灵大学） ； Universität Leipzig（莱比锡大学） ； Max Planck School of Cognition（马克斯·普朗克认知科学学院） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Multimodal Arabic Captioning with Interpretable Visual Concept Integration

Passant Elchafei, Amany Fashwan

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

microCLIP: Unsupervised CLIP Adaptation via Coarse-Fine Token Fusion for Fine-Grained Image Classification

Sathira Silva, Eman Ali, Chetan Arora, Muhammad Haris Khan

机构 * Mohamed Bin Zayed University of Artificial Intelligence（莫莫德·本·扎耶德人工智能大学） ； Alexandria University（亚历山大大学） ； IIT Delhi（德里理工学院） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Alternating Training-based Label Smoothing Enhances Prompt Generalization

Yang Chen, Yanbin Wei, Ke Jin, Yi Kong, James Kwok, Yu Zhang

机构 * Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系） ； Department of Computer Science and Engineering, Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系） ； School of Information and Control Engineering, China University of Mining and Technology（中国矿业大学信息与控制工程学院） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

WaveMind: Towards a Conversational EEG Foundation Model Aligned to Textual and Visual Modalities

Ziyi Zeng, Zhenyang Cai, Yixi Cai, Xidong Wang, Junying Chen, Rongsheng Wang, Yipeng Liu, Siqi Cai, Benyou Wang, Zhiguo Zhang, Haizhou Li

机构 * The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）） ； Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.AI、cs.LG

Imagining Alternatives: Towards High-Resolution 3D Counterfactual Medical Image Generation via Language Guidance

Mohamed Mohamed, Brennan Nichyporuk, Douglas L. Arnold, Tal Arbel

机构 * McGill University（麦吉尔大学） ； Mila – Quebec AI Institute（魁北克人工智能研究所） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted to the 2025 MICCAI ELAMI Workshop

An Experimental Study on Generating Plausible Textual Explanations for Video Summarization

Thomas Eleftheriadis, Evlampios Apostolidis, Vasileios Mezaris

机构 * IEEE CBMI 2025 纠错

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Comments IEEE CBMI 2025. This is the authors' accepted version. The final publication is available at https://ieeexplore.ieee.org/

Importance Sampling for Multi-Negative Multimodal Direct Preference Optimization

Xintong Li, Chuhan Wang, Junda Wu, Rohan Surana, Tong Yu, Julian McAuley, Jingbo Shang

机构 * University of California, San Diego（加州大学圣迭戈分校） ； Adobe Research（Adobe研究） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Preprint

Patient-specific Biomolecular Instruction Tuning

Irsyad Adam, Zekai Chen, David Laub, Shaun Porwal, Arda Pekis, Kevin Brown

机构 * Standard Model Biomedicine（标准模型生物医学） ； University of California, San Diego（加州大学圣地亚哥分校） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.AI、cs.LG

Random Direct Preference Optimization for Radiography Report Generation

Valentin Samokhin, Boris Shirokikh, Mikhail Goncharov, Dmitriy Umerenkov, Maksim Bobrin, Ivan Oseledets, Dmitry Dylov, Mikhail Belyaev

机构 * Artificial Intelligence Research Institute (AIRI)（人工智能研究院） ； Skolkovo Institute of Science and Technology（斯克罗夫学院） ； Institute for Information Transmission Problems（信息传输问题研究所） 纠错

专题命中 VLM训练与架构 ：visual language model(abstract)；分类 cs.CV、cs.AI

Seeing Through Words, Speaking Through Pixels: Deep Representational Alignment Between Vision and Language Models

Zoe Wanying He, Sean Trott, Meenakshi Khosla

机构 * Department of Cognitive Science University of California, San Diego（认知科学系，加州大学圣地亚哥分校） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted at EMNLP 2025 (camera-ready)

AdaSVD: Adaptive Singular Value Decomposition for Large Language Models

Zhiteng Li, Mingyuan Xia, Jingyuan Zhang, Zheng Hui, Haotong Qin, Linghe Kong, Yulun Zhang, Xiaokang Yang

机构 * Shanghai Jiao Tong University（上海交通大学） ； MGTV, Shanhai Academy（MGTV，汕海学院） ； ETH Zürich（苏黎世联邦理工学院） 纠错

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI

Comments The code and models will be available at https://github.com/ZHITENGLI/AdaSVD

Exploration with Foundation Models: Capabilities, Limitations, and Hybrid Approaches

Remo Sasso, Michelangelo Conserva, Dominik Jeurissen, Paulo Rauber

机构 * School of Electronic Engineering and Computer Science（电子工程与计算机科学学院） 纠错

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.AI、cs.LG

Comments 16 pages, 7 figures. Accepted for presentation at the 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Workshop on the Foundations of Reasoning in Language Models (FoRLM)

Leveraging Large Models to Evaluate Novel Content: A Case Study on Advertisement Creativity

Zhaoyi Joey Hou, Adriana Kovashka, Xiang Lorraine Li

机构 * Department of Computer Science University of Pittsburgh（计算机科学系宾夕法尼亚大学） 纠错

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI

Comments To Appear in EMNLP2025

Align Where the Words Look: Cross-Attention-Guided Patch Alignment with Contrastive and Transport Regularization for Bengali Captioning

Riad Ahmed Anonto, Sardar Md. Saffat Zabin, M. Saifur Rahman

机构 * Bangladesh University of Engineering and Technology (BUET)（孟加拉工程与技术大学） 纠错

专题命中 VLM训练与架构 ：grounding(abstract)；分类 cs.CV、cs.AI

Multimodal Health Risk Prediction System for Chronic Diseases via Vision-Language Fusion and Large Language Models

Dingxin Lu, Shurui Wu, Xinyi Huang

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.AI、cs.LG

MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe

Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang, Wenshuo Ma, Zhihui He, Tianchi Cai, Weize Chen, Yuxiang Huang, Yuanqian Zhao, Bokai Xu, Junbo Cui, Yingjing Xu, Liqing Ruan, Luoyuan Zhang, Hanyu Liu, Jingkun Tang, Hongyuan Liu, Qining Guo, Wenhao Hu, Bingxiang He, Jie Zhou, Jie Cai, Ji Qi, Zonghao Guo, Chi Chen, Guoyang Zeng, Yuxuan Li, Ganqu Cui, Ning Ding, Xu Han, Yuan Yao, Zhiyuan Liu, Maosong Sun

机构 * MiniCPM-V Team, OpenBMB（MiniCPM-V团队，OpenBMB） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments Project Website: https://github.com/OpenBMB/MiniCPM-V