视觉大模型 / VLM

Backdoor Cleaning without External Guidance in MLLM Fine-tuning

Xuankun Rong, Wenke Huang, Jian Liang, Jinhe Bi, Xun Xiao, Yiming Li, Bo Du, Mang Ye

专题命中 VLM训练与架构 ：MLLM(title)；multimodal large language model(abstract)；分类 cs.CV

How Well Can General Vision-Language Models Learn Medicine By Watching Public Educational Videos?

Rahul Thapa, Andrew Li, Qingyang Wu, Bryan He, Yuki Sahashi, Christina Binder, Angela Zhang, Ben Athiwaratkun, Shuaiwen Leon Song, David Ouyang, James Zou

机构 * Stanford University（斯坦福大学） ； Together AI ； University of California, Berkeley（加州大学伯克利分校） ； Cedars-Sinai Medical Center（西德斯-西奈医学中心） ； University of California, San Francisco（加州大学旧金山分校） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agents

Junpeng Yue, Xinrun Xu, Börje F. Karlsson, Zongqing Lu

机构 * School of Computer Science, Peking University（北京大学计算机科学学院） ； Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所） ； Beijing Academy of Artificial Intelligence（北京人工智能研究院） 纠错

专题命中 VLM训练与架构 ：MLLM(title,abstract)；分类 cs.LG

Comments ICLR 2025

MiniDrive: More Efficient Vision-Language Models with Multi-Level 2D Features as Text Tokens for Autonomous Driving

Enming Zhang, Xingyuan Dai, Min Huang, Yisheng Lv, Qinghai Miao

机构 * School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

ReSW-VL: Representation Learning for Surgical Workflow Analysis Using Vision-Language Model

Satoshi Kondo

机构 * Muroran Institute of Technology（茂兰技术学院） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

From Local Details to Global Context: Advancing Vision-Language Models with Attention-Based Selection

Lincan Cai, Jingxuan Kang, Shuang Li, Wenxuan Ma, Binhui Xie, Zhida Qin, Jian Liang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Uniformity First: Uniformity-aware Test-time Adaptation of Vision-language Models against Image Corruption

Kazuki Adachi, Shin'ya Yamaguchi, Tomoki Hamagami

机构 * NTT, Inc.（NTT公司） ； Kyoto University（京都大学） ； Yokohama National University（横滨国立大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Code is available at https://github.com/kzkadc/uninfo

Temporal-Oriented Recipe for Transferring Large Vision-Language Model to Video Understanding

Thong Nguyen, Zhiyuan Hu, Xu Lin, Cong-Duy Nguyen, See-Kiong Ng, Luu Anh Tuan

机构 * National University of Singapore（新加坡国立大学） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments In Progress

LLaVA-4D: Embedding SpatioTemporal Prompt into LMMs for 4D Scene Understanding

Hanyu Zhou, Gim Hee Lee

机构 * School of Computing, National University of Singapore（计算学院，新加坡国立大学） 纠错

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Espresso: High Compression For Rich Extraction From Videos for Your Vision-Language Model

Keunwoo Peter Yu, Achal Dave, Rares Ambrus, Jean Mercat

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments 16 pages

Unsupervised Multiview Contrastive Language-Image Joint Learning with Pseudo-Labeled Prompts Via Vision-Language Model for 3D/4D Facial Expression Recognition

Muzammil Behzad

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Understanding and Mitigating Toxicity in Image-Text Pretraining Datasets: A Case Study on LLaVA

Karthik Reddy Kanjula, Surya Guthikonda, Nahid Alam, Shayekh Bin Islam

机构 * Cohere for AI Community（Cohere for AI社区） ； Cisco Meraki ； Indiana University Bloomington（印第安纳大学布卢明顿分校） ； Bangladesh University of Engineering and Technology（孟加拉工程与技术大学） 纠错

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Comments Accepted at ReGenAI CVPR2025 Workshop as Oral

XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models

Omkar Thawakar, Abdelrahman Shaker, Sahal Shaji Mullappilly, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Jorma Laaksonen, Fahad Shahbaz Khan

机构 * Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学） ； Aalto University（艾尔沃斯大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted at ACL 2024-BIONLP Workshop. Code: https://github.com/mbzuai-oryx/XrayGPT

Enhancing the Learning Experience: Using Vision-Language Models to Generate Questions for Educational Videos

Markos Stamatakis, Joshua Berger, Christian Wartena, Ralph Ewerth, Anett Hoppe

机构 * TIB – Leibniz Information Centre for Science and Technology（蒂宾根-莱比锡信息科学与技术研究中心） ； Hochschule Hannover – Data（汉诺威高等学院-数据） ； H Institute for Applied Data Science（汉诺威应用数据科学研究所） ； L3S Research Center – Leibniz University Hannover（L3S研究中心-汉诺威莱比锡大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments 12 pages (excluding references), 8 tables, 1 equation

3D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformer

Jiajun Deng, Tianyu He, Li Jiang, Tianyu Wang, Feras Dayoub, Ian Reid

机构 * Australian Institute for Machine Learning, The University of Adelaide（澳大利亚机器学习研究所，阿德莱德大学） ； Microsoft Research（微软研究院） ； The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）） ； Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学） 纠错

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2025

Enhancing Diagnostic Accuracy in Rare and Common Fundus Diseases with a Knowledge-Rich Vision-Language Model

Meng Wang, Tian Lin, Aidi Lin, Kai Yu, Yuanyuan Peng, Lianyu Wang, Cheng Chen, Ke Zou, Huiyu Liang, Man Chen, Xue Yao, Meiqin Zhang, Binwei Huang, Chaoxin Zheng, Peixin Zhang, Wei Chen, Yilong Luo, Yifan Chen, Honghe Xia, Tingkun Shi, Qi Zhang, Jinming Guo, Xiaolin Chen, Jingcheng Wang, Yih Chung Tham, Dianbo Liu, Wendy Wong, Sahil Thakur, Beau Fenner, Danqi Fang, Siying Liu, Qingyun Liu, Yuqiang Huang, Hongqiang Zeng, Yanda Meng, Yukun Zhou, Zehua Jiang, Minghui Qiu, Changqing Zhang, Xinjian Chen, Sophia Y. Wang, Cecilia S. Lee, Lucia Sobrin, Carol Y Cheung, Chi Pui Pang, Pearse A. Keane, Ching-Yu Cheng, Haoyu Chen, Huazhu Fu

机构 * Centre for Innovation & Precision Eye Health（创新与精准眼科健康中心） ； Yong Loo Lin School of Medicine（尤洛林医学院） ； National University of Singapore（新加坡国立大学） ； Department of Ophthalmology（眼科部门） ； Joint Shantou International Eye Center（汕尾国际眼科中心） ； Shantou University（汕尾大学） ； Chinese University of Hong Kong（香港中文大学） ； Department of Radiology（放射科部门） ； University of Pennsylvania（宾夕法尼亚大学） ； School of Biomedical Engineering（生物医学工程学院） ； College of Computer Science and Technology（计算机科学与技术学院） ； Center of Advanced Medical Computing and Analysis（先进医学计算与分析中心） ； National Key Laboratory of Fundamental Science on Synthetic Vision and the College of Computer Science（合成视觉基础科学国家实验室和计算机科学学院） ； Big Vision Medical Technology Ltd.（大视觉医疗科技有限公司） ； Singapore Eye Research Institute（新加坡眼科研究院） ； Singapore National Eye Centre（新加坡国家眼科中心） ； Ophthalmology & Visual Sciences Academic Clinical Program (EYE ACP)（眼科与视觉科学学术临床计划） ； The Chinese University of Hong Kong（香港中文大学） ； Shenzhen Longgang E.N.T Hospital（深圳龙岗耳鼻喉医院） ； Dongguan Songshan Lake Central Hospital（东莞松山湖中心医院） ； Department of Computer Science（计算机科学部门） ； University of Exeter（埃克塞特大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Compositional Kronecker Context Optimization for Vision-Language Models

Kun Ding, Xiaohui Li, Qiang Yu, Ying Wang, Haojian Zhang, Shiming Xiang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

PACT: Pruning and Clustering-Based Token Reduction for Faster Visual Language Models

Mohamed Dhouib, Davide Buscaldi, Sonia Vanier, Aymen Shabou

专题命中 VLM训练与架构 ：visual language model(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2025

TokenFLEX: Unified VLM Training for Flexible Visual Tokens Inference

Junshan Hu, Jialiang Mao, Zhikang Liu, Zhongpu Xia, Peng Jia, Xianpeng Lang

专题命中 VLM训练与架构 ：VLM(title)；vision-language model(abstract)；分类 cs.CV

InPK: Infusing Prior Knowledge into Prompt for Vision-Language Models

Shuchang Zhou, Jiwei Wei, Shiyuan He, Yuyang Zhou, Chaoning Zhang, Jie Zou, Ning Xie, Yang Yang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understanding

Mingze Xu, Mingfei Gao, Shiyu Li, Jiasen Lu, Zhe Gan, Zhengfeng Lai, Meng Cao, Kai Kang, Yinfei Yang, Afshin Dehghan

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Comments Technical report

CLIP-Adapter: Better Vision-Language Models with Feature Adapters

Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, Yu Qiao

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by IJCV

MAO: Efficient Model-Agnostic Optimization of Prompt Tuning for Vision-Language Models

Haoyang Li, Siyu Zhou, Liang Wang, Guodong Long

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by the IEEE International Conference on Multimedia & Expo 2025 (ICME 2025); 12 pages, 6 figures, 8 tables

PE-CLIP: A Parameter-Efficient Fine-Tuning of Vision Language Models for Dynamic Facial Expression Recognition

Ibtissam Saadi, Abdenour Hadid, Douglas W. Cunningham, Abdelmalik Taleb-Ahmed, Yassin El Hillali

专题命中 VLM训练与架构 ：vision language model(title)；vision-language model(abstract)；分类 cs.CV

LoRASculpt: Sculpting LoRA for Harmonizing General and Specialized Knowledge in Multimodal Large Language Models

Jian Liang, Wenke Huang, Guancheng Wan, Qu Yang, Mang Ye

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2025

Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Model

Longrong Yang, Dong Shen, Chaoxiang Cai, Fan Yang, Tingting Gao, Di Zhang, Xi Li

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

LLaVA-MLB: Mitigating and Leveraging Attention Bias for Training-Free Video LLMs

Leqi Shen, Tao He, Guoqiang Gong, Fan Yang, Yifeng Zhang, Pengzhang Liu, Sicheng Zhao, Guiguang Ding

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Skip Tuning: Pre-trained Vision-Language Models are Effective and Efficient Adapters Themselves

Shihan Wu, Ji Zhang, Pengpeng Zeng, Lianli Gao, Jingkuan Song, Heng Tao Shen

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

BiomedCoOp: Learning to Prompt for Biomedical Vision-Language Models

Taha Koleilat, Hojat Asgariandehkordi, Hassan Rivaz, Yiming Xiao

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2025

Multi-Cue Adaptive Visual Token Pruning for Large Vision-Language Models

Bozhi Luan, Wengang Zhou, Hao Feng, Zhe Wang, Xiaosong Li, Houqiang Li

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV