视觉大模型 / VLM

Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gap

Wenhan Yang, Spencer Stice, Ali Payani, Baharan Mirzasoleiman

机构 * Computer Science Department UCLA（计算机科学系，加州大学洛杉矶分校） ； Cisco Systems Inc.（思科系统公司） 纠错

专题命中 VLM训练与架构 ：VLM(title)；vision-language model(abstract)；LLaVA(abstract)；分类 cs.AI

ID-Align: RoPE-Conscious Position Remapping for Dynamic High-Resolution Adaptation in Vision-Language Models

Bozhou Li, Wentao Zhang

机构 * Peking University（北京大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

CorBenchX: Large-Scale Chest X-Ray Error Dataset and Vision-Language Model Benchmark for Report Error Correction

Jing Zou, Qingqiu Li, Chenyu Lian, Lihao Liu, Xiaohan Yan, Shujun Wang, Jing Qin

机构 * Center for Smart Health, The Hong Kong Polytechnic University（智能健康中心，香港理工大学） ； School of Computer Science, Fudan University（复旦大学计算机科学学院） ； Amazon（亚马逊） ； Tongji University（同济大学） ； Department of Biomedical Engineering, The Hong Kong Polytechnic University（生物医学工程系，香港理工大学） ； Research Institute for Smart Ageing, The Hong Kong Polytechnic University（智能老龄化研究 institute，香港理工大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；InternVL(abstract)；分类 cs.AI

Comments 12 pages, 5figures

TAMP: Token-Adaptive Layerwise Pruning in Multimodal Large Language Models

Jaewoo Lee, Keyang Xuan, Chanakya Ekbote, Sandeep Polisetty, Yi R. Fung, Paul Pu Liang

机构 * University of North Carolina Chapel Hill（北卡罗来纳大学教堂山分校） ； University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； Massachusetts Institute of Technology（麻省理工学院） ； University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校） ； Hong Kong University of Science and Technology（香港科学与技术大学） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments ACL Findings 2025

Evaluating Vision-Language Models as Evaluators in Path Planning

Mohamed Aghzal, Xiang Yue, Erion Plaku, Ziyu Yao

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted to the 2025 IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR)

MMRL++: Parameter-Efficient and Interaction-Aware Representation Learning for Vision-Language Models

Yuncheng Guo, Xiaodong Gu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Due to the limitation "The abstract field cannot be longer than 1,920 characters", the abstract appearing here is slightly shorter than that in the PDF file

Beyond General Prompts: Automated Prompt Refinement using Contrastive Class Alignment Scores for Disambiguating Objects in Vision-Language Models

Lucas Choi, Ross Greer

机构 * University of California, Merced（加州大学默塞德分校） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Beyond CLIP Generalization: Against Forward&Backward Forgetting Adapter for Continual Learning of Vision-Language Models

Songlin Dong, Chenhao Ding, Jiangyang Li, Jizhou Han, Qiang Wang, Yuhang He, Yihong Gong

机构 * College of Artificial Intelligence, Xi’an Jiaotong University（人工智能学院，西安交通大学） ； School of Software Engineering, Xi’an Jiaotong University（软件工程学院，西安交通大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

A Vision-Language Model for Focal Liver Lesion Classification

Song Jian, Hu Yuchang, Wang Hui, Chen Yen-Wei

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments 9 pages,4 figures, 4 tables,Innovation in Medicine and Healthcare Proceedings of 13th KES-InMed 2025

From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment

Yilin Wu, Ran Tian, Gokul Swamy, Andrea Bajcsy

机构 * Carnegie Mellon University（卡内基梅隆大学） ； UC Berkeley（伯克利大学） 纠错

专题命中 VLM训练与架构 ：VLM(title,abstract)；vision language model(abstract)；分类 cs.LG

Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design

Vasudev Sharma, Ahmed Alagha, Abdelhakim Khellaf, Vincent Quoc-Huy Trinh, Mahdi S. Hosseini

机构 * Department of Computer Science and Software Engineering (CSSE) Concordia University（计算机科学与软件工程系（CSSE）康科迪亚大学） ； University of Montreal Hospital Center（蒙特利尔大学医院中心） ； Mila–Quebec AI Institute（魁北克人工智能研究所） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Keyframe-oriented Vision Token Pruning: Enhancing Efficiency of Large Vision Language Models on Long-Form Video Processing

Yudong Liu, Jingwei Sun, Yueqian Lin, Jingyang Zhang, Ming Yin, Qinsi Wang, Jianyi Zhang, Hai Li, Yiran Chen

机构 * Duke University（杜克大学） ； Duke Kunshan University（杜克昆山大学） 纠错

专题命中 VLM训练与架构 ：vision language model(title,abstract)；VLM(abstract)；分类 cs.LG

Taxonomy-Aware Evaluation of Vision-Language Models

Vésteinn Snæbjarnarson, Kevin Du, Niklas Stoehr, Serge Belongie, Ryan Cotterell, Nico Lang, Stella Frank

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments CVPR 2025

Scenario Understanding of Traffic Scenes Through Large Visual Language Models

Esteban Rivera, Jannik Lübberstedt, Nico Uhlemann, Markus Lienkamp

专题命中 VLM训练与架构 ：visual language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments Accepted at WACV2025

ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinement

Runhui Huang, Chunwei Wang, Junwei Yang, Guansong Lu, Yunlong Yuan, Jianhua Han, Lu Hou, Wei Zhang, Lanqing Hong, Hengshuang Zhao, Hang Xu

专题命中 VLM训练与架构 ：MLLM(title,abstract)；LLaVA(abstract)；分类 cs.CV

MLLM-Selector: Necessity and Diversity-driven High-Value Data Selection for Enhanced Visual Instruction Tuning

Yiwei Ma, Guohai Xu, Xiaoshuai Sun, Jiayi Ji, Jie Lou, Debing Zhang, Rongrong Ji

专题命中 VLM训练与架构 ：MLLM(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments Tech Report

FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Model

Jun Zhou, Jiahao Li, Zunnan Xu, Hanhui Li, Yiji Cheng, Fa-Ting Hong, Qin Lin, Qinglin Lu, Xiaodan Liang

专题命中 VLM训练与架构 ：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted to CVPR 2025

Slide-Level Prompt Learning with Vision Language Models for Few-Shot Multiple Instance Learning in Histopathology

Devavrat Tomar, Guillaume Vray, Dwarikanath Mahapatra, Sudipta Roy, Jean-Philippe Thiran, Behzad Bozorgtabar

专题命中 VLM训练与架构 ：vision language model(title)；vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted to ISBI 2025

AlignBot: Aligning VLM-powered Customized Task Planning with User Reminders Through Fine-Tuning for Household Robots

Zhaxizhuoma Zhaxizhuoma, Pengan Chen, Ziniu Wu, Jiawei Sun, Dong Wang, Peng Zhou, Nieqing Cao, Yan Ding, Bin Zhao, Xuelong Li

专题命中 VLM训练与架构 ：VLM(title,abstract)；LLaVA(abstract)；分类 cs.AI

Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM

Xinyu Fang, Zhijian Chen, Kai Lan, Lixin Ma, Shengyuan Ding, Yingji Liang, Xiangyu Zhao, Farong Wen, Zicheng Zhang, Guofeng Zhang, Haodong Duan, Kai Chen, Dahua Lin

专题命中 VLM训练与架构 ：MLLM(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments Evaluation Code and dataset see https://github.com/open-compass/Creation-MMBench

SpatialBot: Precise Spatial Understanding with Vision Language Models

Wenxiao Cai, Iaroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang, Hao Dong, Bo Zhao

专题命中 VLM训练与架构 ：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

HybridGen: VLM-Guided Hybrid Planning for Scalable Data Generation of Imitation Learning

Wensheng Wang, Ning Tan

专题命中 VLM训练与架构 ：VLM(title,abstract)；vision-language model(abstract)；分类 cs.AI

Navigating Heat Exposure: Simulation of Route Planning Based on Visual Language Model Agents

Haoran Ma, Kaihan Zhang, Jiannan Cai

专题命中 VLM训练与架构 ：visual language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

Comments 10 pages, 6 figures

Similarity-Aware Token Pruning: Your VLM but Faster

Ahmadreza Jeddi, Negin Baghbanzadeh, Elham Dolatabadi, Babak Taati

专题命中 VLM训练与架构 ：VLM(title)；vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

Comments 15 pages, 8 figures, 8 tables

PBR3DGen: A VLM-guided Mesh Generation with High-quality PBR Texture

Xiaokang Wei, Bowen Zhang, Xianghui Yang, Yuxuan Wang, Chunchao Guo, Xi Zhao, Yan Luximon

专题命中 VLM训练与架构 ：VLM(title,abstract)；vision language model(abstract)；分类 cs.CV

Comments Homepage: https://pbr3dgen1218.github.io/

TokenCarve: Information-Preserving Visual Token Compression in Multimodal Large Language Models

Xudong Tan, Peng Ye, Chongjun Tu, Jianjian Cao, Yaoxin Yang, Lin Zhang, Dongzhan Zhou, Tao Chen

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models

Zhangyang Qi, Zhixiong Zhang, Ye Fang, Jiaqi Wang, Hengshuang Zhao

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Project page: https://gpt4scene.github.io/

XR-VLM: Cross-Relationship Modeling with Multi-part Prompts and Visual Features for Fine-Grained Recognition

Chuanming Wang, Henming Mao, Huanhuan Zhang, Huiyuan Fu, Huadong Ma

专题命中 VLM训练与架构 ：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV

VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge

Vishwesh Nath, Wenqi Li, Dong Yang, Andriy Myronenko, Mingxin Zheng, Yao Lu, Zhijian Liu, Hongxu Yin, Yucheng Tang, Pengfei Guo, Can Zhao, Ziyue Xu, Yufan He, Greg Heinrich, Yee Man Law, Benjamin Simon, Stephanie Harmon, Stephen Aylward, Marc Edgar, Michael Zephyr, Song Han, Pavlo Molchanov, Baris Turkbey, Holger Roth, Daguang Xu

专题命中 VLM训练与架构 ：vision-language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

Locality Alignment Improves Vision-Language Models

Ian Covert, Tony Sun, James Zou, Tatsunori Hashimoto

专题命中 VLM训练与架构 ：vision-language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

Comments ICLR 2025 Camera-Ready