视觉大模型 / VLM - arXivDaily 专题

2509.02805 2025-09-04 cs.LG 85%

Challenges in Understanding Modality Conflict in Vision-Language Models

Trang Nguyen, Jackson Michaels, Madalina Fiterau, David Jensen

机构 * Manning College of Information \& Computer Sciences, University of Massachusetts Amherst, Amherst, U.S.

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.LG

2503.06794 2025-09-03 cs.CV cs.CL 85%

Does Acceleration Cause Hidden Instability in Vision Language Models? Uncovering Instance-Level Divergence Through a Large-Scale Empirical Study

Yizheng Sun, Hao Li, Chang Xu, Hongpeng Zhou, Chenghua Lin, Riza Batista-Navarro, Jingyuan Sun

机构 * University of Manchester（曼彻斯特大学）； Microsoft Research（微软研究院）

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；VLM(abstract)；LLaVA(abstract)

Comments Accepted to EMNLP 2025 Main Conference

2410.06154 2025-08-22 cs.CV 85%

GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Models

M. Jehanzeb Mirza, Mengjie Zhao, Zhuoyuan Mao, Sivan Doveh, Wei Lin, Paul Gavrikov, Michael Dorkenwald, Shiqi Yang, Saurav Jha, Hiromi Wakaki, Yuki Mitsufuji, Horst Possegger, Rogerio Feris, Leonid Karlinsky, James Glass

机构 * MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）； Sony（索尼）； Weizmann Institute of Science（魏茨曼科学研究院）； JKU（约翰纳斯堡大学）； Tübingen AI Center（图宾根人工智能中心）； UVA（乌得勒支大学）； UNSW（新南威尔士大学）； TU Graz（格拉茨技术大学）； MIT-IBM（麻省理工学院-IBM）

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；VLM(abstract)；LLaVA(abstract)

Comments Code: https://github.com/jmiemirza/GLOV

2503.19386 2025-07-31 cs.CV eess.SP 85%

Exploring Textual Semantics Diversity for Image Transmission in Semantic Communication Systems using Visual Language Model

Peishan Huang, Dong Li

机构 * School of Computer Science and Engineering, Macau University of Science and Technology（计算机科学与工程学院，澳门科技大学）

专题命中 VLM训练与架构：visual language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV

2507.14024 2025-07-21 cs.CV 85%

Moodifier: MLLM-Enhanced Emotion-Driven Image Editing

Jiarong Ye, Sharon X. Huang

机构 * The Pennsylvania State University（宾夕法尼亚州立大学）

专题命中 VLM训练与架构：MLLM(title)；vision-language model(abstract)；LLaVA(abstract)；multimodal large language model(abstract)

2507.04735 2025-07-08 cs.CV 85%

An analysis of vision-language models for fabric retrieval

Francesco Giuliari, Asif Khan Pattan, Mohamed Lamine Mekhalfi, Fabio Poiesi

机构 * Fondazione Bruno Kessler（布雷诺-科塞勒基金会）

专题命中 VLM训练与架构：vision-language model(title,abstract)；vision language model(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments Accepted at Ital-IA 2025

2412.19326 2025-07-01 cs.CV 85%

Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment

Ziang Yan, Zhilin Li, Yinan He, Chenting Wang, Kunchang Li, Xinhao Li, Xiangyu Zeng, Zilei Wang, Yali Wang, Yu Qiao, Limin Wang, Yi Wang

机构 * Shanghai AI Laboratory（上海人工智能实验室）； Zhejiang University（浙江大学）； University of Science and Technology of China（中国科学技术大学）； Shanghai Jiao Tong University（上海交通大学）； Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）； Nanjing University（南京大学）； Shanghai Innovation Institute（上海创新研究院）

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

Comments CVPR2025

2506.22636 2025-07-01 cs.CV 85%

ReCo: Reminder Composition Mitigates Hallucinations in Vision-Language Models

Sotirios Panagiotis Chytas, Miso Choi, Hyunwoo J. Kim, Vikas Singh

机构 * University of Wisconsin-Madison（威斯康星大学麦迪逊分校）； Korea University（韩国大学）； KAIST（韩国科学技术院）

专题命中 VLM训练与架构：vision-language model(title)；vision language model(abstract)；VLM(abstract)；LLaVA(abstract)

2411.04291 2025-06-23 cs.CL cs.CV 85%

Layer-wise Alignment: Examining Safety Alignment Across Image Encoder Layers in Vision Language Models

Saketh Bachu, Erfan Shayegani, Rohit Lal, Trishna Chakraborty, Arindam Dutta, Chengyu Song, Yue Dong, Nael Abu-Ghazaleh, Amit K. Roy-Chowdhury

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；VLM(abstract)；LLaVA(abstract)

Comments Accepted by ICML 2025 as a spotlight poster

2506.12776 2025-06-17 cs.CV 85%

Native Visual Understanding: Resolving Resolution Dilemmas in Vision-Language Models

Junbo Niu, Yuanhong Zheng, Ziyang Miao, Hejun Dong, Chunjiang Ge, Hao Liang, Ma Lu, Bohan Zeng, Qiahao Zheng, Conghui He, Wentao Zhang

机构 * Peking University（北京大学）； Shanghai AI Laboratory（上海人工智能实验室）； Shandong University（山东大学）； Beihang University（北航）； Tsinghua University（清华大学）

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV

2410.04417 2025-06-04 cs.CV 85%

SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference

Yuan Zhang, Chun-Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, Shanghang Zhang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV

Comments Accepted by ICML 2025

2506.00548 2025-06-03 cs.CR cs.CL cs.LG 85%

Con Instruction: Universal Jailbreaking of Multimodal Large Language Models via Non-Textual Modalities

Jiahui Geng, Thy Thy Tran, Preslav Nakov, Iryna Gurevych

专题命中 VLM训练与架构：multimodal large language model(title)；LLaVA(abstract)；InternVL(abstract)；MLLM(abstract)

2505.17436 2025-05-26 cs.AI 85%

Scaling Up Biomedical Vision-Language Models: Fine-Tuning, Instruction Tuning, and Multi-Modal Learning

Cheng Peng, Kai Zhang, Mengxian Lyu, Hongfang Liu, Lichao Sun, Yonghui Wu

专题命中 VLM训练与架构：vision-language model(title,abstract)；vision language model(abstract)；visual question answering(abstract)；分类 cs.AI

2505.08910 2025-05-16 cs.CV cs.CL 85%

Behind Maya: Building a Multilingual Vision Language Model

Nahid Alam, Karthik Reddy Kanjula, Surya Guthikonda, Timothy Chung, Bala Krishna S Vegesna, Abhipsha Das, Anthony Susevski, Ryan Sze-Yin Chan, S M Iftekhar Uddin, Shayekh Bin Islam, Roshan Santhosh, Snegha A, Drishti Sharma, Chen Liu, Isha Chaturvedi, Genta Indra Winata, Ashvanth. S, Snehanshu Mukherjee, Alham Fikri Aji

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；VLM(abstract)；LLaVA(abstract)

Comments Accepted at VLMs4ALL CVPR 2025 Workshop; corrected workshop name spelling

2505.03374 2025-05-07 cs.CV 85%

Reducing Annotation Burden in Physical Activity Research Using Vision-Language Models

Abram Schonfeldt, Benjamin Maylor, Xiaofang Chen, Ronald Clark, Aiden Doherty

机构 * Department of Population Health, University of Oxford（牛津大学流行病学与健康统计学系）； University of Oxford（牛津大学）； School of Epidemiology and Health Statistics, Chengdu Medical College（成都医学院流行病学与健康统计学系）

专题命中 VLM训练与架构：vision-language model(title,abstract)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

2501.18624 2025-02-10 cs.CR cs.AI 85%

Membership Inference Attacks Against Vision-Language Models

Yuke Hu, Zheng Li, Zhihao Liu, Yang Zhang, Zhan Qin, Kui Ren, Chun Chen

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.AI

Comments Accepted by USENIX'25; 22 pages, 28 figures;

2501.04322 2025-01-24 cs.CV 85%

Eve: Efficient Multimodal Vision Language Models with Elastic Visual Experts

Miao Rang, Zhenni Bi, Chuanjian Liu, Yehui Tang, Kai Han, Yunhe Wang

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV

2501.12327 2025-01-22 cs.CV 85%

VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model

Xianwei Zhuang, Yuxin Xie, Yufan Deng, Liming Liang, Jinghan Ru, Yuguo Yin, Yuexian Zou

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

2412.12359 2025-01-08 cs.CV cs.CL 85%

LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering

Jinhe Bi, Yujun Wang, Haokun Chen, Xun Xiao, Artur Hecker, Volker Tresp, Yunpu Ma

专题命中 VLM训练与架构：LLaVA(title,abstract)；grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

2412.20105 2024-12-31 cs.CV 85%

ST$^3$: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming

Jiedong Zhuang, Lu Lu, Ming Dai, Rui Hu, Jian Chen, Qiang Liu, Haoji Hu

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted to AAAI2025

2311.12668 2024-12-10 cs.AI cs.CE 85%

From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design

Cyril Picard, Kristen M. Edwards, Anna C. Doris, Brandon Man, Giorgio Giannone, Md Ferdous Alam, Faez Ahmed

专题命中 VLM训练与架构：vision-language model(title,abstract)；vision language model(abstract)；LLaVA(abstract)；分类 cs.AI

2412.00556 2024-12-10 cs.CV 85%

Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reduction

Shiyu Zhao, Zhenting Wang, Felix Juefei-Xu, Xide Xia, Miao Liu, Xiaofang Wang, Mingfu Liang, Ning Zhang, Dimitris N. Metaxas, Licheng Yu

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

Comments Technical report, 18 pages

2412.00832 2024-12-03 cs.CV 85%

EventGPT: Event Stream Understanding with Multimodal Large Language Models

Shaoyu Liu, Jianing Li, Guanghui Zhao, Yunjian Zhang, Xin Meng, Fei Richard Yu, Xiangyang Ji, Ming Li

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

2411.14505 2024-11-25 cs.CV 85%

LLaVA-MR: Large Language-and-Vision Assistant for Video Moment Retrieval

Weiheng Lu, Jian Li, An Yu, Ming-Ching Chang, Shengpeng Ji, Min Xia

专题命中 VLM训练与架构：LLaVA(title,abstract)；grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

2410.13859 2024-10-18 cs.CV 85%

$γ-$MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models

Yaxin Luo, Gen Luo, Jiayi Ji, Yiyi Zhou, Xiaoshuai Sun, Zhiqiang Shen, Rongrong Ji

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

2409.02278 2024-09-05 cs.CV 85%

Evaluation and Comparison of Visual Language Models for Transportation Engineering Problems

Sanjita Prajapati, Tanu Singh, Chinmay Hegde, Pranamesh Chakraborty

专题命中 VLM训练与架构：visual language model(title)；vision language model(abstract)；VLM(abstract)；LLaVA(abstract)

2406.09913 2024-08-27 cs.CV 85%

OpenECAD: An Efficient Visual Language Model for Editable 3D-CAD Design

Zhe Yuan, Jianqi Shi, Yanhong Huang

专题命中 VLM训练与架构：visual language model(title,abstract)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

Journal ref Computers & Graphics 124C (2024) 104048

2408.06721 2024-08-14 cs.CV 85%

Response Wide Shut: Surprising Observations in Basic Vision Language Model Capabilities

Shivam Chandhok, Wan-Cyuan Fan, Leonid Sigal

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；VLM(abstract)；LLaVA(abstract)

Comments Under Submission

2408.03940 2024-08-08 cs.CV 85%

How Well Can Vision Language Models See Image Details?

Chenhui Gou, Abdulwahab Felemban, Faizan Farooq Khan, Deyao Zhu, Jianfei Cai, Hamid Rezatofighi, Mohamed Elhoseiny

专题命中 VLM训练与架构：vision language model(title,abstract)；vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2406.10165 2024-06-17 cs.CV cs.RO 85%

CarLLaVA: Vision language models for camera-only closed-loop driving

Katrin Renz, Long Chen, Ana-Maria Marcu, Jan Hünermann, Benoit Hanotte, Alice Karnsund, Jamie Shotton, Elahe Arani, Oleg Sinavski

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV

Comments Outstanding Champion & Innovation Award @ CARLA Autonomous Driving Challenge 2024; Project video: https://youtu.be/E1nsEgcHRuc

1. VLM训练与架构 5058 篇

Challenges in Understanding Modality Conflict in Vision-Language Models

Does Acceleration Cause Hidden Instability in Vision Language Models? Uncovering Instance-Level Divergence Through a Large-Scale Empirical Study

GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Models

Exploring Textual Semantics Diversity for Image Transmission in Semantic Communication Systems using Visual Language Model

Moodifier: MLLM-Enhanced Emotion-Driven Image Editing

An analysis of vision-language models for fabric retrieval

Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment

ReCo: Reminder Composition Mitigates Hallucinations in Vision-Language Models

Layer-wise Alignment: Examining Safety Alignment Across Image Encoder Layers in Vision Language Models

Native Visual Understanding: Resolving Resolution Dilemmas in Vision-Language Models

SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference

Con Instruction: Universal Jailbreaking of Multimodal Large Language Models via Non-Textual Modalities

Scaling Up Biomedical Vision-Language Models: Fine-Tuning, Instruction Tuning, and Multi-Modal Learning

Behind Maya: Building a Multilingual Vision Language Model

Reducing Annotation Burden in Physical Activity Research Using Vision-Language Models

Membership Inference Attacks Against Vision-Language Models

Eve: Efficient Multimodal Vision Language Models with Elastic Visual Experts

VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model

LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering

ST$^3$: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming

From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design

Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reduction

EventGPT: Event Stream Understanding with Multimodal Large Language Models

LLaVA-MR: Large Language-and-Vision Assistant for Video Moment Retrieval

$γ-$MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models

Evaluation and Comparison of Visual Language Models for Transportation Engineering Problems

OpenECAD: An Efficient Visual Language Model for Editable 3D-CAD Design

Response Wide Shut: Surprising Observations in Basic Vision Language Model Capabilities

How Well Can Vision Language Models See Image Details?

CarLLaVA: Vision language models for camera-only closed-loop driving