视觉大模型 / VLM - arXivDaily 专题

2502.20108 2025-03-04 cs.CV cs.RO 83%

VDT-Auto: End-to-end Autonomous Driving with VLM-Guided Diffusion Transformers

Ziang Guo, Konstantin Gubernatorov, Selamawit Asfaw, Zakhar Yagudin, Dzmitry Tsetserukou

专题命中 VLM训练与架构：VLM(title,abstract)；visual language model(abstract)；分类 cs.CV

Comments Submitted paper

2501.09695 2025-03-04 cs.CV 83%

Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key

Zhihe Yang, Xufang Luo, Dongqi Han, Yunjian Xu, Dongsheng Li

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments Accepted by CVPR 2025

2409.18459 2025-03-04 cs.CV cs.MM 83%

FoodMLLM-JP: Leveraging Multimodal Large Language Models for Japanese Recipe Generation

Yuki Imajuku, Yoko Yamakata, Kiyoharu Aizawa

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments 15 pages, 5 figures. We found errors in the calculation of evaluation metrics, which were corrected in this version with $\color{blue}{\text{modifications highlighted in blue}}$. Please also see the Appendix

2410.05340 2025-03-03 cs.LG 83%

Generating CAD Code with Vision-Language Models for 3D Designs

Kamel Alrashedy, Pradyumna Tambwekar, Zulfiqar Zaidi, Megan Langwasser, Wei Xu, Matthew Gombolay

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.LG

2410.17247 2025-02-28 cs.CV cs.CL 83%

PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction

Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, Dahua Lin

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments CVPR 2025, code is available at https://github.com/Cooperx521/PyramidDrop

2412.13333 2025-02-26 cs.LG 83%

Beyond Accuracy: On the Effects of Fine-tuning Towards Vision-Language Model's Prediction Rationality

Qitong Wang, Tang Li, Kien X. Nguyen, Xi Peng

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.LG

Comments In Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI), 2025

2502.10250 2025-02-25 cs.CL cs.CV 83%

VisCon-100K: Leveraging Contextual Web Data for Fine-tuning Vision Language Models

Gokul Karthik Kumar, Iheb Chaabane, Kebin Wu

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted at PAKDD 2025

2501.18623 2025-02-19 cs.CV cs.GR 83%

VLMaterial: Procedural Material Generation with Large Vision-Language Models

Beichen Li, Rundi Wu, Armando Solar-Lezama, Changxi Zheng, Liang Shi, Bernd Bickel, Wojciech Matusik

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments ICLR 2025 Spotlight

2410.04780 2025-02-19 cs.CV 83%

Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality

Guanyu Zhou, Yibo Yan, Xin Zou, Kun Wang, Aiwei Liu, Xuming Hu

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted by The Thirteenth International Conference on Learning Representations (ICLR 2025)

2502.11427 2025-02-18 cs.CL cs.CV 83%

Do we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models

Zikang Liu, Kun Zhou, Wayne Xin Zhao, Dawei Gao, Yaliang Li, Ji-Rong Wen

专题命中 VLM训练与架构：vision-language model(title,abstract)；visual reasoning(abstract)；分类 cs.CV

Comments under review

2407.08507 2025-02-18 cs.CV 83%

Bootstrapping Vision-language Models for Self-supervised Remote Physiological Measurement

Zijie Yue, Miaojing Shi, Hanli Wang, Shuai Ding, Qijun Chen, Shanlin Yang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments International Journal of Computer Vision

2501.16065 2025-02-12 cs.CV 83%

CILP-FGDI: Exploiting Vision-Language Model for Generalizable Person Re-Identification

Huazhong Zhao, Lei Qi, Xin Geng

专题命中 VLM训练与架构：vision-language model(title,abstract)；visual language model(abstract)；分类 cs.CV

Comments Accepted by IEEE TIFS

2502.06094 2025-02-11 cs.CV 83%

Fair-MoE: Fairness-Oriented Mixture of Experts in Vision-Language Models

Peiran Wang, Linjie Tong, Jiaxiang Liu, Zuozhu Liu

专题命中 VLM训练与架构：vision-language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

2502.06019 2025-02-11 cs.CV 83%

Noise is an Efficient Learner for Zero-Shot Vision-Language Models

Raza Imam, Asif Hanif, Jian Zhang, Khaled Waleed Dawoud, Yova Kementchedjhieva, Mohammad Yaqub

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Our code is available at https://github.com/Razaimam45/TNT

2412.16158 2025-02-11 cs.CV 83%

HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding

Chenxin Tao, Shiqian Su, Xizhou Zhu, Chenyu Zhang, Zhe Chen, Jiawen Liu, Wenhai Wang, Lewei Lu, Gao Huang, Yu Qiao, Jifeng Dai

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

2502.02458 2025-02-05 cs.CL cs.CV 83%

SAISA: Towards Multimodal Large Language Models with Both Training and Inference Efficiency

Qianhao Yuan, Yanjiang Liu, Yaojie Lu, Hongyu Lin, Ben He, Xianpei Han, Le Sun

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

2501.15040 2025-01-28 cs.CV 83%

Complementary Subspace Low-Rank Adaptation of Vision-Language Models for Few-Shot Classification

Zhongqi Wang, Jia Dai, Kai Li, Xu Li, Yanmeng Guo, Maosheng Xiang

专题命中 VLM训练与架构：vision-language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Preprint version

2501.09039 2025-01-17 cs.CR cs.AI cs.CY 83%

Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models

Abdulkadir Erol, Trilok Padhi, Agnik Saha, Ugur Kursuncu, Mehmet Emin Aktas

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.AI

2408.09698 2025-01-14 cs.IR cs.AI 83%

Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation

Yuyang Ye, Zhi Zheng, Yishan Shen, Tianshu Wang, Hengruo Zhang, Peijun Zhu, Runlong Yu, Kai Zhang, Hui Xiong

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.AI

2403.09792 2025-01-14 cs.CV cs.CL 83%

Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models

Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Ji-Rong Wen

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments ECCV 2024 Oral

2402.16315 2025-01-08 cs.CV cs.CL 83%

Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models

Jeonghwan Kim, Heng Ji

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

Comments EMNLP 2024; Main Conference

2410.17283 2025-01-03 cs.AI 83%

Advancements in Visual Language Models for Remote Sensing: Datasets, Capabilities, and Enhancement Techniques

Lijie Tao, Haokui Zhang, Haizhao Jing, Yu Liu, Dawei Yan, Guoting Wei, Xizhe Xue

专题命中 VLM训练与架构：visual language model(title,abstract)；VLM(abstract)；分类 cs.AI

2412.15614 2024-12-23 cs.CR cs.CV 83%

Technical Report for ICML 2024 TiFA Workshop MLLM Attack Challenge: Suffix Injection and Projected Gradient Descent Can Easily Fool An MLLM

Yangyang Guo, Ziwei Xu, Xilie Xu, YongKang Wong, Liqiang Nie, Mohan Kankanhalli

专题命中 VLM训练与架构：MLLM(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments ICML TiFA Challenge Technical Report

2412.11917 2024-12-20 cs.CV 83%

Does VLM Classification Benefit from LLM Description Semantics?

Pingchuan Ma, Lennart Rietdorf, Dmytro Kotovenko, Vincent Tao Hu, Björn Ommer

专题命中 VLM训练与架构：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV

Comments AAAI-25 (extended version), Code: https://github.com/CompVis/DisCLIP

2406.18868 2024-12-19 cs.CV 83%

Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models

Yicheng Xu, Yuxin Chen, Jiahao Nie, Yusong Wang, Huiping Zhuang, Manabu Okumura

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024

2412.09616 2024-12-16 cs.CV 83%

V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding

Junqi Ge, Ziyi Chen, Jintao Lin, Jinguo Zhu, Xihui Liu, Jifeng Dai, Xizhou Zhu

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments The code and models will be available at https://github.com/OpenGVLab/V2PE

2412.09530 2024-12-13 cs.CV 83%

Dynamic-VLM: Simple Dynamic Visual Token Compression for VideoLLM

Han Wang, Yuxiang Nie, Yongjie Ye, Deng GuanYu, Yanjie Wang, Shuai Li, Haiyang Yu, Jinghui Lu, Can Huang

专题命中 VLM训练与架构：VLM(title)；vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

2404.10595 2024-12-09 cs.CV 83%

Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases

Kai Chen, Yanze Li, Wenhua Zhang, Yanxin Liu, Pengxiang Li, Ruiyuan Gao, Lanqing Hong, Meng Tian, Xinhai Zhao, Zhenguo Li, Dit-Yan Yeung, Huchuan Lu, Xu Jia

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accept by WACV 2025. Project Page: https://coda-dataset.github.io/coda-lm/

2412.03324 2024-12-06 cs.CV 83%

A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs

Wangbo Zhao, Yizeng Han, Jiasheng Tang, Zhikai Li, Yibing Song, Kai Wang, Zhangyang Wang, Yang You

专题命中 VLM训练与架构：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV

2411.11066 2024-11-19 cs.CV 83%

TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models

Tingyu Qu, Mingxiao Li, Tinne Tuytelaars, Marie-Francine Moens

专题命中 VLM训练与架构：LLaVA(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments work in progress

1. VLM训练与架构 5067 篇

VDT-Auto: End-to-end Autonomous Driving with VLM-Guided Diffusion Transformers

Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key

FoodMLLM-JP: Leveraging Multimodal Large Language Models for Japanese Recipe Generation

Generating CAD Code with Vision-Language Models for 3D Designs

PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction

Beyond Accuracy: On the Effects of Fine-tuning Towards Vision-Language Model's Prediction Rationality

VisCon-100K: Leveraging Contextual Web Data for Fine-tuning Vision Language Models

VLMaterial: Procedural Material Generation with Large Vision-Language Models

Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality

Do we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models

Bootstrapping Vision-language Models for Self-supervised Remote Physiological Measurement

CILP-FGDI: Exploiting Vision-Language Model for Generalizable Person Re-Identification

Fair-MoE: Fairness-Oriented Mixture of Experts in Vision-Language Models

Noise is an Efficient Learner for Zero-Shot Vision-Language Models

HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding

SAISA: Towards Multimodal Large Language Models with Both Training and Inference Efficiency

Complementary Subspace Low-Rank Adaptation of Vision-Language Models for Few-Shot Classification

Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models

Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation

Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models

Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models

Advancements in Visual Language Models for Remote Sensing: Datasets, Capabilities, and Enhancement Techniques

Technical Report for ICML 2024 TiFA Workshop MLLM Attack Challenge: Suffix Injection and Projected Gradient Descent Can Easily Fool An MLLM

Does VLM Classification Benefit from LLM Description Semantics?

Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models

V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding

Dynamic-VLM: Simple Dynamic Visual Token Compression for VideoLLM

Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases

A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs

TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models