视觉大模型 / VLM

LLAVADI: What Matters For Multimodal Large Language Models Distillation

Shilin Xu, Xiangtai Li, Haobo Yuan, Lu Qi, Yunhai Tong, Ming-Hsuan Yang

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.CV

Dallah: A Dialect-Aware Multimodal Large Language Model for Arabic

Fakhraddin Alwajih, Gagan Bhatia, Muhammad Abdul-Mageed

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.AI

Test-Time Low Rank Adaptation via Confidence Maximization for Zero-Shot Generalization of Vision-Language Models

Raza Imam, Hanan Gani, Muhammad Huzaifa, Karthik Nandakumar

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Main paper: 11 pages, Supplementary material: 5 pages

EVLM: An Efficient Vision-Language Model for Visual Understanding

Kaibing Chen, Dong Shen, Hanwen Zhong, Huasong Zhong, Kui Xia, Di Xu, Wei Yuan, Yifei Hu, Bin Wen, Tianke Zhang, Changyi Liu, Dewen Fan, Huihui Xiao, Jiahong Wu, Fan Yang, Size Li, Di Zhang

专题命中 VLM训练与架构 ：vision-language model(title)；LLaVA(abstract)；分类 cs.CV

E5-V: Universal Embeddings with Multimodal Large Language Models

Ting Jiang, Minghui Song, Zihan Zhang, Haizhen Huang, Weiwei Deng, Feng Sun, Qi Zhang, Deqing Wang, Fuzhen Zhuang

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.CV

Comments Code and models are available at https://github.com/kongds/E5-V

The First to Know: How Token Distributions Reveal Hidden Knowledge in Large Vision-Language Models?

Qinyu Zhao, Ming Xu, Kartik Gupta, Akshay Asthana, Liang Zheng, Stephen Gould

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments ECCV 2024. Project page: https://github.com/Qinyu-Allen-Zhao/LVLM-LP

Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models

Jinrui Zhang, Teng Wang, Haigang Zhang, Ping Lu, Feng Zheng

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments To appear at ECCV2024

Conceptual Codebook Learning for Vision-Language Models

Yi Zhang, Ke Yu, Siqi Wu, Zhihai He

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2024

MuDPT: Multi-modal Deep-symphysis Prompt Tuning for Large Pre-trained Vision-Language Models

Yongzhu Miao, Shasha Li, Jintao Tang, Ting Wang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments The paper has been accepted by ICME 2023

Dude: Dual Distribution-Aware Context Prompt Learning For Large Vision-Language Model

Duy M. H. Nguyen, An T. Le, Trung Q. Nguyen, Nghiem T. Diep, Tai Nguyen, Duy Duong-Tran, Jan Peters, Li Shen, Mathias Niepert, Daniel Sonntag

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Version 1

Improving Visual Storytelling with Multimodal Large Language Models

Xiaochuan Lin, Xiangyong Chen

专题命中 VLM训练与架构 ：multimodal large language model(title)；vision-language model(abstract)；分类 cs.CV

Comments 10 pages

InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models

Xunguang Wang, Zhenlan Ji, Pingchuan Ma, Zongjie Li, Shuai Wang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

African or European Swallow? Benchmarking Large Vision-Language Models for Fine-Grained Object Classification

Gregor Geigle, Radu Timofte, Goran Glavaš

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Can Better Text Semantics in Prompt Tuning Improve VLM Generalization?

Hari Chandana Kuchibhotla, Sai Srinivas Kancheti, Abbavaram Gowtham Reddy, Vineeth N Balasubramanian

专题命中 VLM训练与架构 ：VLM(title)；vision-language model(abstract)；分类 cs.CV

BaFTA: Backprop-Free Test-Time Adaptation For Zero-Shot Vision-Language Models

Xuefeng Hu, Ke Zhang, Min Sun, Albert Chen, Cheng-Hao Kuo, Ram Nevatia

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Preprint updated from our earlier manuscript submitted to ICLR 2024 (https://openreview.net/forum?id=KNtcoAM5Gy)

ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Models

Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhihong Chen, Jianquan Li, Xiang Wan, Benyou Wang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.AI

Comments 22 pages

Towards Robust Instruction Tuning on Multimodal Large Language Models

Wei Han, Hui Chen, Soujanya Poria

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.AI

Comments 24 pages, 7 figures

Regularized Training with Generated Datasets for Name-Only Transfer of Vision-Language Models

Minho Park, Sunghyun Park, Jooyeol Yun, Jaegul Choo

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Preprint. Under review

Boosting Vision-Language Models with Transduction

Maxime Zanella, Benoît Gérin, Ismail Ben Ayed

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters

Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Ping Hu, Dong Wang, Huchuan Lu, You He

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments This work is accepted by CVPR2024. More modifications may be performed

Low-Rank Few-Shot Adaptation of Vision-Language Models

Maxime Zanella, Ismail Ben Ayed

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Models

Linli Yao, Lei Li, Shuhuai Ren, Lean Wang, Yuanxin Liu, Xu Sun, Lu Hou

专题命中 VLM训练与架构 ：multimodal large language model(title)；MLLM(abstract)；分类 cs.CV

Knowledge-grounded Adaptation Strategy for Vision-language Models: Building Unique Case-set for Screening Mammograms for Residents Training

Aisha Urooj Khan, John Garrett, Tyler Bradshaw, Lonie Salkowski, Jiwoong Jason Jeong, Amara Tariq, Imon Banerjee

专题命中 VLM训练与架构 ：vision-language model(title)；VLM(abstract)；分类 cs.CV

Disease-informed Adaptation of Vision-Language Models

Jiajin Zhang, Ge Wang, Mannudeep K. Kalra, Pingkun Yan

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Early Accepted by MICCAI 2024

Adapting Dual-encoder Vision-language Models for Paraphrased Retrieval

Jiacheng Cheng, Hijung Valentina Shin, Nuno Vasconcelos, Bryan Russell, Fabian Caba Heilbron

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation

Xun Wu, Shaohan Huang, Furu Wei

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.CV

Iterated Learning Improves Compositionality in Large Vision-Language Models

Chenhao Zheng, Jieyu Zhang, Aniruddha Kembhavi, Ranjay Krishna

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments CVPR 2024

Distilling Vision-Language Models on Millions of Videos

Yue Zhao, Long Zhao, Xingyi Zhou, Jialin Wu, Chun-Te Chu, Hui Miao, Florian Schroff, Hartwig Adam, Ting Liu, Boqing Gong, Philipp Krähenbühl, Liangzhe Yuan

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments CVPR 2024. Project page: https://zhaoyue-zephyrus.github.io/video-instruction-tuning

Semantics-aware Motion Retargeting with Vision-Language Models

Haodong Zhang, ZhiKe Chen, Haocheng Xu, Lei Hao, Xiaofei Wu, Songcen Xu, Zhensong Zhang, Yue Wang, Rong Xiong

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted in CVPR2024

Anchor-based Robust Finetuning of Vision-Language Models

Jinwei Han, Zhiwen Lin, Zhongyisun Sun, Yingguo Gao, Ke Yan, Shouhong Ding, Yuan Gao, Gui-Song Xia

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments CVPR2024