视觉大模型 / VLM

Test-time Alignment-Enhanced Adapter for Vision-Language Models

Baoshun Tong, Kaiyu Song, Hanjiang Lai

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Semantic Shield: Defending Vision-Language Models Against Backdooring and Poisoning via Fine-grained Knowledge Alignment

Alvi Md Ishmam, Christopher Thomas

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments CVPR 2024

Purrfessor: A Fine-tuned Multimodal LLaVA Diet Health Chatbot

Linqi Lu, Yifan Deng, Chuan Tian, Sijia Yang, Dhavan Shah

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.AI

Comments 10 pages, 5 figures

GraphVL: Graph-Enhanced Semantic Modeling via Vision-Language Models for Generalized Class Discovery

Bhupendra Solanki, Ashwin Nair, Mainak Singha, Souradeep Mukhopadhyay, Ankit Jha, Biplab Banerjee

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted in ACM ICVGIP 2024

Enhance Image-to-Image Generation with LLaVA-generated Prompts

Zhicheng Ding, Panfeng Li, Qikai Yang, Siyang Li

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Comments Accepted by 2024 5th International Conference on Information Science, Parallel and Distributed Systems

Journal ref Proceedings of the 2024 5th International Conference on Information Science, Parallel and Distributed Systems (ISPDS), 2024, pp. 77-81

HumanVLM: Foundation for Human-Scene Vision-Language Model

Dawei Dai, Xu Long, Li Yutang, Zhang Yuanhui, Shuyin Xia

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.AI

Comments 34 pages,11 figures

Precise Drive with VLM: First Prize Solution for PRCV 2024 Drive LM challenge

Bin Huang, Siyu Wang, Yuanpeng Chen, Yidan Wu, Hui Song, Zifan Ding, Jing Leng, Chengpeng Liang, Peng Xue, Junliang Zhang, Tiankun Zhao

专题命中 VLM训练与架构 ：VLM(title)；InternVL(abstract)；分类 cs.CV

Visual Anchors Are Strong Information Aggregators For Multimodal Large Language Model

Haogeng Liu, Quanzeng You, Xiaotian Han, Yongfei Liu, Huaibo Huang, Ran He, Hongxia Yang

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.CV

Unveiling Encoder-Free Vision-Language Models

Haiwen Diao, Yufeng Cui, Xiaotong Li, Yueze Wang, Huchuan Lu, Xinlong Wang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments 17 pages, 8 figures, Accepted by NeurIPS2024 (spotlight)

Beyond Filtering: Adaptive Image-Text Quality Enhancement for MLLM Pretraining

Han Huang, Yuqi Huo, Zijia Zhao, Haoyu Lu, Shu Wu, Bingning Wang, Qiang Liu, Weipeng Chen, Liang Wang

专题命中 VLM训练与架构 ：MLLM(title)；multimodal large language model(abstract)；分类 cs.CV

VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignment

Lei Li, Zhihui Xie, Mukai Li, Shunian Chen, Peiyi Wang, Liang Chen, Yazheng Yang, Benyou Wang, Lingpeng Kong, Qi Liu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments EMNLP 2024 Main Conference camera-ready version (fixed small typos). This article supersedes arXiv:2312.10665

Granular Privacy Control for Geolocation with Vision Language Models

Ethan Mendes, Yang Chen, James Hays, Sauvik Das, Wei Xu, Alan Ritter

专题命中 VLM训练与架构 ：vision language model(title,abstract)；分类 cs.CV

Comments Accepted to EMNLP 2024 main conference

Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate

Qidong Huang, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Jiaqi Wang, Dahua Lin, Weiming Zhang, Nenghai Yu

专题命中 VLM训练与架构 ：vision-language model(title)；vision language model(abstract)；分类 cs.CV

Comments Project page: https://github.com/shikiw/Modality-Integration-Rate

HeGraphAdapter: Tuning Multi-Modal Vision-Language Models with Heterogeneous Graph Adapter

Yumiao Zhao, Bo Jiang, Xiao Wang, Qin Xu, Jin Tang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Power-LLaVA: Large Language and Vision Assistant for Power Transmission Line Inspection

Jiahao Wang, Mingxuan Li, Haichen Luo, Jinguo Zhu, Aijun Yang, Mingzhe Rong, Xiaohua Wang

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

OCC-MLLM-Alpha:Empowering Multi-modal Large Language Model for the Understanding of Occluded Objects with Self-Supervised Test-Time Learning

Shuxin Yang, Xinhan Di

专题命中 VLM训练与架构 ：MLLM(title)；VLM(abstract)；分类 cs.CV

Comments Accepted by ECCV 2024 Observing and Understanding Hands in Action Workshop (5 pages, 3 figures, 2 tables). arXiv admin note: substantial text overlap with arXiv:2410.01261

Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Model

Hao Cheng, Erjia Xiao, Jindong Gu, Le Yang, Jinhao Duan, Jize Zhang, Jiahang Cao, Kaidi Xu, Renjing Xu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments This paper is accepted by ECCV 2024

Mixture of Prompt Learning for Vision Language Models

Yu Du, Tong Niu, Rong Zhao

专题命中 VLM训练与架构 ：vision language model(title)；vision-language model(abstract)；分类 cs.CV

SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Models

Mingze Xu, Mingfei Gao, Zhe Gan, Hong-You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang, Afshin Dehghan

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Comments Technical report

ComAlign: Compositional Alignment in Vision-Language Models

Ali Abdollah, Amirmohammad Izadi, Armin Saghafian, Reza Vahidimajd, Mohammad Mozafari, Amirreza Mirzaei, Mohammadmahdi Samiei, Mahdieh Soleymani Baghshah

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Revisiting Prompt Pretraining of Vision-Language Models

Zhenyuan Chen, Lingfeng Yang, Shuo Chen, Zhaowei Chen, Jiajun Liang, Xiang Li

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Few-shot Adaptation of Medical Vision-Language Models

Fereshteh Shakeri, Yunshi Huang, Julio Silva-Rodríguez, Houda Bahig, An Tang, Jose Dolz, Ismail Ben Ayed

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments MICCAI 2024 (Spotlight) - Code is available at https://github.com/FereshteShakeri/few-shot-MedVLMs.git

LLaVA-Chef: A Multi-modal Generative Model for Food Recipes

Fnu Mohbat, Mohammed J. Zaki

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.LG

Adapting Vision-Language Models to Open Classes via Test-Time Prompt Tuning

Zhengqing Gao, Xiang Ao, Xu-Yao Zhang, Cheng-Lin Liu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments PRCV 2024

u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model

Jinjin Xu, Liwu Xu, Yuzhe Yang, Xiang Li, Fanyi Wang, Yanchun Xie, Yi-Jie Huang, Yaqian Li

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Making Large Vision Language Models to be Good Few-shot Learners

Fan Liu, Wenwen Cai, Jian Huo, Chuanyi Zhang, Delong Chen, Jun Zhou

专题命中 VLM训练与架构 ：vision language model(title,abstract)；分类 cs.CV

The Phantom Menace: Unmasking Privacy Leakages in Vision-Language Models

Simone Caldarella, Massimiliano Mancini, Elisa Ricci, Rahaf Aljundi

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

PromptKD: Unsupervised Prompt Distillation for Vision-Language Models

Zheng Li, Xiang Li, Xinyi Fu, Xin Zhang, Weiqiang Wang, Shuo Chen, Jian Yang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments CVPR 2024. Project Page: https://zhengli97.github.io/PromptKD. Code: https://github.com/zhengli97/PromptKD

Consistency-guided Prompt Learning for Vision-Language Models

Shuvendu Roy, Ali Etemad

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Published as a conference paper at ICLR 2024

FlexAttention for Efficient High-Resolution Vision-Language Models

Junyan Li, Delin Chen, Tianle Cai, Peihao Chen, Yining Hong, Zhenfang Chen, Yikang Shen, Chuang Gan

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2024