视觉大模型 / VLM - arXivDaily 专题

2502.02740 2025-02-06 cs.LG cs.AI 84%

Vision-Language Model Dialog Games for Self-Improvement

Ksenia Konyushkova, Christos Kaplanis, Serkan Cabi, Misha Denil

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.AI、cs.LG

2501.07888 2025-01-27 cs.CV cs.AI 84%

Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understanding

Liping Yuan, Jiawei Wang, Haomiao Sun, Yuchen Zhang, Yuan Lin

专题命中 VLM训练与架构：vision-language model(title,abstract)；grounding(abstract)；分类 cs.CV、cs.AI

2501.12418 2025-01-23 cs.CV cs.AI 84%

ImageRef-VL: Enabling Contextual Image Referencing in Vision-Language Models

Jingwei Yi, Junhao Yin, Ju Xu, Peng Bao, Yongliang Wang, Wei Fan, Hao Wang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

2312.04746 2025-01-14 cs.CV cs.AI cs.CL 84%

Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videos

Mehmet Saygin Seyfioglu, Wisdom O. Ikezogwo, Fatemeh Ghezloo, Ranjay Krishna, Linda Shapiro

专题命中 VLM训练与架构：LLaVA(title,abstract)；grounding(abstract)；分类 cs.CV、cs.AI

2501.03968 2025-01-13 cs.RO cs.AI cs.CV cs.HC 84%

VLM-driven Behavior Tree for Context-aware Task Planning

Naoki Wake, Atsushi Kanehira, Jun Takamatsu, Kazuhiro Sasabuchi, Katsushi Ikeuchi

专题命中 VLM训练与架构：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 11 figures, 5 tables. Last updated on January 9th, 2024

2501.02461 2025-01-07 cs.CV cs.AI 84%

FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Models

Hui Lin, Chao Zhang, Danfeng Hong, Kexin Dong, Congcong Wen

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

2501.01282 2025-01-03 cs.AI cs.CL cs.CV 84%

CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries

Shudong Liu, Yiqiao Jin, Cheng Li, Derek F. Wong, Qingsong Wen, Lichao Sun, Haipeng Chen, Xing Xie, Jindong Wang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Technical report; 26 pages

2407.07638 2024-12-31 cs.CV cs.AI 84%

Tuning Vision-Language Models with Candidate Labels by Prompt Alignment

Zhifang Zhang, Yuwei Niu, Xin Liu, Beibei Li

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

2408.10945 2024-12-30 cs.CV cs.AI 84%

HiRED: Attention-Guided Token Dropping for Efficient Inference of High-Resolution Vision-Language Models

Kazi Hasan Ibn Arif, JinYi Yoon, Dimitrios S. Nikolopoulos, Hans Vandierendonck, Deepu John, Bo Ji

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments Accepted in AAAI 2025

2406.13362 2024-12-20 cs.CV cs.CL cs.LG 84%

VisualRWKV: Exploring Recurrent Neural Networks for Visual Language Models

Haowen Hou, Peigen Zeng, Fei Ma, Fei Richard Yu

专题命中 VLM训练与架构：visual language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.LG

Comments Accepted at COLING 2025 main conference

2410.08584 2024-12-19 cs.CV cs.AI 84%

ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsification

Yefei He, Feng Chen, Jing Liu, Wenqi Shao, Hong Zhou, Kaipeng Zhang, Bohan Zhuang

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments 13 pages

2412.12735 2024-12-18 cs.CV cs.AI cs.CL 84%

GIRAFFE: Design Choices for Extending the Context Length of Visual Language Models

Mukai Li, Lei Li, Shansan Gong, Qi Liu

专题命中 VLM训练与架构：visual language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Working in progress

2407.06438 2024-12-17 cs.CV cs.CL cs.LG 84%

SOLO: A Single Transformer for Scalable Vision-Language Modeling

Yangyi Chen, Xingyao Wang, Hao Peng, Heng Ji

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.LG

Comments Accepted to TMLR

2412.08111 2024-12-12 cs.CV cs.CL cs.LG 84%

Seeing Syntax: Uncovering Syntactic Learning Limitations in Vision-Language Models

Sri Harsha Dumpala, David Arps, Sageev Oore, Laura Kallmeyer, Hassan Sajjad

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.LG

2403.16442 2024-12-06 cs.CL cs.CV cs.LG 84%

If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions

Reza Esfandiarpoor, Cristina Menghini, Stephen H. Bach

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Comments EMNLP 2024

2406.09400 2024-12-05 cs.CV cs.LG 84%

Yo'LLaVA: Your Personalized Language and Vision Assistant

Thao Nguyen, Haotian Liu, Yuheng Li, Mu Cai, Utkarsh Ojha, Yong Jae Lee

专题命中 VLM训练与架构：LLaVA(title,abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Comments NeurIPS 2024; Project page: https://thaoshibe.github.io/YoLLaVA

2411.15203 2024-11-26 cs.LG cs.AI cs.CL 84%

Multimodal large language model for wheat breeding: a new exploration of smart breeding

Guofeng Yang, Yu Li, Yong He, Zhenjiang Zhou, Lingzhen Ye, Hui Fang, Yiqi Luo, Xuping Feng

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；InternVL(abstract)；分类 cs.AI、cs.LG

2411.10888 2024-11-19 eess.IV cs.AI cs.CV 84%

MpoxVLM: A Vision-Language Model for Diagnosing Skin Lesions from Mpox Virus Infection

Xu Cao, Wenqian Ye, Kenny Moise, Megan Coffee

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ML4H 2024

2411.04097 2024-11-07 cs.CV cs.AI 84%

RaVL: Discovering and Mitigating Spurious Correlations in Fine-Tuned Vision-Language Models

Maya Varma, Jean-Benoit Delbrouck, Zhihong Chen, Akshay Chaudhari, Curtis Langlotz

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024

2411.03707 2024-11-07 cs.CV cs.AI 84%

Fine-Tuning Vision-Language Model for Automated Engineering Drawing Information Extraction

Muhammad Tayyab Khan, Lequn Chen, Ye Han Ng, Wenhe Feng, Nicholas Yew Jin Tan, Seung Ki Moon

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Paper has been submitted to the 9th International Conference on Innovation in Artificial Intelligence (ICIAI 2025)

2410.23317 2024-11-01 cs.CV cs.AI cs.CL cs.DC cs.PF 84%

VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration

Dezhan Tu, Danylo Vashchilenko, Yuzhe Lu, Panpan Xu

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

2410.17637 2024-10-24 cs.CV cs.AI 84%

MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models

Ziyu Liu, Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Haodong Duan, Conghui He, Yuanjun Xiong, Dahua Lin, Jiaqi Wang

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments Project URL: https://github.com/Liuziyu77/MIA-DPO

2410.15881 2024-10-22 cs.CV cs.AI 84%

MI-VisionShot: Few-shot adaptation of vision-language models for slide-level classification of histopathological images

Pablo Meseguer, Rocío del Amor, Valery Naranjo

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Manuscript accepted for oral presentation at KES-InnovationInMedicine 2024 held on Madeira, Portugal

2410.14072 2024-10-21 cs.CV cs.AI cs.CL 84%

Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers

Yuxin Wen, Qingqing Cao, Qichen Fu, Sachin Mehta, Mahyar Najibi

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

2410.13976 2024-10-21 cs.CV cs.CL cs.LG 84%

Debiasing Large Vision-Language Models by Ablating Protected Attribute Representations

Neale Ratzlaff, Matthew Lyle Olson, Musashi Hinck, Shao-Yen Tseng, Vasudev Lal, Phillip Howard

专题命中 VLM训练与架构：vision-language model(title)；vision language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.LG

Comments NeurIPS workshop on SafeGenAI, 10 pages, 2 figures

2410.11665 2024-10-16 cs.CV cs.AI cs.CL 84%

VisualRWKV-HD and UHD: Advancing High-Resolution Processing for Visual Language Models

Zihang Li, Haowen Hou

专题命中 VLM训练与架构：visual language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

2402.06659 2024-10-15 cs.CR cs.AI cs.LG 84%

Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models

Yuancheng Xu, Jiarui Yao, Manli Shu, Yanchao Sun, Zichu Wu, Ning Yu, Tom Goldstein, Furong Huang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.AI、cs.LG

Comments Accepted by Thirty-Eighth Annual Conference on Neural Information Processing Systems (Neurips 2024)

2410.09750 2024-10-15 cs.CV cs.AI 84%

Surgical-LLaVA: Toward Surgical Scenario Understanding via Large Language and Vision Models

Juseong Jin, Chang Wook Jeong

专题命中 VLM训练与架构：LLaVA(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024 AIM-FM Workshop

2410.09416 2024-10-15 cs.CV cs.AI 84%

Can Vision-Language Models Replace Human Annotators: A Case Study with CelebA Dataset

Haoming Lu, Feifei Zhong

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2024 Workshop (EvalEval 2024)

2410.09047 2024-10-14 cs.CL cs.AI cs.LG 84%

Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models

Qin Liu, Chao Shang, Ling Liu, Nikolaos Pappas, Jie Ma, Neha Anna John, Srikanth Doss, Lluis Marquez, Miguel Ballesteros, Yassine Benajiba

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.AI、cs.LG

Comments Preprint

1. VLM训练与架构 5067 篇

Vision-Language Model Dialog Games for Self-Improvement

Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understanding

ImageRef-VL: Enabling Contextual Image Referencing in Vision-Language Models

Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videos

VLM-driven Behavior Tree for Context-aware Task Planning

FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Models

CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries

Tuning Vision-Language Models with Candidate Labels by Prompt Alignment

HiRED: Attention-Guided Token Dropping for Efficient Inference of High-Resolution Vision-Language Models

VisualRWKV: Exploring Recurrent Neural Networks for Visual Language Models

ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsification

GIRAFFE: Design Choices for Extending the Context Length of Visual Language Models

SOLO: A Single Transformer for Scalable Vision-Language Modeling

Seeing Syntax: Uncovering Syntactic Learning Limitations in Vision-Language Models

If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions

Yo'LLaVA: Your Personalized Language and Vision Assistant

Multimodal large language model for wheat breeding: a new exploration of smart breeding

MpoxVLM: A Vision-Language Model for Diagnosing Skin Lesions from Mpox Virus Infection

RaVL: Discovering and Mitigating Spurious Correlations in Fine-Tuned Vision-Language Models

Fine-Tuning Vision-Language Model for Automated Engineering Drawing Information Extraction

VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration

MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models

MI-VisionShot: Few-shot adaptation of vision-language models for slide-level classification of histopathological images

Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers

Debiasing Large Vision-Language Models by Ablating Protected Attribute Representations

VisualRWKV-HD and UHD: Advancing High-Resolution Processing for Visual Language Models

Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models

Surgical-LLaVA: Toward Surgical Scenario Understanding via Large Language and Vision Models

Can Vision-Language Models Replace Human Annotators: A Case Study with CelebA Dataset

Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models