视觉大模型 / VLM - arXivDaily 专题

2411.10928 2024-11-19 cs.CL cs.AI 83%

Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning

Wenke Huang, Jian Liang, Zekun Shi, Didi Zhu, Guancheng Wan, He Li, Bo Du, Dacheng Tao, Mang Ye

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.AI

2411.10803 2024-11-19 cs.CV 83%

Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model

Ting Liu, Liangtao Shi, Richang Hong, Yue Hu, Quanjun Yin, Linfeng Zhang

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments 8 pages, 4figures

2402.18068 2024-11-19 cs.CV 83%

SynArtifact: Classifying and Alleviating Artifacts in Synthetic Images via Vision-Language Model

Bin Cao, Jianhao Yuan, Yexin Liu, Jian Li, Shuyang Sun, Jing Liu, Bo Zhao

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

2411.09301 2024-11-15 cs.CV 83%

LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation

Zhenshi Li, Dilxat Muhtar, Feng Gu, Xueliang Zhang, Pengfeng Xiao, Guangjun He, Xiaoxiang Zhu

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

2411.04942 2024-11-08 cs.CV 83%

A Reinforcement Learning-Based Automatic Video Editing Method Using Pre-trained Vision-Language Model

Panwen Hu, Nan Xiao, Feifei Li, Yongquan Chen, Rui Huang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

2410.16261 2024-11-08 cs.CV 83%

Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance

Zhangwei Gao, Zhe Chen, Erfei Cui, Yiming Ren, Weiyun Wang, Jinguo Zhu, Hao Tian, Shenglong Ye, Junjun He, Xizhou Zhu, Lewei Lu, Tong Lu, Yu Qiao, Jifeng Dai, Wenhai Wang

专题命中 VLM训练与架构：InternVL(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments Technical report

2410.23089 2024-10-31 cs.CV 83%

PIP-MM: Pre-Integrating Prompt Information into Visual Encoding via Existing MLLM Structures

Tianxiang Wu, Minxin Nie, Ziqiang Cao

专题命中 VLM训练与架构：MLLM(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

2410.22313 2024-10-30 cs.CV cs.RO 83%

Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving

Bo Jiang, Shaoyu Chen, Bencheng Liao, Xingyu Zhang, Wei Yin, Qian Zhang, Chang Huang, Wenyu Liu, Xinggang Wang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Project Page: https://github.com/hustvl/Senna

2410.17827 2024-10-24 cs.AI 83%

RE-tune: Incremental Fine Tuning of Biomedical Vision-Language Models for Multi-label Chest X-ray Classification

Marco Mistretta, Andrew D. Bagdanov

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.AI

Comments Accepted for publication at Medical Imaging meets NeurIPS (NeurIPS23)

2408.15881 2024-10-24 cs.CV 83%

LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillation

Fangxun Shu, Yue Liao, Le Zhuo, Chenning Xu, Lei Zhang, Guanghao Zhang, Haonan Shi, Long Chen, Tao Zhong, Wanggui He, Siming Fu, Haoyuan Li, Bolin Li, Zhelun Yu, Si Liu, Hongsheng Li, Hao Jiang

专题命中 VLM训练与架构：LLaVA(title,abstract)；MLLM(abstract)；分类 cs.CV

2410.13861 2024-10-22 cs.CV 83%

PUMA: Empowering Unified MLLM with Multi-granular Visual Generation

Rongyao Fang, Chengqi Duan, Kun Wang, Hao Li, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, Hongsheng Li, Xihui Liu

专题命中 VLM训练与架构：MLLM(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments Project page: https://rongyaofang.github.io/puma/

2403.13756 2024-10-16 cs.CV 83%

Enhancing Gait Video Analysis in Neurodegenerative Diseases by Knowledge Augmentation in Vision Language Model

Diwei Wang, Kun Yuan, Candice Muller, Frédéric Blanc, Nicolas Padoy, Hyewon Seo

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments MICCAI 2024

2410.07133 2024-10-11 cs.CV 83%

EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Models

Rui Zhao, Hangjie Yuan, Yujie Wei, Shiwei Zhang, Yuchao Gu, Lingmin Ran, Xiang Wang, Zhangjie Wu, Junhao Zhang, Yingya Zhang, Mike Zheng Shou

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

2410.06456 2024-10-10 cs.CV 83%

From Generalist to Specialist: Adapting Vision Language Models via Task-Specific Visual Instruction Tuning

Yang Bai, Yang Zhou, Jun Zhou, Rick Siow Mong Goh, Daniel Shu Wei Ting, Yong Liu

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

2403.16167 2024-10-07 cs.CV cs.CL 83%

ESREAL: Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models

Minchan Kim, Minyeong Kim, Junik Bae, Suhwan Choi, Sungkyung Kim, Buru Chang

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments ECCV 2024

2311.10122 2024-10-02 cs.CV 83%

Video-LLaVA: Learning United Visual Representation by Alignment Before Projection

Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, Li Yuan

专题命中 VLM训练与架构：LLaVA(title,abstract)；vision-language model(abstract)；分类 cs.CV

2311.01477 2024-09-30 cs.CV 83%

FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models

Liqiang Jing, Ruosen Li, Yunmo Chen, Xinya Du

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments Accepted by Findings of EMNLP 2024

2409.17805 2024-09-27 cs.CV 83%

Cascade Prompt Learning for Vision-Language Model Adaptation

Ge Wu, Xin Zhang, Zheng Li, Zhaowei Chen, Jiajun Liang, Jian Yang, Xiang Li

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments ECCV2024

2409.16723 2024-09-27 cs.CV 83%

EAGLE: Towards Efficient Arbitrary Referring Visual Prompts Comprehension for Multimodal Large Language Models

Jiacheng Zhang, Yang Jiao, Shaoxiang Chen, Jingjing Chen, Yu-Gang Jiang

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

2409.12597 2024-09-20 cs.CV 83%

LARE: Latent Augmentation using Regional Embedding with Vision-Language Model

Kosuke Sakurai, Tatsuya Ishii, Ryotaro Shimizu, Linxin Song, Masayuki Goto

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments 10 pages, 4 figures

2408.16930 2024-09-02 cs.CV 83%

VLM-KD: Knowledge Distillation from VLM for Long-Tail Visual Recognition

Zaiwei Zhang, Gregory P. Meyer, Zhichao Lu, Ashish Shrivastava, Avinash Ravichandran, Eric M. Wolff

专题命中 VLM训练与架构：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV

2408.15556 2024-08-29 cs.CV 83%

Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models

Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, Dacheng Tao

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

2408.09530 2024-08-20 cs.AI 83%

PA-LLaVA: A Large Language-Vision Assistant for Human Pathology Image Understanding

Dawei Dai, Yuanhui Zhang, Long Xu, Qianlan Yang, Xiaojing Shen, Shuyin Xia, Guoyin Wang

专题命中 VLM训练与架构：LLaVA(title,abstract)；vision-language model(abstract)；分类 cs.AI

Comments 8 pages, 4 figs

2408.01800 2024-08-06 cs.CV 83%

MiniCPM-V: A GPT-4V Level MLLM on Your Phone

Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, Qianyu Chen, Huarong Zhou, Zhensheng Zou, Haoye Zhang, Shengding Hu, Zhi Zheng, Jie Zhou, Jie Cai, Xu Han, Guoyang Zeng, Dahai Li, Zhiyuan Liu, Maosong Sun

专题命中 VLM训练与架构：MLLM(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments preprint

2405.11537 2024-08-06 cs.RO cs.AI cs.ET 83%

VR-GPT: Visual Language Model for Intelligent Virtual Reality Applications

Mikhail Konenkov, Artem Lykov, Daria Trinitatova, Dzmitry Tsetserukou

专题命中 VLM训练与架构：visual language model(title,abstract)；VLM(abstract)；分类 cs.AI

Comments Updated version

2407.09043 2024-07-24 cs.AI 83%

Vision Language Model is NOT All You Need: Augmentation Strategies for Molecule Language Models

Namkyeong Lee, Siddhartha Laghuvarapu, Chanyoung Park, Jimeng Sun

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.AI

Comments CIKM 2024 / ACL 2024 Workshop on Languages and Molecule

2407.12709 2024-07-18 cs.CV 83%

MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Models

Leyang Shen, Gongwei Chen, Rui Shao, Weili Guan, Liqiang Nie

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Github: https://github.com/JiuTian-VL/MoME

2403.11150 2024-07-11 cs.CV 83%

Training A Small Emotional Vision Language Model for Visual Art Comprehension

Jing Zhang, Liang Zheng, Meng Wang, Dan Guo

专题命中 VLM训练与架构：vision language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments 16 pages

2406.17770 2024-06-28 cs.CV 83%

MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning

Xiangyu Zhao, Xiangtai Li, Haodong Duan, Haian Huang, Yining Li, Kai Chen, Hua Yang

专题命中 VLM训练与架构：LLaVA(title,abstract)；MLLM(abstract)；分类 cs.CV

2406.17659 2024-06-26 cs.AI cs.RO 83%

DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planning

Xiaohan Zhang, Zainab Altaweel, Yohei Hayamizu, Yan Ding, Saeid Amiri, Hao Yang, Andy Kaminski, Chad Esselink, Shiqi Zhang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.AI

1. VLM训练与架构 5067 篇

Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning

Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model

SynArtifact: Classifying and Alleviating Artifacts in Synthetic Images via Vision-Language Model

LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation

A Reinforcement Learning-Based Automatic Video Editing Method Using Pre-trained Vision-Language Model

Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance

PIP-MM: Pre-Integrating Prompt Information into Visual Encoding via Existing MLLM Structures

Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving

RE-tune: Incremental Fine Tuning of Biomedical Vision-Language Models for Multi-label Chest X-ray Classification

LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillation

PUMA: Empowering Unified MLLM with Multi-granular Visual Generation

Enhancing Gait Video Analysis in Neurodegenerative Diseases by Knowledge Augmentation in Vision Language Model

EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Models

From Generalist to Specialist: Adapting Vision Language Models via Task-Specific Visual Instruction Tuning

ESREAL: Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models

Video-LLaVA: Learning United Visual Representation by Alignment Before Projection

FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models

Cascade Prompt Learning for Vision-Language Model Adaptation

EAGLE: Towards Efficient Arbitrary Referring Visual Prompts Comprehension for Multimodal Large Language Models

LARE: Latent Augmentation using Regional Embedding with Vision-Language Model

VLM-KD: Knowledge Distillation from VLM for Long-Tail Visual Recognition

Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models

PA-LLaVA: A Large Language-Vision Assistant for Human Pathology Image Understanding

MiniCPM-V: A GPT-4V Level MLLM on Your Phone

VR-GPT: Visual Language Model for Intelligent Virtual Reality Applications

Vision Language Model is NOT All You Need: Augmentation Strategies for Molecule Language Models

MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Models

Training A Small Emotional Vision Language Model for Visual Art Comprehension

MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning

DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planning