视觉大模型 / VLM - arXivDaily 专题

2403.16697 2024-07-16 cs.CV 57%

DPStyler: Dynamic PromptStyler for Source-Free Domain Generalization

Yunlong Tang, Yuxuan Wan, Lei Qi, Xin Geng

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted by IEEE TMM

2307.03073 2024-07-16 cs.CV cs.RO 57%

Proto-CLIP: Vision-Language Prototypical Network for Few-Shot Learning

Jishnu Jaykumar P, Kamalesh Palanisamy, Yu-Wei Chao, Xinya Du, Yu Xiang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted at 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

2404.19286 2024-07-15 cs.CV 57%

Soft Prompt Generation for Domain Generalization

Shuanghao Bai, Yuedi Zhang, Wanqi Zhou, Zhirong Luan, Badong Chen

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV

Comments 25 pages, 4 figures, accepted by ECCV 2024

2404.05052 2024-07-15 cs.CV 57%

Facial Affective Behavior Analysis with Instruction Tuning

Yifan Li, Anh Dao, Wentao Bao, Zhen Tan, Tianlong Chen, Huan Liu, Yu Kong

专题命中 VLM训练与架构：MLLM(abstract)；分类 cs.CV

Comments V2.0, project page: https://johnx69.github.io/FABA/

2407.08196 2024-07-12 cs.AI 57%

SoupLM: Model Integration in Large Language and Multi-Modal Models

Yue Bai, Zichen Zhang, Jiasen Lu, Yun Fu

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.AI

2405.08786 2024-07-11 cs.CV 57%

Incorporating Clinical Guidelines through Adapting Multi-modal Large Language Model for Prostate Cancer PI-RADS Scoring

Tiantian Zhang, Manxi Lin, Hongda Guo, Xiaofan Zhang, Ka Fung Peter Chiu, Aasa Feragen, Qi Dou

专题命中 VLM训练与架构：MLLM(abstract)；分类 cs.CV

2402.09353 2024-07-10 cs.CL cs.CV 57%

DoRA: Weight-Decomposed Low-Rank Adaptation

Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, Min-Hung Chen

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

Comments ICML2024(Oral)

2407.06084 2024-07-09 cs.CV 57%

3D Vision and Language Pretraining with Large-Scale Synthetic Data

Dejie Yang, Zhu Xu, Wentao Mo, Qingchao Chen, Siyuan Huang, Yang Liu

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.CV

Comments accepted by IJCAI2024

2403.06407 2024-07-09 cs.CV 57%

Can LLMs' Tuning Methods Work in Medical Multimodal Domain?

Jiawei Chen, Yue Jiang, Dingkang Yang, Mingcheng Li, Jinjie Wei, Ziyun Qian, Lihua Zhang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted by MICCAI 2024

2407.05419 2024-07-09 cs.CV cs.IR 57%

Multimodal Language Models for Domain-Specific Procedural Video Summarization

Nafisa Hussain

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV

Comments 6 pages, 3 figures

2407.04689 2024-07-08 cs.RO cs.CV 57%

RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulation

Yuxuan Kuang, Junjie Ye, Haoran Geng, Jiageng Mao, Congyue Deng, Leonidas Guibas, He Wang, Yue Wang

专题命中 VLM训练与架构：VLM(abstract)；分类 cs.CV

2407.04003 2024-07-08 cs.CV 57%

Fully Fine-tuned CLIP Models are Efficient Few-Shot Learners

Mushui Liu, Bozheng Li, Yunlong Yu

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments 8 pages, 9 figures

2407.03036 2024-07-04 cs.CV 57%

SAFT: Towards Out-of-Distribution Generalization in Fine-Tuning

Bac Nguyen, Stefan Uhlich, Fabien Cardinaux, Lukas Mauch, Marzieh Edraki, Aaron Courville

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2407.02477 2024-07-03 cs.CV cs.CL 57%

Understanding Alignment in Multimodal LLMs: A Comprehensive Study

Elmira Amirloo, Jean-Philippe Fauconnier, Christoph Roesmann, Christian Kerl, Rinu Boney, Yusu Qian, Zirui Wang, Afshin Dehghan, Yinfei Yang, Zhe Gan, Peter Grasch

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV

2405.12752 2024-07-03 cs.CV 57%

C3L: Content Correlated Vision-Language Instruction Tuning Data Generation via Contrastive Learning

Ji Ma, Wei Suo, Peng Wang, Yanning Zhang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted by IJCAI-24

2407.00959 2024-07-02 cs.AI cs.RO 57%

Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Driving

Ran Tian, Boyi Li, Xinshuo Weng, Yuxiao Chen, Edward Schmerling, Yue Wang, Boris Ivanovic, Marco Pavone

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.AI

2405.04940 2024-07-02 cs.CV 57%

Harnessing the Power of MLLMs for Transferable Text-to-Image Person ReID

Wentao Tan, Changxing Ding, Jiayu Jiang, Fei Wang, Yibing Zhan, Dapeng Tao

专题命中 VLM训练与架构：MLLM(abstract)；分类 cs.CV

Comments CVPR 2024

2407.00203 2024-07-02 cs.CV 57%

PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaboration

Yuxuan Sun, Yunlong Zhang, Yixuan Si, Chenglu Zhu, Zhongyi Shui, Kai Zhang, Jingxiong Li, Xingheng Lyu, Tao Lin, Lin Yang

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV

Comments 13 pages, 3 figures

2406.19255 2024-06-28 cs.CV cs.CL 57%

Enhancing Video-Language Representations with Structural Spatio-Temporal Alignment

Hao Fei, Shengqiong Wu, Meishan Zhang, Min Zhang, Tat-Seng Chua, Shuicheng Yan

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.CV

Comments Accepted by IEEE TPAMI 2024

Journal ref [J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

2401.09181 2024-06-28 cs.LG 57%

Beyond Anti-Forgetting: Multimodal Continual Instruction Tuning with Positive Forward Transfer

Junhao Zheng, Qianli Ma, Zhen Liu, Binquan Wu, Huawen Feng

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.LG

2406.18139 2024-06-27 cs.CL cs.CV 57%

LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference

Zhongwei Wan, Ziang Wu, Che Liu, Jinfa Huang, Zhihong Zhu, Peng Jin, Longyue Wang, Li Yuan

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV

2402.13561 2024-06-27 cs.CL cs.CV 57%

Cognitive Visual-Language Mapper: Advancing Multimodal Comprehension with Enhanced Visual Knowledge Alignment

Yunxin Li, Xinyu Chen, Baotian Hu, Haoyuan Shi, Min Zhang

专题命中 VLM训练与架构：visual question answering(abstract)；分类 cs.CV

Comments 12 pages,4 figures; Accepted by ACL 2024 Main Conference

2406.17057 2024-06-26 astro-ph.IM astro-ph.GA cs.AI 57%

At First Sight: Zero-Shot Classification of Astronomical Images with Large Multimodal Models

Dimitrios Tanoglidis, Bhuvnesh Jain

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.AI

Comments 5 pages, 3 images. Prepared for submission to RNAAS

2401.03149 2024-06-21 cs.CV 57%

CaMML: Context-Aware Multimodal Learner for Large Models

Yixin Chen, Shuai Zhang, Boran Han, Tong He, Bo Li

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

Comments Preprint

2406.12211 2024-06-19 cs.CV 57%

PCIE_LAM Solution for Ego4D Looking At Me Challenge

Kanokphan Lertniphonphan, Jun Xie, Yaqing Meng, Shijing Wang, Feng Chen, Zhepeng Wang

专题命中 VLM训练与架构：InternVL(abstract)；分类 cs.CV

2402.03746 2024-06-18 cs.CV 57%

Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback

Daechul Ahn, Yura Choi, Youngjae Yu, Dongyeop Kang, Jonghyun Choi

专题命中 VLM训练与架构：VLM(abstract)；分类 cs.CV

Comments ACL 2024

2312.08578 2024-06-18 cs.CV 57%

A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions

Jack Urbanek, Florian Bordes, Pietro Astolfi, Mary Williamson, Vasu Sharma, Adriana Romero-Soriano

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2402.10986 2024-06-17 cs.CL cs.AI 57%

FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models

Gagan Bhatia, El Moatez Billah Nagoudi, Hasan Cavusoglu, Muhammad Abdul-Mageed

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.AI

2406.09240 2024-06-14 cs.CV 57%

Comparison Visual Instruction Tuning

Wei Lin, Muhammad Jehanzeb Mirza, Sivan Doveh, Rogerio Feris, Raja Giryes, Sepp Hochreiter, Leonid Karlinsky

专题命中 VLM训练与架构：visual reasoning(abstract)；分类 cs.CV

Comments Project page: https://wlin-at.github.io/cad_vi ; Huggingface dataset repo: https://huggingface.co/datasets/wlin21at/CaD-Inst

2402.09871 2024-06-14 cs.SD cs.AI cs.MM eess.AS 57%

MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Music

Zihao Wang, Shuyu Li, Tao Zhang, Qi Wang, Pengfei Yu, Jinyang Luo, Yan Liu, Ming Xi, Kejun Zhang

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.AI

Comments Accepted by International Joint Conference on Artificial Intelligence 2024 (IJCAI 2024)

1. VLM训练与架构 5058 篇

DPStyler: Dynamic PromptStyler for Source-Free Domain Generalization

Proto-CLIP: Vision-Language Prototypical Network for Few-Shot Learning

Soft Prompt Generation for Domain Generalization

Facial Affective Behavior Analysis with Instruction Tuning

SoupLM: Model Integration in Large Language and Multi-Modal Models

Incorporating Clinical Guidelines through Adapting Multi-modal Large Language Model for Prostate Cancer PI-RADS Scoring

DoRA: Weight-Decomposed Low-Rank Adaptation

3D Vision and Language Pretraining with Large-Scale Synthetic Data

Can LLMs' Tuning Methods Work in Medical Multimodal Domain?

Multimodal Language Models for Domain-Specific Procedural Video Summarization

RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulation

Fully Fine-tuned CLIP Models are Efficient Few-Shot Learners

SAFT: Towards Out-of-Distribution Generalization in Fine-Tuning

Understanding Alignment in Multimodal LLMs: A Comprehensive Study

C3L: Content Correlated Vision-Language Instruction Tuning Data Generation via Contrastive Learning

Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Driving

Harnessing the Power of MLLMs for Transferable Text-to-Image Person ReID

PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaboration

Enhancing Video-Language Representations with Structural Spatio-Temporal Alignment

Beyond Anti-Forgetting: Multimodal Continual Instruction Tuning with Positive Forward Transfer

LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference

Cognitive Visual-Language Mapper: Advancing Multimodal Comprehension with Enhanced Visual Knowledge Alignment

At First Sight: Zero-Shot Classification of Astronomical Images with Large Multimodal Models

CaMML: Context-Aware Multimodal Learner for Large Models

PCIE_LAM Solution for Ego4D Looking At Me Challenge

Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback

A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions

FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models

Comparison Visual Instruction Tuning

MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Music