视觉大模型 / VLM - arXivDaily 专题

2412.16855 2025-04-02 cs.CL cs.IR 67%

GME: Improving Universal Multimodal Retrieval by Multimodal LLMs

Xin Zhang, Yanzhao Zhang, Wen Xie, Mingxin Li, Ziqi Dai, Dingkun Long, Pengjun Xie, Meishan Zhang, Wenjie Li, Min Zhang

专题命中其他VLM ：multimodal large language model(abstract)；MLLM(abstract)

Comments Accepted to CVPR 2025, models at https://huggingface.co/Alibaba-NLP/gme-Qwen2-VL-2B-Instruct

2410.23156 2025-03-04 cs.AI cs.CV cs.LG cs.RO 67%

VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning

Yichao Liang, Nishanth Kumar, Hao Tang, Adrian Weller, Joshua B. Tenenbaum, Tom Silver, João F. Henriques, Kevin Ellis

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICLR 2025 (Spotlight)

2502.04263 2025-02-07 cs.CV cs.AI cs.LG 67%

Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion

Marco Mistretta, Alberto Baldrati, Lorenzo Agnolucci, Marco Bertini, Andrew D. Bagdanov

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted for publication at ICLR 2025

2412.15188 2025-02-06 cs.CL cs.AI cs.CV cs.LG 67%

LMFusion: Adapting Pretrained Language Models for Multimodal Generation

Weijia Shi, Xiaochuang Han, Chunting Zhou, Weixin Liang, Xi Victoria Lin, Luke Zettlemoyer, Lili Yu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Name change: LlamaFusion to LMFusion

2501.17171 2025-01-30 cs.CV cs.AI cs.LG eess.IV 67%

Separated Inter/Intra-Modal Fusion Prompts for Compositional Zero-Shot Learning

Sua Jung

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments AIAP 2025

Journal ref Published at AIAP 2025

2412.12627 2025-01-07 cs.CL 67%

Make Imagination Clearer! Stable Diffusion-based Visual Imagination for Multimodal Machine Translation

Andong Chen, Yuchen Song, Kehai Chen, Muyun Yang, Tiejun Zhao, Min Zhang

专题命中其他VLM ：multimodal large language model(abstract)；MLLM(abstract)

Comments Work in progress

2412.12232 2024-12-18 cs.CV cs.AI cs.LG 67%

You Only Submit One Image to Find the Most Suitable Generative Model

Zhi Zhou, Lan-Zhe Guo, Peng-Xiao Song, Yu-Feng Li

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by NeurIPS 2023 Workshop on Diffusion Models

2411.10063 2024-11-18 cs.AI cs.CV cs.LG 67%

Federated Domain Generalization via Prompt Learning and Aggregation

Shuai Gong, Chaoran Cui, Chunyun Zhang, Wenna Wang, Xiushan Nie, Lei Zhu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This work has been submitted to the IEEE for possible publication

2402.05741 2024-10-24 cs.RO cs.AI cs.CV cs.LG 67%

Real-World Robot Applications of Foundation Models: A Review

Kento Kawaharazuka, Tatsuya Matsushima, Andrew Gambardella, Jiaxian Guo, Chris Paxton, Andy Zeng

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2410.08612 2024-10-14 cs.CV cs.AI cs.LG 67%

Synth-SONAR: Sonar Image Synthesis with Enhanced Diversity and Realism via Dual Diffusion Models and GPT Prompting

Purushothaman Natarajan, Kamal Basha, Athira Nambiar

专题命中其他VLM ：visual language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 12 pages, 5 tables and 9 figures

2410.08172 2024-10-11 cs.RO cs.AI cs.CV cs.LG 67%

On the Evaluation of Generative Robotic Simulations

Feng Chen, Botian Xu, Pu Hua, Peiqi Duan, Yanchao Yang, Yi Ma, Huazhe Xu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Project website: https://sites.google.com/view/evaltasks

2404.16193 2024-09-23 cs.CV cs.AI cs.LG cs.MM eess.IV 67%

Improving Multi-label Recognition using Class Co-Occurrence Probabilities

Samyak Rawlekar, Shubhang Bhatnagar, Vishnuvardhan Pogunulu Srinivasulu, Narendra Ahuja

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to ICPR 2024, CVPR workshops 2024

2409.09362 2024-09-17 cs.CL 67%

Generating Event-oriented Attribution for Movies via Two-Stage Prefix-Enhanced Multimodal LLM

Yuanjie Lyu, Tong Xu, Zihan Niu, Bo Peng, Jing Ke, Enhong Chen

专题命中其他VLM ：multimodal large language model(abstract)；MLLM(abstract)

2407.08583 2024-08-06 cs.AI cs.CV cs.LG 67%

The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective

Zhen Qin, Daoyuan Chen, Wenhao Zhang, Liuyi Yao, Yilun Huang, Bolin Ding, Yaliang Li, Shuiguang Deng

专题命中其他VLM ：MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Ongoing work. 21 pages. Related materials are continually maintained and available at https://github.com/modelscope/data-juicer/blob/main/docs/awesome_llm_data.md

2408.01417 2024-08-05 cs.CL cs.AI cs.CV cs.LG 67%

Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMs

Yilun Hua, Yoav Artzi

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to COLM 2024

2407.03056 2024-07-31 cs.CV cs.AI cs.LG 67%

Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

Marco Mistretta, Alberto Baldrati, Marco Bertini, Andrew D. Bagdanov

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted for publication at ECCV24

2404.15592 2024-07-23 cs.CV cs.AI cs.CL cs.IR cs.LG 67%

ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extraction

Henry Peng Zou, Vinay Samuel, Yue Zhou, Weizhi Zhang, Liancheng Fang, Zihe Song, Philip S. Yu, Cornelia Caragea

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by ACL 2024 (Findings) - Scores: Soundness - 4/4/4, Dataset - 4/4/4, Overall Assessment - 4/3.5/3.5, Meta - 4

2407.14412 2024-07-22 cs.CV cs.AI cs.LG 67%

DEAL: Disentangle and Localize Concept-level Explanations for VLMs

Tang Li, Mengmeng Ma, Xi Peng

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments In Proceedings of the European Conference on Computer Vision (ECCV), 2024

2406.03586 2024-06-11 cs.CV cs.AI cs.LG 67%

CountCLIP -- [Re] Teaching CLIP to Count to Ten

Harshvardhan Mestha, Tejas Agrawal, Karan Bania, Shreyas V, Yash Bhisikar

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2406.02780 2024-06-06 cs.CV cs.AI cs.LG 67%

LADI v2: Multi-label Dataset and Classifiers for Low-Altitude Disaster Imagery

Samuel Scheele, Katherine Picchione, Jeffrey Liu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2406.02554 2024-06-06 eess.AS cs.AI cs.CL cs.CV cs.LG cs.MM 67%

Hear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognition

Shijian Deng, Erin E. Kosloski, Siddhi Patel, Zeke A. Barnett, Yiyang Nan, Alexander Kaplan, Sisira Aarukapalli, William T. Doan, Matthew Wang, Harsh Singh, Pamela R. Rollins, Yapeng Tian

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2404.18532 2024-05-16 cs.CL cs.AI cs.CV cs.LG 67%

MileBench: Benchmarking MLLMs in Long Context

Dingjie Song, Shunian Chen, Guiming Hardy Chen, Fei Yu, Xiang Wan, Benyou Wang

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 31 pages, 13 figures, 14 tables; We add results of GPT-4o in this version

2404.16305 2024-04-29 cs.MM cs.SD eess.AS 67%

Semantically consistent Video-to-Audio Generation using Multimodal Language Large Model

Gehui Chen, Guan'an Wang, Xiaowen Huang, Jitao Sang

专题命中其他VLM ：multimodal large language model(abstract)；MLLM(abstract)

2404.08937 2024-04-16 cs.CV cs.AI cs.LG 67%

ChimpVLM: Ethogram-Enhanced Chimpanzee Behaviour Recognition

Otto Brookes, Majid Mirmehdi, Hjalmar Kuhl, Tilo Burghardt

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2402.08473 2024-02-14 cs.CV cs.AI cs.LG 67%

Intriguing Differences Between Zero-Shot and Systematic Evaluations of Vision-Language Transformer Models

Shaeke Salman, Md Montasir Bin Shams, Xiuwen Liu, Lingjiong Zhu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 30 pages, 30 figures

2308.07931 2024-01-02 cs.CV cs.AI cs.CL cs.LG cs.RO 67%

Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation

William Shen, Ge Yang, Alan Yu, Jansen Wong, Leslie Pack Kaelbling, Phillip Isola

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Project website at https://f3rm.csail.mit.edu, Accepted at the 7th Annual Conference on Robot Learning (CoRL), 2023 in Atlanta, US

2312.03818 2023-12-15 cs.CV cs.AI cs.CL cs.LG 67%

Alpha-CLIP: A CLIP Model Focusing on Wherever You Want

Zeyi Sun, Ye Fang, Tong Wu, Pan Zhang, Yuhang Zang, Shu Kong, Yuanjun Xiong, Dahua Lin, Jiaqi Wang

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments project page: https://aleafy.github.io/alpha-clip code: https://github.com/SunzeY/AlphaCLIP

2312.04344 2023-12-13 cs.CL cs.AI cs.CV cs.LG 67%

Enhancing Medical Task Performance in GPT-4V: A Comprehensive Study on Prompt Engineering Strategies

Pengcheng Chen, Ziyan Huang, Zhongying Deng, Tianbin Li, Yanzhou Su, Haoyu Wang, Jin Ye, Yu Qiao, Junjun He

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2310.10625 2023-10-17 cs.CV cs.AI cs.LG cs.RO 67%

Video Language Planning

Yilun Du, Mengjiao Yang, Pete Florence, Fei Xia, Ayzaan Wahid, Brian Ichter, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B. Tenenbaum, Leslie Kaelbling, Andy Zeng, Jonathan Tompson

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments https://video-language-planning.github.io/

2206.00621 2023-06-13 cs.CL cs.AI cs.CV cs.LG 67%

Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training

Yan Zeng, Wangchunshu Zhou, Ao Luo, Ziming Cheng, Xinsong Zhang

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ACL 2023

1. 其他VLM 1578 篇

GME: Improving Universal Multimodal Retrieval by Multimodal LLMs

VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning

Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion

LMFusion: Adapting Pretrained Language Models for Multimodal Generation

Separated Inter/Intra-Modal Fusion Prompts for Compositional Zero-Shot Learning

Make Imagination Clearer! Stable Diffusion-based Visual Imagination for Multimodal Machine Translation

You Only Submit One Image to Find the Most Suitable Generative Model

Federated Domain Generalization via Prompt Learning and Aggregation

Real-World Robot Applications of Foundation Models: A Review

Synth-SONAR: Sonar Image Synthesis with Enhanced Diversity and Realism via Dual Diffusion Models and GPT Prompting

On the Evaluation of Generative Robotic Simulations

Improving Multi-label Recognition using Class Co-Occurrence Probabilities

Generating Event-oriented Attribution for Movies via Two-Stage Prefix-Enhanced Multimodal LLM

The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective

Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMs

Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extraction

DEAL: Disentangle and Localize Concept-level Explanations for VLMs

CountCLIP -- [Re] Teaching CLIP to Count to Ten

LADI v2: Multi-label Dataset and Classifiers for Low-Altitude Disaster Imagery

Hear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognition

MileBench: Benchmarking MLLMs in Long Context

Semantically consistent Video-to-Audio Generation using Multimodal Language Large Model

ChimpVLM: Ethogram-Enhanced Chimpanzee Behaviour Recognition

Intriguing Differences Between Zero-Shot and Systematic Evaluations of Vision-Language Transformer Models

Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation

Alpha-CLIP: A CLIP Model Focusing on Wherever You Want

Enhancing Medical Task Performance in GPT-4V: A Comprehensive Study on Prompt Engineering Strategies

Video Language Planning

Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training