视觉大模型 / VLM - arXivDaily 专题

2412.07679 2025-02-11 cs.CV cs.AI 62%

RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models

Greg Heinrich, Mike Ranzinger, Hongxu, Yin, Yao Lu, Jan Kautz, Andrew Tao, Bryan Catanzaro, Pavlo Molchanov

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV、cs.AI

2411.19309 2025-02-05 cs.RO cs.CV cs.LG 62%

GRAPE: Generalizing Robot Policy via Preference Alignment

Zijian Zhang, Kaiyuan Zheng, Zhaorun Chen, Joel Jang, Yi Li, Siwei Han, Chaoqi Wang, Mingyu Ding, Dieter Fox, Huaxiu Yao

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Website: https://grape-vla.github.io/

2501.19086 2025-02-03 cs.CV cs.AI 62%

Fairness Analysis of CLIP-Based Foundation Models for X-Ray Image Classification

Xiangyu Sun, Xiaoguang Zou, Yuanquan Wu, Guotai Wang, Shaoting Zhang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments This paper has been accepted for presentation at the 2025 IEEE International Symposium on Biomedical Imaging (ISBI 2025)

2501.15074 2025-01-28 cs.CV cs.AI 62%

PatentLMM: Large Multimodal Model for Generating Descriptions for Patent Figures

Shreya Shukla, Nakul Sharma, Manish Gupta, Anand Mishra

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted at AAAI 2025 (Main Track). Project page: https://vl2g.github.io/projects/PatentLMM/

2411.16027 2025-01-28 cs.CV cs.AI 62%

From Dashcam Videos to Driving Simulations: Stress Testing Automated Vehicles against Rare Events

Yan Miao, Georgios Fainekos, Bardh Hoxha, Hideki Okamoto, Danil Prokhorov, Sayan Mitra

专题命中 VLM训练与架构：VLM(abstract)；分类 cs.CV、cs.AI

2501.13787 2025-01-24 cs.CL cs.AI cs.LG 62%

Parameter-Efficient Fine-Tuning for Foundation Models

Dan Zhang, Tao Feng, Lilong Xue, Yuandong Wang, Yuxiao Dong, Jie Tang

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.AI、cs.LG

Comments 25 pages, 6 figures, 7 tables

2501.08347 2025-01-16 cs.CV cs.AI 62%

SCOT: Self-Supervised Contrastive Pretraining For Zero-Shot Compositional Retrieval

Bhavin Jawade, Joao V. B. Soares, Kapil Thadani, Deen Dayal Mohan, Amir Erfan Eshratifar, Benjamin Culpepper, Paloma de Juan, Srirangaraj Setlur, Venu Govindaraju

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Paper accepted at WACV 2025 in round 1

2501.07978 2025-01-15 cs.CV cs.AI 62%

Facial Dynamics in Video: Instruction Tuning for Improved Facial Expression Perception and Contextual Awareness

Jiaxing Zhao, Boyuan Sun, Xiang Chen, Xihan Wei

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV、cs.AI

2412.18842 2024-12-30 cs.CV cs.LG 62%

Context-Based Semantic-Aware Alignment for Semi-Supervised Multi-Label Learning

Heng-Bo Fan, Ming-Kun Xie, Jia-Hao Xiao, Sheng-Jun Huang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV、cs.LG

2412.18150 2024-12-30 cs.CV cs.AI 62%

EvalMuse-40K: A Reliable and Fine-Grained Benchmark with Comprehensive Human Annotations for Text-to-Image Generation Model Evaluation

Shuhao Han, Haotian Fan, Jiachen Fu, Liang Li, Tao Li, Junhui Cui, Yunqiu Wang, Yang Tai, Jingwei Sun, Chunle Guo, Chongyi Li

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV、cs.AI

2412.16201 2024-12-24 cs.RO cs.AI cs.LG cs.SY eess.SY 62%

CLIP-RLDrive: Human-Aligned Autonomous Driving via CLIP-Based Reward Shaping in Reinforcement Learning

Erfan Doroudian, Hamid Taghavifar

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.AI、cs.LG

2406.10522 2024-12-19 cs.LG cs.AI cs.CL 62%

Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning

Jifan Zhang, Lalit Jain, Yang Guo, Jiayi Chen, Kuan Lok Zhou, Siddharth Suresh, Andrew Wagenmaker, Scott Sievert, Timothy Rogers, Kevin Jamieson, Robert Mankoff, Robert Nowak

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.AI、cs.LG

2412.10594 2024-12-17 cs.CV cs.LG 62%

Towards Unified Benchmark and Models for Multi-Modal Perceptual Metrics

Sara Ghazanfari, Siddharth Garg, Nicolas Flammarion, Prashanth Krishnamurthy, Farshad Khorrami, Francesco Croce

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV、cs.LG

2412.10348 2024-12-16 cs.CV cs.AI 62%

A dual contrastive framework

Yuan Sun, Zhao Zhang, Jorge Ortiz

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV、cs.AI

2412.08979 2024-12-13 cs.LG cs.CV 62%

A Wander Through the Multimodal Landscape: Efficient Transfer Learning via Low-rank Sequence Multimodal Adapter

Zirun Guo, Xize Cheng, Yangyang Wu, Tao Jin

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted at AAAI 2025

2412.08635 2024-12-12 cs.CL cs.CV cs.LG 62%

Multimodal Latent Language Modeling with Next-Token Diffusion

Yutao Sun, Hangbo Bao, Wenhui Wang, Zhiliang Peng, Li Dong, Shaohan Huang, Jianyong Wang, Furu Wei

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV、cs.LG

2412.05536 2024-12-10 eess.IV cs.AI cs.CL cs.CV 62%

Comprehensive Evaluation of Multimodal AI Models in Medical Imaging Diagnosis: From Data Augmentation to Preference-Based Comparison

Cailian Ruan, Chengyue Huang, Yahe Yang

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV、cs.AI

2407.00978 2024-12-10 cs.AI cs.LG 62%

Hybrid RAG-empowered Multi-modal LLM for Secure Data Management in Internet of Medical Things: A Diffusion-based Contract Approach

Cheng Su, Jinbo Wen, Jiawen Kang, Yonghua Wang, Yuanjia Su, Hudan Pan, Zishao Zhong, M. Shamim Hossain

专题命中 VLM训练与架构：MLLM(abstract)；分类 cs.AI、cs.LG

Comments 13 pages, 7 figures

2412.04652 2024-12-09 cs.CV cs.AI 62%

Cross-Self KV Cache Pruning for Efficient Vision-Language Inference

Xiaohuan Pei, Tao Huang, Chang Xu

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV、cs.AI

2412.03467 2024-12-05 cs.CV cs.AI 62%

Training-Free Mitigation of Language Reasoning Degradation After Multimodal Instruction Tuning

Neale Ratzlaff, Man Luo, Xin Su, Vasudev Lal, Phillip Howard

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV、cs.AI

2412.01289 2024-12-05 cs.CV cs.AI 62%

Enhancing Perception Capabilities of Multimodal LLMs with Training-Free Fusion

Zhuokun Chen, Jinwu Hu, Zeshuai Deng, Yufeng Wang, Bohan Zhuang, Mingkui Tan

专题命中 VLM训练与架构：MLLM(abstract)；分类 cs.CV、cs.AI

2409.14607 2024-12-02 cs.CV cs.LG 62%

Patch Ranking: Efficient CLIP by Learning to Rank Local Patches

Cheng-En Wu, Jinhong Lin, Yu Hen Hu, Pedro Morgado

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted by WACV 2025

2411.17891 2024-11-28 cs.CL cs.AI cs.CV 62%

HOPPR Medical-Grade Platform for Medical Imaging AI

Kalina P. Slavkova, Melanie Traughber, Oliver Chen, Robert Bakos, Shayna Goldstein, Dan Harms, Bradley J. Erickson, Khan M. Siddiqui

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV、cs.AI

Comments 6 pages, 3 figures

2405.14129 2024-11-26 cs.CL cs.AI cs.CV 62%

AlignGPT: Multi-modal Large Language Models with Adaptive Alignment Capability

Fei Zhao, Taotian Pang, Chunhui Li, Zhen Wu, Junjie Guo, Shangyu Xing, Xinyu Dai

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV、cs.AI

2411.15099 2024-11-25 cs.CV cs.CL cs.LG 62%

Context-Aware Multimodal Pretraining

Karsten Roth, Zeynep Akata, Dima Damen, Ivana Balažević, Olivier J. Hénaff

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV、cs.LG

2411.12713 2024-11-20 cs.CV cs.AI 62%

CATCH: Complementary Adaptive Token-level Contrastive Decoding to Mitigate Hallucinations in LVLMs

Zhehan Kan, Ce Zhang, Zihan Liao, Yapeng Tian, Wenming Yang, Junyuan Xiao, Xu Li, Dongmei Jiang, Yaowei Wang, Qingmin Liao

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV、cs.AI

2411.06142 2024-11-12 cs.CV cs.AI 62%

Aquila-plus: Prompt-Driven Visual-Language Models for Pixel-Level Remote Sensing Image Understanding

Kaixuan Lu

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV、cs.AI

2411.06074 2024-11-12 cs.CV cs.AI 62%

Aquila: A Hierarchically Aligned Visual-Language Model for Enhanced Remote Sensing Image Comprehension

Kaixuan Lu, Ruiqian Zhang, Xiao Huang, Yuxing Xie

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV、cs.AI

2405.17871 2024-11-06 cs.CV cs.AI cs.CL 62%

Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment

Xin Xiao, Bohong Wu, Jiacong Wang, Chunyuan Li, Xun Zhou, Haoyuan Guo

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV、cs.AI

Comments NeurlPS 2024, Camera ready

2405.10301 2024-11-06 stat.ML cs.AI cs.LG 62%

Conformal Alignment: Knowing When to Trust Foundation Models with Guarantees

Yu Gui, Ying Jin, Zhimei Ren

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.AI、cs.LG

1. VLM训练与架构 5067 篇

RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models

GRAPE: Generalizing Robot Policy via Preference Alignment

Fairness Analysis of CLIP-Based Foundation Models for X-Ray Image Classification

PatentLMM: Large Multimodal Model for Generating Descriptions for Patent Figures

From Dashcam Videos to Driving Simulations: Stress Testing Automated Vehicles against Rare Events

Parameter-Efficient Fine-Tuning for Foundation Models

SCOT: Self-Supervised Contrastive Pretraining For Zero-Shot Compositional Retrieval

Facial Dynamics in Video: Instruction Tuning for Improved Facial Expression Perception and Contextual Awareness

Context-Based Semantic-Aware Alignment for Semi-Supervised Multi-Label Learning

EvalMuse-40K: A Reliable and Fine-Grained Benchmark with Comprehensive Human Annotations for Text-to-Image Generation Model Evaluation

CLIP-RLDrive: Human-Aligned Autonomous Driving via CLIP-Based Reward Shaping in Reinforcement Learning

Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning

Towards Unified Benchmark and Models for Multi-Modal Perceptual Metrics

A dual contrastive framework

A Wander Through the Multimodal Landscape: Efficient Transfer Learning via Low-rank Sequence Multimodal Adapter

Multimodal Latent Language Modeling with Next-Token Diffusion

Comprehensive Evaluation of Multimodal AI Models in Medical Imaging Diagnosis: From Data Augmentation to Preference-Based Comparison

Hybrid RAG-empowered Multi-modal LLM for Secure Data Management in Internet of Medical Things: A Diffusion-based Contract Approach

Cross-Self KV Cache Pruning for Efficient Vision-Language Inference

Training-Free Mitigation of Language Reasoning Degradation After Multimodal Instruction Tuning

Enhancing Perception Capabilities of Multimodal LLMs with Training-Free Fusion

Patch Ranking: Efficient CLIP by Learning to Rank Local Patches

HOPPR Medical-Grade Platform for Medical Imaging AI

AlignGPT: Multi-modal Large Language Models with Adaptive Alignment Capability

Context-Aware Multimodal Pretraining

CATCH: Complementary Adaptive Token-level Contrastive Decoding to Mitigate Hallucinations in LVLMs

Aquila-plus: Prompt-Driven Visual-Language Models for Pixel-Level Remote Sensing Image Understanding

Aquila: A Hierarchically Aligned Visual-Language Model for Enhanced Remote Sensing Image Comprehension

Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment

Conformal Alignment: Knowing When to Trust Foundation Models with Guarantees