视觉大模型 / VLM - arXivDaily 专题

2406.03907 2024-06-07 cs.CV 83%

Exploring the Zero-Shot Capabilities of Vision-Language Models for Improving Gaze Following

Anshul Gupta, Pierre Vuillecard, Arya Farkhondeh, Jean-Marc Odobez

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted at the GAZE Workshop at CVPR 2024

2311.17091 2024-06-04 cs.CV 83%

Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Models

Zhihe Lu, Jiawang Bai, Xin Li, Zeyu Xiao, Xinchao Wang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted on ICML 2024

2405.17247 2024-05-28 cs.LG 83%

An Introduction to Vision-Language Modeling

Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay, Alexander C. Li, Adrien Bardes, Suzanne Petryk, Oscar Mañas, Zhiqiu Lin, Anas Mahmoud, Bargav Jayaraman, Mark Ibrahim, Melissa Hall, Yunyang Xiong, Jonathan Lebensold, Candace Ross, Srihari Jayakumar, Chuan Guo, Diane Bouchacourt, Haider Al-Tahan, Karthik Padthe, Vasu Sharma, Hu Xu, Xiaoqing Ellen Tan, Megan Richards, Samuel Lavoie, Pietro Astolfi, Reyhane Askari Hemmat, Jun Chen, Kushal Tirumala, Rim Assouel, Mazda Moayeri, Arjang Talattof, Kamalika Chaudhuri, Zechun Liu, Xilun Chen, Quentin Garrido, Karen Ullrich, Aishwarya Agrawal, Kate Saenko, Asli Celikyilmaz, Vikas Chandra

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.LG

2405.11850 2024-05-21 cs.CV 83%

Rethinking Overlooked Aspects in Vision-Language Models

Yuan Liu, Le Tian, Xiao Zhou, Jie Zhou

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

2404.18746 2024-04-30 cs.MM cs.CV 83%

Enhancing Interactive Image Retrieval With Query Rewriting Using Large Language Models and Vision Language Models

Hongyi Zhu, Jia-Hong Huang, Stevan Rudinac, Evangelos Kanoulas

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

2404.16994 2024-04-30 cs.CV 83%

PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning

Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng, Jiashi Feng

专题命中 VLM训练与架构：LLaVA(title,abstract)；VLM(abstract)；分类 cs.CV

2310.02992 2024-04-29 cs.CV cs.CL 83%

Kosmos-G: Generating Images in Context with Multimodal Large Language Models

Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, Furu Wei

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Code: https://aka.ms/Kosmos-G Project Page: https://xichenpan.github.io/kosmosg

2404.16033 2024-04-25 cs.CV cs.CL 83%

Cantor: Inspiring Multimodal Chain-of-Thought of MLLM

Timin Gao, Peixian Chen, Mengdan Zhang, Chaoyou Fu, Yunhang Shen, Yan Zhang, Shengchuan Zhang, Xiawu Zheng, Xing Sun, Liujuan Cao, Rongrong Ji

专题命中 VLM训练与架构：MLLM(title)；visual reasoning(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments The project page is available at https://ggg0919.github.io/cantor/

2404.07824 2024-04-12 cs.CV cs.CL 83%

Heron-Bench: A Benchmark for Evaluating Vision Language Models in Japanese

Yuichi Inoue, Kento Sasaki, Yuma Ochi, Kazuki Fujii, Kotaro Tanahashi, Yu Yamaguchi

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

2305.07490 2024-04-08 cs.CL cs.CV 83%

ArtGPT-4: Towards Artistic-understanding Large Vision-Language Models with Enhanced Adapter

Zhengqing Yuan, Yunhong He, Kun Wang, Yanfang Ye, Lichao Sun

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

2403.17830 2024-03-27 cs.CV 83%

Assessment of Multimodal Large Language Models in Alignment with Human Values

Zhelun Shi, Zhipin Wang, Hongxing Fan, Zaibin Zhang, Lijun Li, Yongting Zhang, Zhenfei Yin, Lu Sheng, Yu Qiao, Jing Shao

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2311.02692

2310.08255 2024-03-12 cs.CV 83%

Leveraging Vision-Language Models for Improving Domain Generalization in Image Classification

Sravanti Addepalli, Ashish Ramayee Asokan, Lakshay Sharma, R. Venkatesh Babu

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Project page: http://val.cds.iisc.ac.in/VL2V-ADiP/

2312.13103 2024-03-05 cs.CL cs.CV 83%

Exploring Multimodal Large Language Models for Radiology Report Error-checking

Jinge Wu, Yunsoo Kim, Eva C. Keller, Jamie Chow, Adam P. Levine, Nikolas Pontikos, Zina Ibrahim, Paul Taylor, Michelle C. Williams, Honghan Wu

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

2402.15021 2024-03-04 cs.CV cs.CL 83%

CLoVe: Encoding Compositional Language in Contrastive Vision-Language Models

Santiago Castro, Amir Ziai, Avneesh Saluja, Zhuoning Yuan, Rada Mihalcea

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

2305.18010 2024-02-22 cs.CV cs.MM 83%

Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models

Shuai Zhao, Xiaohan Wang, Linchao Zhu, Yi Yang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments accepted by ICLR 2024, project page at https://mzhaoshuai.github.io/RLCF/, code is at https://github.com/mzhaoshuai/RLCF

2310.09199 2023-10-19 cs.CV 83%

PaLI-3 Vision Language Models: Smaller, Faster, Stronger

Xi Chen, Xiao Wang, Lucas Beyer, Alexander Kolesnikov, Jialin Wu, Paul Voigtlaender, Basil Mustafa, Sebastian Goodman, Ibrahim Alabdulmohsin, Piotr Padlewski, Daniel Salz, Xi Xiong, Daniel Vlasic, Filip Pavetic, Keran Rong, Tianli Yu, Daniel Keysers, Xiaohua Zhai, Radu Soricut

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

2309.07866 2023-09-21 cs.CV 83%

Gradient constrained sharpness-aware prompt learning for vision-language models

Liangchen Liu, Nannan Wang, Dawei Zhou, Xinbo Gao, Decheng Liu, Xi Yang, Tongliang Liu

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments 19 pages 11 figures

2308.13437 2023-09-15 cs.CV 83%

Position-Enhanced Visual Instruction Tuning for Multimodal Large Language Models

Chi Chen, Ruoyu Qin, Fuwen Luo, Xiaoyue Mi, Peng Li, Maosong Sun, Yang Liu

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

2309.00661 2023-09-06 cs.CV 83%

Zero-Shot Video Moment Retrieval from Frozen Vision-Language Models

Dezhao Luo, Jiabo Huang, Shaogang Gong, Hailin Jin, Yang Liu

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted by WACV 2024

2305.18786 2023-06-01 cs.CV cs.CL 83%

Scalable Performance Analysis for Vision-Language Models

Santiago Castro, Oana Ignat, Rada Mihalcea

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Camera-ready version for *SEM 2023

2303.17644 2023-04-03 cs.CV 83%

Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime

Rhydian Windsor, Amir Jamaludin, Timor Kadir, Andrew Zisserman

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted to MIDL 2023

2210.07795 2022-10-17 cs.CL cs.CV 83%

EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning

Tiannan Wang, Wangchunshu Zhou, Yan Zeng, Xinsong Zhang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments work in progress

2507.01790 2025-07-03 cs.CL cs.AI cs.CV cs.LG 83%

How Do Vision-Language Models Process Conflicting Information Across Modalities?

Tianze Hua, Tian Yun, Ellie Pavlick

机构 * Brown University（布朗大学）

专题命中 VLM训练与架构：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG；VLM(comments)

Comments All code and resources are available at: https://github.com/ethahtz/vlm_conflicting_info_processing

2608.16927 2026-08-19 cs.LG cs.CV 新提交 82%

Hierarchical Data Selection via Manifold Coverage and Sparse Feature Coverage in LLM Post-training

大语言模型后训练中基于流形覆盖与稀疏特征覆盖的分层数据选择

Peng Sun, Yi Yang, Antong Zhang, Chunxiao Li, Yanbo Wang, Dianbo Liu, xin chen, Kai Yu, Lu Chen, Tianfan Fu

专题命中 VLM训练与架构：LLaVA(summary_cn,abstract)；分类 cs.CV、cs.LG

AI总结本文提出MASS算法，将LLM后训练的数据选择建模为分层覆盖问题，在Vision Flan与LLaVA-CoT上仅用少量数据即可达到或超过全量训练效果，且优于现有基线方法。

2608.14130 2026-08-17 cs.MM cs.AI cs.CV cs.HC 新提交 82%

AlignFace: Human-Aligned Face Similarity Metric with Interpretable Concept Relations

AlignFace：具有可解释概念关系的人类对齐人脸相似度度量

Ying Huang, Wencan Zhang, Brian Y. Lim

专题命中 VLM训练与架构：VLM(summary_cn,abstract)；分类 cs.CV、cs.AI

AI总结本研究针对人脸生成模型评估中现有度量未实现认知对齐的问题，提出AlignFace度量，结合VLM、CA、CBM、GAM等技术，利用FACETS数据集，提升了与人类亚群感知的对齐度。

Comments 10 pages, 10 figures, 2 tables, ACM MM 26

2608.10835 2026-08-12 cs.CV cs.LG 新提交 82%

UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations

UniProbe：基于多结构内部表征的可学习大视觉语言模型（VLM） token 级幻觉检测器

Dvir Samuel, Guy Bar-Shalom, Fabrizio Frasca, Ethan Fetaya, Yftah Ziser, Gal Chechik, Haggai Maron

机构 * NVIDIA Research（英伟达研究院）； Technion（以色列理工学院）； Bar-Ilan University（巴伊兰大学）； University of Groningen（格罗宁根大学）

专题命中 VLM训练与架构：VLM(title_cn)；vision-language model(abstract)；visual reasoning(abstract)；分类 cs.CV、cs.LG

AI总结 UniProbe 是基于 LVLM 异构计算轨迹的可学习 token 级幻觉检测器，通过多结构模块交互实现 SOTA 检测性能，解码时可降 55% 对象幻觉且延迟仅增 6%

Comments Project Page: https://research.nvidia.com/labs/par/uniprobe/

2607.23488 2026-07-28 cs.LG cs.CV 新提交 82%

Learning Sampling Parameters for Diffusion Models

学习扩散模型的采样参数

Arisrei Lim, Yossi Gandelsman

专题命中 VLM训练与架构：VLM(summary_cn,abstract)；分类 cs.CV、cs.LG

AI总结研究针对扩散模型推理时采样参数固定的问题，提出LeSAMP框架，将参数选择转化为强化学习问题，通过人类偏好模型和VLM评判优化，实验表明该方法相比基线有更高胜率，为改进扩散模型输出提供补充途径。

Comments Code will be released

2601.22709 2026-06-29 cs.CV cs.AI 版本更新 82%

Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs

基于置信度蒸馏的门控关系对齐用于高效视觉语言模型

Yanlong Chen, Amirhossein Habibian, Luca Benini, Yawei Li

机构 * Department of Information Technology（信息科技系）； Electrical Engineering, ETH Zurich, Zurich, Switzerland（电气工程，苏黎世联邦理工学院，苏黎世，瑞士）； Qualcomm AI Research, Amsterdam, the Netherlands（高通人工智能研究，阿姆斯特丹，荷兰）； Department of Electrical, Electronic and Information Engineering（电气、电子与信息工程系）； University of Bologna, Bologna, Italy（博洛尼亚大学，博洛尼亚，意大利）； School of Electrical and Electronic Engineering（电气与电子工程学院）

专题命中 VLM训练与架构：LLaVA(abstract,abstract_cn)；vision-language model(abstract)；VLM(abstract_cn)；分类 cs.CV、cs.AI

AI总结提出GRACE框架，通过信息瓶颈原理统一知识蒸馏与量化感知训练，使用置信度门控解耦蒸馏、关系中心核对齐和自适应控制器，在INT4量化下实现性能超越FP16基线并接近教师模型，同时显著降低内存和提升吞吐量。

Comments Accepted to the International Conference on Machine Learning (ICML 2026)

2606.11670 2026-06-11 cs.CV cs.AI 新提交 82%

ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generation

ARGUS: 堆叠多视角身份马赛克注入用于主体保持的视频生成

Zijie Meng, Jiwen Liu, Yufei Liu, Chengzhuo Tong, Xiaoqiang Liu, Yuanxing Zhang, Yulong Xu, Pengfei Wan

机构 * Peking University（北京大学）； Kuaishou Technology（快手科技）； Xiamen University（厦门大学）

专题命中 VLM训练与架构：MLLM(summary_cn,abstract)；分类 cs.CV、cs.AI

AI总结提出ARGUS框架，通过堆叠多视角身份马赛克注入（SMII）将身份表示为紧凑动态分布，结合MLLM身份导演、无交叉对反事实训练等模块，在主体保持视频生成中达到SOTA。

Comments 13 pages, 3 figures

2606.11661 2026-06-11 cs.CV cs.LG 新提交 82%

Learning Instance-Adaptive Low-Rank Orthogonal Subspaces for Clothes-Changing Person Re-Identification

学习实例自适应低秩正交子空间用于换衣行人重识别

Dong-Woo Kim, Tae-Kyun Kim

机构 * Dongwoo Kim（金东吾）； Tae-Kyun Kim（金泰勋）

专题命中 VLM训练与架构：VLM(summary_cn,abstract)；分类 cs.CV、cs.LG

AI总结提出Ortho-ReID方法，通过从VLM文本描述中显式建模低秩服装子空间，并利用几何约束提取服装不变特征，在多个基准数据集上取得最优性能。

Comments Accepted to the ICML 2026 Workshop on CoLoRAI

1. VLM训练与架构 5067 篇

Exploring the Zero-Shot Capabilities of Vision-Language Models for Improving Gaze Following

Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Models

An Introduction to Vision-Language Modeling

Rethinking Overlooked Aspects in Vision-Language Models

Enhancing Interactive Image Retrieval With Query Rewriting Using Large Language Models and Vision Language Models

PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning

Kosmos-G: Generating Images in Context with Multimodal Large Language Models

Cantor: Inspiring Multimodal Chain-of-Thought of MLLM

Heron-Bench: A Benchmark for Evaluating Vision Language Models in Japanese

ArtGPT-4: Towards Artistic-understanding Large Vision-Language Models with Enhanced Adapter

Assessment of Multimodal Large Language Models in Alignment with Human Values

Leveraging Vision-Language Models for Improving Domain Generalization in Image Classification

Exploring Multimodal Large Language Models for Radiology Report Error-checking

CLoVe: Encoding Compositional Language in Contrastive Vision-Language Models

Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models

PaLI-3 Vision Language Models: Smaller, Faster, Stronger

Gradient constrained sharpness-aware prompt learning for vision-language models

Position-Enhanced Visual Instruction Tuning for Multimodal Large Language Models

Zero-Shot Video Moment Retrieval from Frozen Vision-Language Models

Scalable Performance Analysis for Vision-Language Models

Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime

EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning

How Do Vision-Language Models Process Conflicting Information Across Modalities?

Hierarchical Data Selection via Manifold Coverage and Sparse Feature Coverage in LLM Post-training

AlignFace: Human-Aligned Face Similarity Metric with Interpretable Concept Relations

UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations

Learning Sampling Parameters for Diffusion Models

Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs

ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generation

Learning Instance-Adaptive Low-Rank Orthogonal Subspaces for Clothes-Changing Person Re-Identification