视觉大模型 / VLM - arXivDaily 专题

2511.17793 2025-11-25 cs.CV cs.LG 86%

Attention Guided Alignment in Efficient Vision-Language Models

注意力引导的高效视觉-语言模型

Shweta Mahajan, Hoang Le, Hyojin Park, Farzad Farhadzadeh, Munawar Hayat, Fatih Porikli

机构 * Qualcomm AI Research（高通人工智能研究）

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；grounding(abstract)；分类 cs.CV、cs.LG

AI总结本文提出AGE-VLM，通过交错交叉注意力层和空间知识提取，减少高效视觉-语言模型中的幻觉问题。

Comments 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Workshop on Efficient Reasoning

2511.01588 2025-11-24 cs.LG cs.CV 86%

Explore More, Learn Better: Parallel MLLM Embeddings under Mutual Information Minimization

探索更多，学习更好：基于互信息最小化的并行 MLLM 嵌入

Zhicheng Wang, Chen Ju, Xu Chen, Shuai Xiao, Jinsong Lan, Xiaoyong Zhu, Ying Chen, Zhiguo Cao

机构 * Zhejiang University（浙江大学）； Alibaba Group（阿里巴巴集团）； Huazhong University of Science and Technology（华中科技大学）

专题命中 VLM训练与架构：MLLM(title,abstract)；LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

AI总结本文提出基于互信息最小化的并行解耦框架，通过多条并行路径提升多模态嵌入质量，实现高效且鲁棒的嵌入空间。

2510.21794 2025-10-28 cs.CV cs.AI 86%

Token-Level Inference-Time Alignment for Vision-Language Models

Kejia Chen, Jiawen Zhang, Jiacong Hu, Kewei Gao, Jian Lou, Zunlei Feng, Mingli Song

机构 * Zhejiang University（浙江大学）； Sun Yat-sen University（中山大学）

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

2503.06073 2025-10-21 cs.CL cs.AI cs.CV 86%

GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images

Xiang Lan, Feng Wu, Kai He, Qinghao Zhao, Shenda Hong, Mengling Feng

机构 * National University of Singapore（新加坡国立大学）； Peking University People’s Hospital（北京大学人民医院）； Peking University（北京大学）

专题命中 VLM训练与架构：MLLM(title,abstract)；grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2025 Camera-Ready

2501.17391 2025-10-01 cs.CV cs.AI cs.CL 86%

LFTR: Learning-Free Token Reduction for Multimodal Large Language Models

Zihui Zhao, Yingxin Li, Yang Li

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2509.11815 2025-09-23 cs.CV cs.AI 86%

SpecVLM: Fast Speculative Decoding in Vision-Language Models

Haiduo Huang, Fuwei Yang, Zhenhua Liu, Xuanwu Yin, Dong Li, Pengju Ren, Emad Barsoum

机构 * Advanced Micro Devices, Inc.（先进微器件公司）； Institute of Artificial Intelligence and Robotics（人工智能与机器人研究院）

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

2509.00676 2025-09-03 cs.CV cs.LG 86%

LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model

Xiyao Wang, Chunyuan Li, Jianwei Yang, Kai Zhang, Bo Liu, Tianyi Xiong, Furong Huang

机构 * University of Maryland College Park（马里兰大学 College Park 分校）； The Ohio State University（俄亥俄州立大学）； National University of Singapore（新加坡国立大学）

专题命中 VLM训练与架构：LLaVA(title,abstract)；vision-language model(abstract)；visual reasoning(abstract)；分类 cs.CV、cs.LG

2508.21732 2025-09-01 cs.CV cs.AI 86%

CAD2DMD-SET: Synthetic Generation Tool of Digital Measurement Device CAD Model Datasets for fine-tuning Large Vision-Language Models

João Valente, Atabak Dehban, Rodrigo Ventura

机构 * Institute for Systems and Robotics（系统与机器人研究所）； University of Lisbon（里斯本大学）

专题命中 VLM训练与架构：vision-language model(title,abstract)；InternVL(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

2503.08525 2025-07-14 cs.CV cs.AI 86%

GTR: Guided Thought Reinforcement Prevents Thought Collapse in RL-based VLM Agent Training

Tong Wei, Yijun Yang, Junliang Xing, Yuanchun Shi, Zongqing Lu, Deheng Ye

机构 * Tsinghua University（清华大学）； Tencent（腾讯）； Peking University（北京大学）

专题命中 VLM训练与架构：VLM(title,abstract)；vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICCV 2025

2505.14260 2025-05-21 cs.CV cs.AI 86%

Speculative Decoding Reimagined for Multimodal Large Language Models

Luxi Lin, Zhihang Lin, Zhanpeng Zeng, Rongrong Ji

机构 * Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（多媒体可信感知与高效计算重点实验室、中华人民共和国教育部、厦门大学）

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 12 pages

2410.07149 2025-04-29 cs.CV cs.LG 86%

Towards Interpreting Visual Information Processing in Vision-Language Models

Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, Fazl Barez

机构 * Nanyang Technological University（南洋理工大学）； University of Oxford（牛津大学）； Tel Aviv University（特拉维夫大学）； MILA（蒙特利尔人工智能研究院）； ERA-Krueger AI Safety Lab（ERA-Krueger人工智能安全实验室）

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV、cs.LG

Comments Published at ICLR 2025

2503.21307 2025-03-28 cs.CV cs.AI 86%

InternVL-X: Advancing and Accelerating InternVL Series with Efficient Visual Token Compression

Dongchen Lu, Yuyao Sun, Zilu Zhang, Leping Huang, Jianliang Zeng, Mao Shu, Huo Cao

专题命中 VLM训练与架构：InternVL(title,abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2412.20682 2024-12-31 cs.CV cs.LG 86%

Learning to Rank Pre-trained Vision-Language Models for Downstream Tasks

Yuhe Ding, Bo Jiang, Aihua Zheng, Qin Xu, Jian Liang

专题命中 VLM训练与架构：vision-language model(title,abstract)；vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

2412.14197 2024-12-20 cs.CV cs.LG 86%

Advancing Vehicle Plate Recognition: Multitasking Visual Language Models with VehiclePaliGemma

Nouar AlDahoul, Myles Joshua Toledo Tan, Raghava Reddy Tera, Hezerul Abdul Karim, Chee How Lim, Manish Kumar Mishra, Yasir Zaki

专题命中 VLM训练与架构：visual language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV、cs.LG

Comments 33 pages, 9 figures

2412.03927 2024-12-06 cs.CV cs.LG 86%

MegaCOIN: Enhancing Medium-Grained Color Perception for Vision-Language Models

Ming-Chang Chiu, Shicheng Wen, Pin-Yu Chen, Xuezhe Ma

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；LLaVA(abstract)；分类 cs.CV、cs.LG

Comments 8 pages, 13 tables, 2 figures

2409.03206 2024-09-06 cs.CV cs.AI 86%

TC-LLaVA: Rethinking the Transfer from Image to Video Understanding with Temporal Considerations

Mingze Gao, Jingyu Liu, Mingda Li, Jiangtao Xie, Qingbin Liu, Bo Zhao, Xi Chen, Hui Xiong

专题命中 VLM训练与架构：LLaVA(title,abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2405.13581 2024-05-24 cs.CV cs.AI 86%

Safety Alignment for Vision Language Models

Zhendong Liu, Yuanbi Nie, Yingshui Tan, Xiangyu Yue, Qiushi Cui, Chongjun Wang, Xiaoyong Zhu, Bo Zheng

专题命中 VLM训练与架构：vision language model(title,abstract)；visual language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments 23 pages, 15 figures

2403.13447 2024-03-21 cs.AI cs.CL cs.CV 86%

HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models

Wenqiao Zhang, Tianwei Lin, Jiang Liu, Fangxun Shu, Haoyuan Li, Lei Zhang, He Wanggui, Hao Zhou, Zheqi Lv, Hao Jiang, Juncheng Li, Siliang Tang, Yueting Zhuang

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2309.10313 2023-12-06 cs.CL cs.AI cs.LG 86%

Investigating the Catastrophic Forgetting in Multimodal Large Language Models

Yuexiang Zhai, Shengbang Tong, Xiao Li, Mu Cai, Qing Qu, Yong Jae Lee, Yi Ma

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

2603.24696 2026-03-27 cs.CV 86%

LLaVA-LE: Large Language-and-Vision Assistant for Lunar Exploration

LLaVA-LE：用于月球探索的大型语言和视觉助手

Gokce Inal, Pouyan Navard, Alper Yilmaz

机构 * The Ohio State University（俄亥俄州立大学）

专题命中 VLM训练与架构：LLaVA(title,abstract)；vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

AI总结本文提出LLaVA-LE，一个专为月球表面和地下特征分析设计的多模态模型，通过构建大规模月球数据集LLUCID和两阶段训练方法，提升了行星科学领域的视觉语言模型性能。

Comments Accepted in AI4Space Workshop CVPR2026. Website: https://osupcvlab.github.io/LLaVA-LE/, Dataset: https://huggingface.co/datasets/pcvlab/lucid

2603.09877 2026-03-11 cs.CV 86%

InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing

InternVL-U: 使统一多模态模型在理解、推理、生成和编辑方面更加普及

Changyao Tian, Danni Yang, Guanzhou Chen, Erfei Cui, Zhaokai Wang, Yuchen Duan, Penghao Yin, Sitao Chen, Ganlin Yang, Mingxin Liu, Zirun Zhu, Ziqian Fan, Leyao Gu, Haomin Wang, Qi Wei, Jinhui Yin, Xue Yang, Zhihang Zhong, Qi Qin, Yi Xin, Bin Fu, Yihao Liu, Jiaye Ge, Qipeng Guo, Gen Luo, Hongsheng Li, Yu Qiao, Kai Chen, Hongjie Zhang

机构 * Shanghai AI Laboratory（上海人工智能实验室）； Fudan University（复旦大学）； University of Science and Technology of China（中国科学技术大学）； Shanghai Jiao Tong University（上海交通大学）； South China University of Technology（华南理工大学）； Nanjing University（南京大学）； Xiamen University（厦门大学）； CUHK MMLab（香港中文大学MMLab）

专题命中 VLM训练与架构：InternVL(title,abstract)；multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

AI总结 InternVL-U通过轻量级统一多模态模型，在保持强大生成能力的同时，实现了在理解、推理、生成和编辑方面的高效平衡。

Comments technical report, 61 pages, https://github.com/OpenGVLab/InternVL-U

2306.00890 2023-06-02 cs.CV cs.CL 86%

LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, Jianfeng Gao

专题命中 VLM训练与架构：LLaVA(title,abstract)；vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments 17 pages; Website: https://aka.ms/llava-med

2509.14001 2026-06-24 cs.CV cs.AI cs.LG 版本更新 86%

MOCHA: Multi-modal Objects-aware Cross-arcHitecture Alignment

MOCHA: 多模态对象感知的跨架构对齐

Elena Camuffo, Francesco Barbato, Mete Ozay, Simone Milani, Umberto Michieli

机构 * University of Padova（帕多瓦大学）

专题命中 VLM训练与架构：VLM(summary_cn,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结提出MOCHA蒸馏框架，将冻结VLM教师的多模态区域知识迁移至轻量视觉检测器，通过双重损失实现局部对齐与全局关系一致性，在少样本个性化检测中平均提升10.1%。

Comments 18 pages main paper, 10 pages supplementary material

Journal ref Proceedings of ECCV 2026

2606.10066 2026-06-10 cs.CV cs.AI cs.LG 新提交 86%

A Controlled Audit of Pretraining Contamination in Public Medical Vision-Language Benchmarks

公共医学视觉语言基准中预训练污染的受控审计

Bruce Changlong Xu, Lan Wu, Alexander Ryu

机构 * Stanford University（斯坦福大学）； University of California, Berkeley（加州大学伯克利分校）； Mayo Clinic（梅奥诊所）

专题命中 VLM训练与架构：VLM(summary_cn,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结审计发现公共医学VLM基准存在图像源重叠和文本规范顺序交换性信号，但确认的像素级重复罕见，且现有成员推理检测器在小规模医学VLM队列中不可靠。

Comments 30 pages, 7 figures, 9 tables. Preprint

2510.19496 2026-06-02 cs.CV cs.AI cs.LG 86%

CARES: Context-Aware Resolution Selector for VLMs

CARES: 面向视觉语言模型的上下文感知分辨率选择器

Moshe Kimhi, Nimrod Shabtay, Raja Giryes, Chaim Baskin, Eli Schwartz

机构 * Technion（技术ion大学）； IBM Research（IBM研究院）； Tel-Aviv University（特拉维夫大学）； Ben-Gurion University（本· Gurion大学）

专题命中 VLM训练与架构：VLM(summary_cn,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结提出CARES轻量级预处理模块，通过紧凑型VLM预测图像-查询对的最小足够分辨率，在保持任务性能的同时最多减少80%计算量。

Comments Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) Accepted to ACL 2026 (Oral presentation). Code available at https://github.com/mkimhi/CARES

2605.26533 2026-05-27 cs.CV cs.AI cs.CL cs.LG 86%

A Hybrid Vision-Language Architecture for Automated Defect Reasoning and Report Generation in Industrial Inspection

一种用于工业检测中自动缺陷推理与报告生成的混合视觉-语言架构

Malikussaid, Imad Gohar

机构 * School of Computing, Telkom University（Telkom大学计算机学院）； Faculty of Engineering and Technology, School of Computing and Artificial Intelligence（工程与技术学院，计算与人工智能学院）

专题命中 VLM训练与架构：VLM(summary_cn,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结本文提出一种解耦的边缘可部署管道，结合YOLO26-x-obb检测器、确定性编码模块和QLoRA微调的Qwen-2.5-1.5B模型，实现风电叶片缺陷定位与结构化报告生成，在BLEU-4、幻觉率和专家评分上显著优于零样本VLM基线。

Comments 23 pages, 6 figures, 9 equations, and 6 tables

2605.17602 2026-05-22 cs.AI cs.CV cs.LG 86%

AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment

AutoRubric-T2I: 一种用于文本到图像对齐的鲁棒基于规则的奖励模型

Kuei-Chun Kao, Daixuan Huo, Yuanhao Ban, Cho-Jui Hsieh

机构 * University of California, Los Angeles（加州大学洛杉矶分校）

专题命中 VLM训练与架构：VLM(summary_cn,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结本文提出AutoRubric-T2I，一种首个用于文本到图像生成的规则学习框架，通过自动合成和选择显式规则来指导视觉语言模型（VLM）法官。该方法通过合成偏好对的推理轨迹生成候选规则，并利用VLM法官在每种规则下对配对图像进行评分，产生配对规则评分差异用于偏好学习。通过ℓ1正则化逻辑回归精简器去除噪声和冗余规则，从而在少量标注偏好数据下生成高质量、可解释的奖励信号，并在多个图像奖励基准测试中优于现有奖励模型基线。

Comments 27 pages

2605.04128 2026-05-21 cs.GR cs.AI cs.CL cs.CV cs.LG 86%

JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation

JoyAI-Image: 激活统一多模态理解和生成中的空间智能

Lin Song, Wenbo Li, Guoqing Ma, Wei Tang, Bo Wang, Yuan Zhang, Yijun Yang, Yicheng Xiao, Jianhui Liu, Yanbing Zhang, Guohui Zhang, Wenhu Zhang, Hang Xu, Nan Jiang, Xin Han, Haoze Sun, Maoquan Zhang, Haoyang Huang, Nan Duan

机构 * Joy Future Academy, JD（京东探索研究院）

专题命中 VLM训练与架构：MLLM(summary_cn,abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结本文提出JoyAI-Image，一种统一的多模态基础模型，用于视觉理解、文本到图像生成和指令引导的图像编辑。该模型结合了空间增强的多模态大语言模型（MLLM）和多模态扩散Transformer（MMDiT），通过共享的多模态接口实现感知与生成的交互。构建可扩展的训练配方，结合统一指令微调、长文本渲染监督、空间 grounded 数据和通用及空间编辑信号，使模型具备广泛的多模态能力，同时增强几何感知推理和可控视觉合成。实验表明，JoyAI-Image在理解、生成、长文本渲染和编辑基准上达到最先进的性能。更重要的是，增强的理解、可控的空间编辑和新视角辅助推理之间的双向循环使模型超越一般视觉能力，向更强的空间智能发展。

Comments Code: https://github.com/jd-opensource/JoyAI-Image

2511.19820 2026-04-15 cs.CV cs.AI cs.CL cs.LG 86%

CropVLM: Learning to Zoom for Fine-Grained Vision-Language Perception

CropVLM: 为细粒度视觉语言感知学习动态聚焦

Miguel Carvalho, Helder Dias, Bruno Martins

机构 * INESC-ID, Instituto Superior Técnico, University of Lisbon（INESC-ID，理工学院，里斯本大学）

专题命中 VLM训练与架构：VLM(summary_cn,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 CropVLM通过强化学习提升VLM在细粒度图像理解任务中的性能，无需人工标注或合成数据，有效增强模型对细节的捕捉能力。

Comments Accepted to the GRAIL-V Workshop at CVPR 2026

2503.21227 2025-06-26 cs.CL 86%

LLaVA-CMoE: Towards Continual Mixture of Experts for Large Vision-Language Models

Hengyuan Zhao, Ziqin Wang, Qixin Sun, Kaiyou Song, Yilin Li, Xiaolin Hu, Qingpei Guo, Si Liu

机构 * School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）； Tsinghua University（清华大学）

专题命中 VLM训练与架构：LLaVA(title,abstract)；vision-language model(title)

Comments Preprint

1. VLM训练与架构 5058 篇

Attention Guided Alignment in Efficient Vision-Language Models

Explore More, Learn Better: Parallel MLLM Embeddings under Mutual Information Minimization

Token-Level Inference-Time Alignment for Vision-Language Models

GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images

LFTR: Learning-Free Token Reduction for Multimodal Large Language Models

SpecVLM: Fast Speculative Decoding in Vision-Language Models

LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model

CAD2DMD-SET: Synthetic Generation Tool of Digital Measurement Device CAD Model Datasets for fine-tuning Large Vision-Language Models

GTR: Guided Thought Reinforcement Prevents Thought Collapse in RL-based VLM Agent Training

Speculative Decoding Reimagined for Multimodal Large Language Models

Towards Interpreting Visual Information Processing in Vision-Language Models

InternVL-X: Advancing and Accelerating InternVL Series with Efficient Visual Token Compression

Learning to Rank Pre-trained Vision-Language Models for Downstream Tasks

Advancing Vehicle Plate Recognition: Multitasking Visual Language Models with VehiclePaliGemma

MegaCOIN: Enhancing Medium-Grained Color Perception for Vision-Language Models

TC-LLaVA: Rethinking the Transfer from Image to Video Understanding with Temporal Considerations

Safety Alignment for Vision Language Models

HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models

Investigating the Catastrophic Forgetting in Multimodal Large Language Models

LLaVA-LE: Large Language-and-Vision Assistant for Lunar Exploration

InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing

LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

MOCHA: Multi-modal Objects-aware Cross-arcHitecture Alignment

A Controlled Audit of Pretraining Contamination in Public Medical Vision-Language Benchmarks

CARES: Context-Aware Resolution Selector for VLMs

A Hybrid Vision-Language Architecture for Automated Defect Reasoning and Report Generation in Industrial Inspection

AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment

JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation

CropVLM: Learning to Zoom for Fine-Grained Vision-Language Perception

LLaVA-CMoE: Towards Continual Mixture of Experts for Large Vision-Language Models