视觉大模型 / VLM

Contrastive Regularization over LoRA for Multimodal Biomedical Image Incremental Learning

Haojie Zhang, Yixiong Liang, Hulin Kuang, Lihui Cen, Zhe Qu, Yigang Cen, Min Zeng, Shichao Kan

机构 * School of Computer Science and Engineering, Central South University（计算机科学与工程学院，中南大学） ； School of Automation, Central South University（自动化学院，中南大学） ； School of Computer Science and Technology, Beijing Jiaotong University（计算机科学与技术学院，北京交通大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 10 pages, 3 figures, submitted to ACM Multimedia 2025

Bridging Modality Gaps in e-Commerce Products via Vision-Language Alignment

Yipeng Zhang, Hongju Yu, Aritra Mandal, Canran Xu, Qunzhi Zhou, Zhe Wu

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)

M3LLM: Model Context Protocol-aided Mixture of Vision Experts For Multimodal LLMs in Networks

Yongjie Zeng, Hongyang Du

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)

SceneLoom: Communicating Data with Scene Context

Lin Gao, Leixian Shen, Yuheng Zhao, Jiexiang Lan, Huamin Qu, Siming Chen

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

Hierarchical Vision-Language Planning for Multi-Step Humanoid Manipulation

André Schakkal, Ben Zandonati, Zhutian Yang, Navid Azizan

机构 * Massachusetts Institute of Technology（麻省理工学院） ； École Polytechnique Fédérale de Lausanne（瑞士联邦理工学院） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

Comments Accepted at the RSS 2025 Workshop on Robot Planning in the Era of Foundation Models

Bridging Perception and Language: A Systematic Benchmark for LVLMs' Understanding of Amodal Completion Reports

Amane Watahiki, Tomoki Doi, Taiga Shinozaki, Satoshi Nishida, Takuya Niikawa, Katsunori Miyahara, Hitomi Yanaka

机构 * The University of Tokyo（东京大学） ； Keio University（庆应大学） ； NICT（国家信息与通信技术研究所） ； Osaka University（大阪大学） ； Hokkaido University（北海道大学） ； CiNET（信息网络研究中心） ； Kobe University（Kobe 大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；LLaVA(abstract)

Comments To appear in the Proceedings of the 47th Annual Meeting of the Cognitive Science Society (COGSCI 2025)

Training-free Generation of Temporally Consistent Rewards from VLMs

Yinuo Zhao, Jiale Yuan, Zhiyuan Xu, Xiaoshuai Hao, Xinyi Zhang, Kun Wu, Zhengping Che, Chi Harold Liu, Jian Tang

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

CLIP-MG: Guiding Semantic Attention with Skeletal Pose Features and RGB Data for Micro-Gesture Recognition on the iMiGUE Dataset

Santosh Patapati, Trisanth Srinivasan, Amith Adiraju

机构 * Cyrion Labs, Texas, United States（Cyrion实验室，德克萨斯州，美国） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

PIPE: Physics-Informed Position Encoding for Alignment of Satellite Images and Time Series

Haobo Li, Eunseo Jung, Zixin Chen, Zhaowei Wang, Yueya Wang, Huamin Qu, Alexis Kai Hon Lau

机构 * Department of Computer Science & Engineering（香港理工大学计算机科学与工程系） ； Division of Environment & Sustainability（香港理工大学环境与可持续发展学院） 纠错

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG

RobotSmith: Generative Robotic Tool Design for Acquisition of Complex Manipulation Skills

Chunru Lin, Haotian Yuan, Yian Wang, Xiaowen Qiu, Tsun-Hsuan Wang, Minghao Guo, Bohan Wang, Yashraj Narang, Dieter Fox, Chuang Gan

机构 * University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校） ； Massachusetts Institute of Technology（麻省理工学院） ； National University of Singapore（新加坡国立大学） ； NVIDIA（英伟达） ； MIT-IBM Watson AI Lab（麻省理工-IBM沃森人工智能实验室） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence

Yibo Yang, Sihao Liu, Chuan Rao, Bang An, Tiancheng Shen, Philip H. S. Torr, Ming-Hsuan Yang, Bernard Ghanem

机构 * King Abdullah University of Science and Technology, Saudi Arabia（卡布尔大学） ； University of California, Merced, USA（加州大学梅尔德分校） ； University of Oxford, U.K.（牛津大学） 纠错

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Efficient Medical Vision-Language Alignment Through Adapting Masked Vision Models

Chenyu Lian, Hong-Yu Zhou, Dongyun Liang, Jing Qin, Liansheng Wang

机构 * School of Informatics, Xiamen University（厦门大学信息学院） ； Center for Smart Health, School of Nursing, The Hong Kong Polytechnic University（香港理工大学护理学院智能健康中心） ； Department of Biomedical Informatics, Harvard Medical School（哈佛医学院生物医学信息学系） ； Department of Radiology, Zhongshan Hospital (Xiamen), Fudan University（复旦大学中山医院放射科） ； National Institute for Data Science in Health and Medicine（医学与数据科学国家研究院） ； Department of Computer Science, School of Informatics, Xiamen University（厦门大学信息学院计算机科学系） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments TMI 2025

Taking Flight with Dialogue: Enabling Natural Language Control for PX4-based Drone Agent

Shoon Kit Lim, Melissa Jia Ying Chong, Jing Huey Khor, Ting Yang Ling

机构 * University of Southampton Malaysia（英国南安普顿大学马来西亚分校） ； Connected Intelligence Research Group（连接智能研究组） ； Sustainable Electronic Technologies（可持续电子技术） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

Comments Source code available at: https://github.com/limshoonkit/ros2-agent-ws

ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT

Mikołaj Pokrywka, Wojciech Kusa, Mieszko Rutkowski, Mikołaj Koszowski

机构 * Machine Learning Research Allegro.com（Allegro公司机器学习研究部） ； NASK 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

Comments Accepted at ACL 2025 (The 63rd Annual Meeting of the Association for Computational Linguistics)

GuessBench: Sensemaking Multimodal Creativity in the Wild

Zifeng Zhu, Shangbin Feng, Herun Wan, Ningnan Wang, Minnan Luo, Yulia Tsvetkov

机构 * Xi’an Jiaotong University（西安交通大学） ； University of Washington（华盛顿大学） 纠错

专题命中 VLM训练与架构 ：vision language model(abstract)；VLM(abstract)

Contrastive Visual Data Augmentation

Yu Zhou, Bingxuan Li, Mohan Tang, Xiaomeng Jin, Te-Lin Wu, Kuan-Hao Huang, Heng Ji, Kai-Wei Chang, Nanyun Peng

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Journal ref ICML 2025

Data-Juicer Sandbox: A Feedback-Driven Suite for Multimodal Data-Model Co-development

Daoyuan Chen, Haibin Wang, Yilun Huang, Ce Ge, Yaliang Li, Bolin Ding, Jingren Zhou

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by ICML 2025 (Spotlight). 33 pages, 16 tables, 14 figures

On VLMs for Diverse Tasks in Multimodal Meme Classification

Deepesh Gavit, Debajyoti Mazumder, Samiran Das, Jasabanta Patro

机构 * Peng Wang and Shuai Bai and Sinan Tan and Shijie Wang and Zhihao Fan and Jinze Bai and Keqin Chen and Xuejing Liu and Jialin Wang and Wenbin Ge and Yang Fan and Kai Dang and Mengfei Du and Xuancheng Ren and Rui Men and Dayiheng Liu and Chang Zhou and Jingren Zhou and Junyang Lin（研究人员） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

Comments 16 pages

Parrot: Multilingual Visual Instruction Tuning

Hai-Long Sun, Da-Wei Zhou, Yang Li, Shiyin Lu, Chao Yi, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, De-Chuan Zhan, Han-Jia Ye

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to ICML 2025. Code and dataset are available at: https://github.com/AIDC-AI/Parrot

What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluation

Michal Golovanevsky, William Rudman, Vedant Palit, Ritambhara Singh, Carsten Eickhoff

机构 * Department of Computer Science, Brown University（布朗大学计算机科学系） ； Indian Institute of Technology Kharagpur（印度理工学院Khargpur分校） ； Center for Computational Molecular Biology, Brown University（布朗大学计算分子生物学中心） ； School of Medicine, University of Tübingen（图宾根大学医学院） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

MA-ROESL: Motion-aware Rapid Reward Optimization for Efficient Robot Skill Learning from Single Videos

Xianghui Wang, Xinming Zhang, Yanjun Chen, Xiaoyu Shen, Wei Zhang

机构 * College of Information Science and Technology, Eastern Institute of Technology（信息科学与技术学院，东部技术学院） ； School of Computer Science and Technology, The University of Science and Technology of China（计算机科学与技术学院，中国科学技术大学） ； Department of Computing, The Hong Kong Polytechnic University（计算学院，香港理工大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

UAV-VLRR: Vision-Language Informed NMPC for Rapid Response in UAV Search and Rescue

Yasheerah Yaqoot, Muhammad Ahsan Mustafa, Oleg Sautenkov, Artem Lykov, Valerii Serpiva, Dzmitry Tsetserukou

机构 * Intelligent Space Robotics Laboratory, Center for Digital Engineering, Skolkovo Institute of Science and Technology（智能空间机器人实验室、数字工程中心、斯克尔科沃科学与技术研究所） 纠错

专题命中 VLM训练与架构 ：VLM(abstract)；visual language model(abstract)

Comments UAV-VLRR

An LLM-Empowered Low-Resolution Vision System for On-Device Human Behavior Understanding

Siyang Jiang, Bufang Yang, Lilin Xu, Mu Yuan, Yeerzhati Abudunuer, Kaiwei Liu, Liekang Zeng, Hongkai Chen, Zhenyu Yan, Xiaofan Jiang, Guoliang Xing

机构 * The Chinese University of Hong Kong, Hong Kong SAR（香港中文大学） ； Columbia University, United States（哥伦比亚大学） 纠错

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations?

Letitia Parcalabescu, Anette Frank

机构 * Computational Linguistics Department（计算语言学系） ； Heidelberg University（海德堡大学） 纠错

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 30 pages, 8 figures, 11 tables

Can VLMs Assess Similarity Between Graph Visualizations?

Seokweon Jung, Hyeon Jeon, Jeongmin Rhee, Jinwook Seo

专题命中 VLM训练与架构 ：vision language model(abstract)；VLM(abstract)

TULIP: Towards Unified Language-Image Pretraining

Zineng Tang, Long Lian, Seun Eisape, XuDong Wang, Roei Herzig, Adam Yala, Alane Suhr, Trevor Darrell, David M. Chan

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments (v2) Clarified fine-tuning process, updated appendix

Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal LLMs on Academic Resources

Weizhi Wang, Yu Tian, Linjie Yang, Heng Wang, Xifeng Yan

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)

Fwd2Bot: LVLM Visual Token Compression with Double Forward Bottleneck

Adrian Bulat, Yassine Ouali, Georgios Tzimiropoulos

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG

MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D Content Creation

Sankalp Sinha, Mohammad Sadil Khan, Muhammad Usama, Shino Sam, Didier Stricker, Sk Aziz Ali, Muhammad Zeshan Afzal

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

RankCLIP: Ranking-Consistent Language-Image Pretraining

Yiming Zhang, Zhuokai Zhao, Zhaorun Chen, Zhili Feng, Zenghui Ding, Yining Sun

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Code and model checkpoints are available at https://github.com/Jam1ezhang/RankCLIP