视觉大模型 / VLM

Learning Hyperspectral Images with Curated Text Prompts for Efficient Multimodal Alignment

Abhiroop Chatterjee, Susmita Ghosh

机构 * Jadavpur University（贾瓦帕尔大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at the IEEE/CVF International Conference on Computer Vision (ICCV 2025), Workshop on Curated Data for Efficient Learning

Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgetting

Asher J. Hancock, Xindi Wu, Lihan Zha, Olga Russakovsky, Anirudha Majumdar

机构 * Princeton University（普林斯顿大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；visual question answering(abstract)

Cross-Modal Instructions for Robot Motion Generation

William Barron, Xiaoxiang Dong, Matthew Johnson-Roberson, Weiming Zhi

机构 * College of Connected Computing, Vanderbilt University（连接计算学院，范德比尔特大学） ； Robotics Institute, Carnegie Mellon University（机器人研究所，卡内基梅隆大学） ； School of Computer Science, The University of Sydney（计算机科学学院，悉尼大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

No Labels Needed: Zero-Shot Image Classification with Collaborative Self-Learning

Matheus Vinícius Todescato, Joel Luís Carbonera

机构 * Institute of Informatics（信息学院） ； UFRGS（乌拉圭国家研究学院） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This paper was accepted at International Conference on Tools with Artificial Intelligence (ICTAI) 2025

GLAM: Geometry-Guided Local Alignment for Multi-View VLP in Mammography

Yuexi Du, Lihui Chen, Nicha C. Dvornek

机构 * Department of Biomedical Engineering（生物医学工程系） ； Department of Radiology & Biomedical Imaging（放射科与生物医学成像系） ； Yale University（耶鲁大学） 纠错

专题命中 VLM训练与架构 ：VLM(abstract)；visual language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by MICCAI 2025

Adaptive Token Merging for Efficient Transformer Semantic Communication at the Edge

Omar Erak, Omar Alhussein, Hatem Abou-Zeid, Mehdi Bennis, Sami Muhaidat

机构 * Department of Electrical Engineering and Computer Science（电气工程与计算机科学系） ； Khalifa University（卡里马大学） 纠错

专题命中 VLM训练与架构 ：LLaVA(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Submitted to IEEE Journals

Retrieval-Augmented VLMs for Multimodal Melanoma Diagnosis

Jihyun Moon, Charmgil Hong

机构 * Handong Global University（-handong全球大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Medical Image Computing and Computer-Assisted Intervention (MICCAI) ISIC Skin Image Analysis Workshop (MICCAI ISIC) 2025; 10 pages

X-Prompt: Towards Universal In-Context Image Generation in Auto-Regressive Vision Language Foundation Models

Zeyi Sun, Ziyang Chu, Pan Zhang, Tong Wu, Xiaoyi Dong, Yuhang Zang, Yuanjun Xiong, Dahua Lin, Jiaqi Wang

机构 * Shanghai Jiao Tong University（上海交通大学） ； Shanghai AI Laboratory（上海人工智能实验室） ； Tsinghua University（清华大学） ； The Chinese University of Hong Kong（香港中文大学） ； CPII under InnoHK（创新香港科技促进会） ； MThreads AI 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments code: https://github.com/SunzeY/X-Prompt

Identify, Isolate, and Purge: Mitigating Hallucinations in LVLMs via Self-Evolving Distillation

Wenhao Li, Xiu Su, Jingyi Wu, Feng Yang, Yang Liu, Yi Chen, Shan You, Chang Xu

专题命中 VLM训练与架构 ：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments In Figure 2, the correlation coefficient and the scatter plot do not match. I calculated this correlation using two sets of settings. I used the scatter plot from setting A, but accidentally wrote the correlation coefficient, r, from setting B

Libra: Leveraging Temporal Images for Biomedical Radiology Analysis

Xi Zhang, Zaiqiao Meng, Jake Lever, Edmond S. L. Ho

机构 * Information Retrieval Group（信息检索组） ； AI4BioMed Lab（AI4BioMed实验室） ； School of Computing Science（计算科学学院） ； University of Glasgow（格拉斯哥大学） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 30 pages, 5 figures, Adding Appendix

Journal ref Association for Computational Linguistics, 2025

NeuroVoxel-LM: Language-Aligned 3D Perception via Dynamic Voxelization and Meta-Embedding

Shiyu Liu, Lianlei Shan

机构 * School of Electrical and Electronic Engineering（电气与电子工程学院） ； Nanyang Technological University（南洋理工大学） ； School of Computer Science and Technology（计算机科学与技术学院） ； University of Chinese Academy of Sciences（中国科学院大学） 纠错

专题命中 VLM训练与架构 ：visual language model(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments **14 pages, 3 figures, 2 tables

Pixels to Principles: Probing Intuitive Physics Understanding in Multimodal Language Models

Mohamad Ballout, Serwan Jassim, Elia Bruni

机构 * Institute of Cognitive Science, University of Osnabrück（认知科学研究所，奥斯纳布吕克大学） 纠错

专题命中 VLM训练与架构 ：LLaVA(abstract)；InternVL(abstract)；multimodal large language model(abstract)

RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignment

Difei Gu, Yunhe Gao, Yang Zhou, Mu Zhou, Dimitris Metaxas

机构 * Rutgers University（罗格斯大学） ； Stanford University（斯坦福大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to MICCAI 2025

Can Visual Encoder Learn to See Arrows?

Naoyuki Terashita, Yusuke Tozaki, Hideaki Omote, Congkha Nguyen, Ryosuke Nakamoto, Yuta Koreeda, Hiroaki Ozaki

机构 * Hitachi, Ltd.（日本日立株式会社） ； Kyoto Sangyo University（京都 Sangyo 大学） ； Gifu University（岐阜大学） 纠错

专题命中 VLM训练与架构 ：vision language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This work has been accepted for poster presentation at the Second Workshop on Visual Concepts in CVPR 2025

Galaxy Walker: Geometry-aware VLMs For Galaxy-scale Understanding

Tianyu Chen, Xingcheng Fu, Yisen Gao, Haodong Qian, Yuecen Wei, Kun Yan, Haoyi Zhou, Jianxin Li

机构 * SKLCCSE, School of Computer Science and Engineering, Beihang University, China（信息与通信工程学院，北京航空航天大学） ； School of Software, Beihang University, China（软件学院，北京航空航天大学） ； Key Lab of Education Blockchain and Intelligent Technology, Guangxi Normal University, China（教育区块链与智能技术重点实验室，广西师范大学） ； Institute of Artificial Intelligence, Beihang University, Beijing, China（人工智能研究院，北京航空航天大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR(Highlight)

Analyzing Fine-Grained Alignment and Enhancing Vision Understanding in Multimodal Language Models

Jiachen Jiang, Jinxin Zhou, Bo Peng, Xia Ning, Zhihui Zhu

机构 * Department of Computer Science and Engineering, The Ohio State University（计算机科学与工程系，俄亥俄州立大学） ； Translational Data Analytics Institute, The Ohio State University（转化数据分析研究所，俄亥俄州立大学） ； Department of Biomedical Informatics, The Ohio State University（生物医学信息学系，俄亥俄州立大学） 纠错

专题命中 VLM训练与架构 ：grounding(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

VCM: Vision Concept Modeling Based on Implicit Contrastive Learning with Vision-Language Instruction Fine-Tuning

Run Luo, Renke Shan, Longze Chen, Ziqiang Liu, Lu Wang, Min Yang, Xiaobo Xia

机构 * Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所） ； University of Chinese Academy of Sciences（中国科学院大学） ； National University of Singapore（新加坡国立大学） ； MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China（中国科学技术大学脑启发智能感知与认知教育部重点实验室） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments VCM

SuperEdit: Rectifying and Facilitating Supervision for Instruction-Based Image Editing

Ming Li, Xin Gu, Fan Chen, Xiaoying Xing, Longyin Wen, Chen Chen, Sijie Zhu

机构 * ByteDance Intelligent Creation (USA)（字节跳动智能创作（美国）） ； Center for Research in Computer Vision, University of Central Florida（计算机视觉研究中心，中央佛罗里达大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Code, Data and Models are available at: https://github.com/bytedance/SuperEdit

Expertized Caption Auto-Enhancement for Video-Text Retrieval

Baoyao Yang, Junxiang Chen, Wanyun Li, Wenbin Yao, Yang Zhou

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Filter Images First, Generate Instructions Later: Pre-Instruction Data Selection for Visual Instruction Tuning

Bardia Safaei, Faizan Siddiqui, Jiacong Xu, Vishal M. Patel, Shao-Yuan Lo

专题命中 VLM训练与架构 ：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at CVPR 2025 (Highlight)

F$^3$OCUS -- Federated Finetuning of Vision-Language Foundation Models with Optimal Client Layer Updating Strategy via Multi-objective Meta-Heuristics

Pramit Saha, Felix Wagner, Divyanshu Mishra, Can Peng, Anshul Thakur, David Clifton, Konstantinos Kamnitsas, J. Alison Noble

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted in CVPR 2025

Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuning

Tian Liu, Huixin Zhang, Shubham Parashar, Shu Kong

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to CVPR 2025. Website and code: https://tian1327.github.io/SWAT/

Visual Modality Prompt for Adapting Vision-Language Object Detectors

Heitor R. Medeiros, Atif Belal, Srikanth Muralidharan, Eric Granger, Marco Pedersoli

专题命中 VLM训练与架构 ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Can We Talk Models Into Seeing the World Differently?

Paul Gavrikov, Jovita Lukasik, Steffen Jung, Robert Geirhos, M. Jehanzeb Mirza, Margret Keuper, Janis Keuper

专题命中 VLM训练与架构 ：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at ICLR 2025

MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs

Sheng-Chieh Lin, Chankyu Lee, Mohammad Shoeybi, Jimmy Lin, Bryan Catanzaro, Wei Ping

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at ICLR 2025. We release the model weights at: https://huggingface.co/nvidia/MM-Embed

LLaRA: Supercharging Robot Learning Data for Vision-Language Policy

Xiang Li, Cristina Mata, Jongwoo Park, Kumara Kahatapitiya, Yoo Sung Jang, Jinghuan Shang, Kanchana Ranasinghe, Ryan Burgert, Mu Cai, Yong Jae Lee, Michael S. Ryoo

专题命中 VLM训练与架构 ：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICLR 2025

Explainable artificial intelligence (XAI): from inherent explainability to large language models

Fuseini Mumuni, Alhassan Mumuni

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

AI-Driven Early Mental Health Screening: Analyzing Selfies of Pregnant Women

Gustavo A. Basílio, Thiago B. Pereira, Alessandro L. Koerich, Hermano Tavares, Ludmila Dias, Maria das Graças da S. Teixeira, Rafael T. Sousa, Wilian H. Hisatugu, Amanda S. Mota, Anilton S. Garcia, Marco Aurélio K. Galletta, Thiago M. Paixão

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This article has been accepted for publication in HEALTHINF25 at the 18th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2025)

OneLLM: One Framework to Align All Modalities with Language

Jiaming Han, Kaixiong Gong, Yiyuan Zhang, Jiaqi Wang, Kaipeng Zhang, Dahua Lin, Yu Qiao, Peng Gao, Xiangyu Yue

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by CVPR 2024. Code: https://github.com/csuhan/OneLLM

Continual SFT Matches Multimodal RLHF with Negative Supervision

Ke Zhu, Yu Wang, Yanpeng Sun, Qiang Chen, Jiangjiang Liu, Gang Zhang, Jingdong Wang

专题命中 VLM训练与架构 ：vision-language model(abstract)；vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG