视觉大模型 / VLM

Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs)

Leander Girrbach, Stephan Alaniz, Yiran Huang, Trevor Darrell, Zeynep Akata

专题命中 VLM训练与架构 ：LLaVA(abstract)；InternVL(abstract)

Comments Accepted at ICLR 2025

COAP: Memory-Efficient Training with Correlation-Aware Gradient Projection

Jinqi Xiao, Shen Sang, Tiancheng Zhi, Jing Liu, Qing Yan, Yuqian Zhang, Linjie Luo, Bo Yuan

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR 2025

Trinity: A Modular Humanoid Robot AI System

Jingkai Sun, Qiang Zhang, Gang Han, Wen Zhao, Zhe Yong, Yan He, Jiaxu Wang, Jiahang Cao, Yijie Guo, Renjing Xu

专题命中 VLM训练与架构 ：VLM(abstract)；visual language model(abstract)

LVPruning: An Effective yet Simple Language-Guided Vision Token Pruning Approach for Multi-modal Large Language Models

Yizheng Sun, Yanze Xin, Hao Li, Jingyuan Sun, Chenghua Lin, Riza Batista-Navarro

专题命中 VLM训练与架构 ：LLaVA(abstract)；MLLM(abstract)

Comments Accepted to NAACL 2025 Findings

Can Large Language Models Grasp Concepts in Visual Content? A Case Study on YouTube Shorts about Depression

Jiaying "Lizzy" Liu, Yiheng Su, Praneel Seth

专题命中 VLM训练与架构 ：LLaVA(abstract)；MLLM(abstract)

Comments 11 pages

Journal ref CHI Conference on Human Factors in Computing Systems (CHI EA 2025)

Zero-Shot Defense Against Toxic Images via Inherent Multimodal Alignment in LVLMs

Wei Zhao, Zhe Li, Yige Li, Jun Sun

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Training-free Diffusion Model Alignment with Sampling Demons

Po-Hung Yeh, Kuang-Huei Lee, Jun-Cheng Chen

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 35 pages

Journal ref Proceedings of the Thirteenth International Conference on Learning Representations (ICLR 2025)

NOTA: Multimodal Music Notation Understanding for Visual Large Language Model

Mingni Tang, Jiajia Li, Lu Yang, Zhiqiang Zhang, Jinghao Tian, Zuchao Li, Lefei Zhang, Ping Wang

专题命中 VLM训练与架构 ：visual language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Routing Experts: Learning to Route Dynamic Experts in Multi-modal Large Language Models

Qiong Wu, Zhaoxi Ke, Yiyi Zhou, Xiaoshuai Sun, Rongrong Ji

专题命中 VLM训练与架构 ：LLaVA(abstract)；MLLM(abstract)

EnVisionVR: A Scene Interpretation Tool for Visual Accessibility in Virtual Reality

Junlong Chen, Rosella P. Galindo Esparza, Vanja Garaj, Per Ola Kristensson, John Dudley

专题命中 VLM训练与架构 ：vision language model(abstract)；VLM(abstract)

Comments The online appendix is available at https://osf.io/zb2ak/

OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints

Mingjie Pan, Jiyao Zhang, Tianshu Wu, Yinghao Zhao, Wenlong Gao, Hao Dong

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data

Shuhao Gu, Jialing Zhang, Siyuan Zhou, Kevin Yu, Zhaohu Xing, Liangdong Wang, Zhou Cao, Jintao Jia, Zhuoyi Zhang, Yixuan Wang, Zhenchong Hu, Bo-Wen Zhang, Jijie Li, Dong Liang, Yingli Zhao, Songjing Wang, Yulong Ao, Yiming Ju, Huanhuan Ma, Xiaotong Li, Haiwen Diao, Yufeng Cui, Xinlong Wang, Yaoqi Liu, Fangxiang Feng, Guang Liu

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

RealMind: Advancing Visual Decoding and Language Interaction via EEG Signals

Dongyang Li, Haoyang Qin, Mingyang Wu, Jiahua Tang, Yuang Cao, Chen Wei, Quanying Liu

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)

MMFactory: A Universal Solution Search Engine for Vision-Language Tasks

Wan-Cyuan Fan, Tanzila Rahman, Leonid Sigal

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Advancing Comprehensive Aesthetic Insight with Multi-Scale Text-Guided Self-Supervised Learning

Yuti Liu, Shice Liu, Junyuan Gao, Pengtao Jiang, Hao Zhang, Jinwei Chen, Bo Li

专题命中 VLM训练与架构 ：MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by AAAI 2025

Visual Lexicon: Rich Image Features in Language Space

XuDong Wang, Xingyi Zhou, Alireza Fathi, Trevor Darrell, Cordelia Schmid

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Tech report. 16 pages, 10 figures

Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora

Michael Y. Hu, Aaron Mueller, Candace Ross, Adina Williams, Tal Linzen, Chengxu Zhuang, Ryan Cotterell, Leshem Choshen, Alex Warstadt, Ethan Gotlieb Wilcox

专题命中 VLM训练与架构 ：vision language model(abstract)；grounding(abstract)

BYE: Build Your Encoder with One Sequence of Exploration Data for Long-Term Dynamic Scene Understanding

Chenguang Huang, Shengchao Yan, Wolfram Burgard

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Beyond Sight: Towards Cognitive Alignment in LVLM via Enriched Visual Knowledge

Yaqi Zhao, Yuanyang Yin, Lin Li, Mingan Lin, Victor Shea-Jay Huang, Siwei Chen, Weipeng Chen, Baoqun Yin, Zenan Zhou, Wentao Zhang

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

GCI-ViTAL: Gradual Confidence Improvement with Vision Transformers for Active Learning on Label Noise

Moseli Mots'oehli, kyungim Baek

专题命中 VLM训练与架构 ：grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments under review

Analyzing The Language of Visual Tokens

David M. Chan, Rodolfo Corona, Joonyong Park, Cheol Jun Cho, Yutong Bai, Trevor Darrell

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Aligning Audio-Visual Joint Representations with an Agentic Workflow

Shentong Mo, Yibing Song

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

GenRL: Multimodal-foundation world models for generalization in embodied agents

Pietro Mazzaglia, Tim Verbelen, Bart Dhoedt, Aaron Courville, Sai Rajeswar

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Presented at NeurIPS 2024

MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs

Ziyu Liu, Tao Chu, Yuhang Zang, Xilin Wei, Xiaoyi Dong, Pan Zhang, Zijian Liang, Yuanjun Xiong, Yu Qiao, Dahua Lin, Jiaqi Wang

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This project is available at https://github.com/Liuziyu77/MMDU

CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning

Emanuele Frascaroli, Aniello Panariello, Pietro Buzzega, Lorenzo Bonicelli, Angelo Porrello, Simone Calderara

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 15 pages, 1 figure. Accepted as ORAL at the The 35th British Machine Vision Conference 2024 (BMVC 2024), Glasgow, UK

Bootstrap3D: Improving Multi-view Diffusion Model with Synthetic Data

Zeyi Sun, Tong Wu, Pan Zhang, Yuhang Zang, Xiaoyi Dong, Yuanjun Xiong, Dahua Lin, Jiaqi Wang

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Project Page: https://sunzey.github.io/Bootstrap3D/

PRE: Vision-Language Prompt Learning with Reparameterization Encoder

Thi Minh Anh Pham, An Duc Nguyen, Cephas Svosve, Vasileios Argyriou, Georgios Tzimiropoulos

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 10 pages excluding References and Appendix

Parameter-Efficient Quantized Mixture-of-Experts Meets Vision-Language Instruction Tuning for Semiconductor Electron Micrograph Analysis

Sakhinana Sagar Srinivas, Chidaksh Ravuru, Geethan Sannidhi, Venkataramana Runkana

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Paper published at ICML 2024 Workshop on Foundation Models in the Wild

mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models

Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, Jingren Zhou

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Matryoshka Multimodal Models

Mu Cai, Jianwei Yang, Jianfeng Gao, Yong Jae Lee

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Project Page: https://matryoshka-mm.github.io/