Vision-language models (VLMs) achieve strong performance on standard, high-quality datasets, but we still do not fully understand how they perform under real-world image distortions. We present VLM-RobustBench, a benchmark spanning 49 augmentation types across noise, blur, weather, digital, and geometric perturbations, evaluated under graded severities (low/mid/high) and binary transforms, yielding 133 corrupted settings. We evaluate VLMs from four families (Qwen, InternVL, Molmo, Gemma) on two complementary benchmarks: MMBench (visually grounded) and MMMU-Pro (reasoning-oriented). Our results reveal that visual severity is a weak predictor of difficulty: low-severity spatial perturbations often degrade performance more than visually severe photometric corruptions. In particular, low-severity glass_blur reduces MMBench accuracy by about 8 pp on average across models, while the largest drops arise from resampling and geometric distortions (e.g., upsample, elastic_transform), reaching up to 34 pp. Overall, our findings suggest current VLMs are semantically strong but spatially fragile, motivating the definition of novel robustness evaluation protocols and training regimes that emphasize resampling and geometric invariances.
Your Vision-Language Model Can't Even Count to 20: Exposing the Failures of VLMs in Compositional Counting
Guilin University of Electronic Technology(桂林电子科技大学);The Ohio State University(俄亥俄州立大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);University of California, Berkeley(加州大学伯克利分校)
GeoWorld-VLM: Geometry from World Models for Vision-Language Models
Harvard AI and Robotics Lab(哈佛人工智能与机器人实验室);Kempner Institute for the Study of Natural and Artificial Intelligence(凯普纳自然与人工智能研究 institute);Harvard University(哈佛大学)
RE-VLM: Event-Augmented Vision-Language Model for Scene Understanding
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,BIGAI)
LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMs
Foshan Graduate School of Innovation, Northeastern University(创新研究生院,东北大学);Faculty of Robot Science and Engineering, Northeastern University(机器人科学与工程学院,东北大学);School of Aeronautic Science and Engineering, Beihang University(航空科学与工程学院,北航);QingniaoAI, China(清北AI,中国)
VLM-PAR: A Vision Language Model for Pedestrian Attribute Recognition
Department of Innovation Engineering(创新工程系);University of Salento, Italy(意大利萨伦托大学);Institute of Applied Sciences and Intelligent Systems - CNR(应用科学与智能系统研究所 - CNR);University of the Basque Country UPV/EHU(巴斯克国家大学UPV/EHU);IKERBASQUE, Basque Foundation for Science(伊基塔斯克巴塞克基金会);Sorbonne University Abu Dhabi(索邦大学阿布扎比分校)
VLM Can Be a Good Assistant: Enhancing Embodied Visual Tracking with Self-Improving Vision-Language Models
State Key Laboratory of Complex & Critical Software Environment, Beihang University(北京航空航天大学复杂与关键软件环境国家重点实验室);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院);City University of Macau(澳门城市大学);Center on Frontiers of Computing Studies, School of Computer Science, Nat’l Eng. Research Center of Visual Technology, Peking University(北京大学计算机学院前沿计算研究中心、国家工程视觉技术研究中心)
Geometric Similarity in VLM Low-Level Vision Representations
Duke University(杜克大学);Texas A&M University(德克萨斯A&M大学);Florida State University(佛罗里达州立大学);UC San Diego(加州大学圣迭戈分校);University of Pennsylvania(宾夕法尼亚大学);UNC Chapel Hill(北卡罗来纳大学教堂山分校)
CHIRP: A Fine-Grained Benchmark for Open-Ended Response Evaluation in Vision-Language Models
Mila - Quebec AI Institute(魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);Tokyo Institute of Technology(东京技术大学);McGill University(麦吉尔大学);EleutherAI;University College London(伦敦大学学院)
EmbodiedMidtrain: Bridging the Gap between Vision-Language Models and Vision-Language-Action Models via Mid-training
Language Technologies Institute, Carnegie Mellon University(卡内基梅隆大学语言技术研究所);Bosch Research North America & Bosch Center for Artificial Intelligence (BCAI)(博世北美研究部及博世人工智能中心(BCAI))
Promoting AI Equity in Science: Generalized Domain Prompt Learning for Accessible VLM Research
Shanghai Jiao Tong University(上海交通大学);Eastern Institute of Technology(东方理工大学);Yale University(耶鲁大学);Renmin University of China(中国人民大学);Southern University of Science and Technology(南方科技大学);Ningbo Institute of Digital Twin(宁波数字孪生研究院)