Vision-Language Models (VLMs) are known to struggle with spatial reasoning and visual alignment. To help overcome these limitations, we introduce iVISPAR, an interactive multimodal benchmark designed to evaluate the spatial reasoning capabilities of VLMs acting as agents. \mbox{iVISPAR} is based on a variant of the sliding tile puzzle, a classic problem that demands logical planning, spatial awareness, and multi-step reasoning. The benchmark supports visual 3D, 2D, and text-based input modalities, enabling comprehensive assessments of VLMs' planning and reasoning skills. We evaluate a broad suite of state-of-the-art open-source and closed-source VLMs, comparing their performance while also providing optimal path solutions and a human baseline to assess the task's complexity and feasibility for humans. Results indicate that while VLMs perform better on 2D tasks compared to 3D or text-based settings, they struggle with complex spatial configurations and consistently fall short of human performance, illustrating the persistent challenge of visual alignment. This underscores critical gaps in current VLM capabilities, highlighting their limitations in achieving human-level cognition. Project website: https://microcosm.ai/ivispar
Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Wuhan AI Research(武汉人工智能研究院)
VLMs Guided Interpretable Decision Making for Autonomous Driving
Department of Computer Science, Tulane University(路易斯安那大学计算机科学系);Qualcomm(高通公司);Department of Industrial and Systems Engineering, North Carolina State University(北卡罗来纳州立大学工业与系统工程系)
UCSF-PDGM-VQA: Visual Question Answering dataset for brain tumor MRI interpretation
Fung Institute for Engineering Leadership(工程领导力基金会);University of California, Berkeley(加州大学伯克利分校);Department of Radiology(放射科);University of California, San Francisco(加州大学旧金山分校);Division of Clinical Informatics and Digital Transformation(临床信息学与数字转型部);Department of Neurological Surgery(神经外科部)
Look-Closer-Then-Diagnose: Confidence-Aware Ultrasound VQA via Active Zooming
Computer Aided Medical Procedures (CAMP)(计算机辅助医疗程序);TU Munich, Germany(慕尼黑工业大学,德国);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Munich, Germany(慕尼黑,德国);Zhongshan Hospital, Fudan University, China(复旦大学中山医院);The University of Hong Kong, Hongkong, China(香港大学,香港,中国)
Evaluation of Safety Cognition Capability in Vision-Language Models for Autonomous Driving
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室)
From Perception to Action: An Interactive Benchmark for Vision Reasoning
Singapore University of Technology(新加坡科技设计大学);Singapore Management University (SMU), Singapore(新加坡管理学院);Nanyang Technological University (NTU), Singapore(南洋理工大学);University of Science(科学大学)
SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Zhongguancun Academy(中关村学院);Tsinghua University(清华大学);Helsinki University(赫尔辛基大学)
SkinGEN: an Explainable Dermatology Diagnosis-to-Generation Framework with Interactive Vision-Language Models
Zhejiang University(浙江大学);Binjiang Institute of Zhejiang University(浙江大学滨江研究院);The First Affiliated Hospital, Zhejiang University School of Medicine(浙江大学医学院附属第一医院);Shanghai Institute for Advanced Study of Zhejiang University(浙江大学上海高等研究院);Hunan University(湖南大学)
EO-Gym: A Multimodal, Interactive Environment for Earth Observation Agents
The Australian National University(澳大利亚国立大学);Royal Melbourne Institute of Technology(皇家墨尔本理工学院);University of Sydney(悉尼大学);The Hong Kong University of Science and Technology(香港科学与技术大学)
JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Brown University(布朗大学);University of Michigan Ann Arbor(密歇根大学安娜堡分校);Hong Kong University of Science and Technology(香港科学与技术大学);Carnegie Mellon University(卡内基梅隆大学);Boise State University(博伊西州立大学)
MedVL-SAM2: A unified 3D medical vision-language model for multimodal reasoning and prompt-driven segmentation
Department of Biomedical Engineering, University of Florida(佛罗里达大学生物医学工程系);Department of Radiology, University of Florida(佛罗里达大学放射学系);Research Computing, University of Florida(佛罗里达大学研究计算中心);Department of Medicine, University of Florida(佛罗里达大学医学系);Department of Radiology, UC San Francisco(旧金山大学放射学系)