Benchmarking spatial reasoning in multimodal large language models (MLLMs) has attracted growing interest in computer vision due to its importance for embodied AI and other agentic systems that require precise interaction with the physical world. However, spatial reasoning on Earth imagery has lagged behind, as it uniquely involves grounding objects in georeferenced images and quantitatively reasoning about distances, directions, and topological relations using both visual cues and vector geometry coordinates (e.g., 2D bounding boxes, polylines, and polygons). Existing benchmarks for Earth imagery primarily focus on 2D spatial grounding, image captioning, and coarse spatial relations (e.g., simple directional or proximity cues). They lack support for quantitative direction and distance reasoning, systematic topological relations, and complex object geometries beyond bounding boxes. To fill this gap, we propose \textbf{EarthSpatialBench}, a comprehensive benchmark for evaluating spatial reasoning in MLLMs on Earth imagery. The benchmark contains over 325K question-answer pairs spanning: (1) qualitative and quantitative reasoning about spatial distance and direction; (2) systematic topological relations; (3) single-object queries, object-pair queries, and compositional aggregate group queries; and (4) object references expressed via textual descriptions, visual overlays, and explicit geometry coordinates, including 2D bounding boxes, polylines, and polygons. We conducted extensive experiments on both open-source and proprietary models to identify limitations in the spatial reasoning of MLLMs.
Beyond RGB: Benchmarking and Enhancing MLLMs for Hyperspectral Image Understanding via Training-Free Reasoning Framework
Sun Yat-sen University（中山大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；China Agricultural University（中国农业大学）；Southwest Jiaotong University（西南交通大学）；Southwest University（西南大学）；National Supercomputing Center in Shenzhen（国家超级计算深圳中心）
OpenMined；Manifold Computing Group（曼尼福德计算集团）；Ford R&A（福特研发部门）；Harvard University（哈佛大学）；Indian Institute of Technology Jodhpur（印度理工学院焦特布尔分校）；University of Oxford（牛津大学）
In this paper we measure the effectiveness of $ε$-Differential Privacy (DP) when applied to medical imaging. We compare two robust differential privacy mechanisms: Local-DP and DP-SGD and benchmark their performance when analyzing medical imagery records. We analyze the trade-off between the model's accuracy and the level of privacy it guarantees, and also take a closer look to evaluate how useful these theoretical privacy guarantees actually prove to be in the real world medical setting.
MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education
AI Singapore, National University of Singapore（新加坡国立大学新加坡人工智能）；School of Computing, National University of Singapore（新加坡国立大学计算学院）；Institute of Advanced Intelligence and Computing, A*STAR（新加坡科技研究局先进智能与计算研究所）
Multimodal Mathematical Reasoning Embedded in Aerial Vehicle Imagery: Benchmarking, Analysis, and Exploration
Department of Physics, J.K. Institute of Science（J.K.科学研究院物理系）；World Scientific University（世界科学大学）；University of Intelligent Studies（智能研究大学）；East China Normal University（华东师范大学）；Nanyang Technological University（南洋理工大学）；SenseTime Research（商汤科技研究院）；Shanghai Jiao Tong University（上海交通大学）；Harbin Institute of Technology（哈尔滨工业大学）
GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University College London（伦敦大学学院）；Linköping University（林雪平大学）；IBM Research Europe（IBM欧洲研究院）；ServiceNow Research（ServiceNow研究院）；Australian National University（澳大利亚国立大学）
UrbanWell: Benchmarking Multimodal Large Language Models for Spatio-Temporal Urban Wellbeing Analytics
University of Helsinki（赫尔辛基大学）；Zhongguancun Academy（中关村学院）；University of Oxford（牛津大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；George Mason University（乔治梅森大学）
We present a new public dataset with a focus on simulating robotic vision tasks in everyday indoor environments using real imagery. The dataset includes 20,000+ RGB-D images and 50,000+ 2D bounding boxes of object instances densely captured in 9 unique scenes. We train a fast object category detector for instance detection on our data. Using the dataset we show that, although increasingly accurate and fast, the state of the art for object detection is still severely impacted by object scale, occlusion, and viewing direction all of which matter for robotics applications. We next validate the da…
TU Delft（代尔夫特理工大学）；TU Eindhoven（埃因霍温理工大学）；Cyclomedia B.V（Cyclomedia公司）
The current paradigm in privacy protection in street-view images is to detect and blur sensitive information. In this paper, we propose a framework that is an alternative to blurring, which automatically removes and inpaints moving objects (e.g. pedestrians, vehicles) in street-view imagery. We propose a novel moving object segmentation algorithm exploiting consistencies in depth across multiple street-view images that are later combined with the results of a segmentation network. The detected moving objects are removed and inpainted with information from other views, to obtain a realistic out…
WiSE-OD: Benchmarking Robustness in Infrared Object Detection
Laboratoire d’imagerie, de vision et d’intelligence artificielle (LIVIA)（图像、视觉与人工智能实验室）；International Laboratory on Learning Systems (ILLS)（学习系统国际实验室）；Dept. of Systems Engineering, ETS Montreal, Canada（系统工程系，蒙特利尔大学）
Benchmarking CNN and Transformer-Based Object Detectors for UAV Solar Panel Inspection
Faculty of Engineering, University of Moratuwa（摩图瓦大学工程学院）；STEM College, School of Computing Technologies, RMIT University（皇家墨尔本理工学院计算机技术学院）；School of Engineering, University of Southern Queensland（南方昆士兰大学工程学院）
Benchmarking Composed Image Retrieval for Applied Earth Observation
organization= Visual Recognition Group, Department of Cybernetics, Czech Technical University in Prague , country= Czechia；organization= Remote Sensing Laboratory, School of Rural, Surveying；Geoinformatics Engineering, National Technical University of Athens , country= Greece；organization= Institute of Informatics \& Telecommunications, National Centre for Scientific Research ``Demokritos'' , country= Greece；organization= Department of Informatics；Kapodistrian University of Athens , country= Greece
