arXiv 2605.225362026-06-29cs.CVcs.CL

SpaceDG: 在视觉退化下评估空间智能的基准测试

SpaceDG: Benchmarking Spatial Intelligence under Visual Degradation

Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Electronic Science and Technology of China（电子科技大学）；Chongqing University（重庆大学）；The University of Tokyo（东京大学）；Beihang University（北航）；Northwestern Polytechnical University（西北工业大学）

Xiaolong Zhou, Yifei Liu, Ziyang Gong, Jiarui Li, Qiyue Zhao, Muyao Niu, Yuanyuan Gao, Le Ma, Xue Yang, Hongjie Zhang, Zhihang Zhong

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出SpaceDG，首个针对退化感知空间理解的大型数据集，通过物理基础的退化合成引擎生成9种退化类型，评估多模态大语言模型在视觉退化下的空间推理能力，并展示在退化条件下微调可提升模型鲁棒性。

摘要：

多模态大语言模型（MLLMs）在空间智能方面取得了快速进展，但现有空间推理基准大多假设纯净的视觉输入，忽略了现实部署中常见的退化现象，如运动模糊、低光照、恶劣天气、镜头畸变和压缩伪影。这提出了一个根本性问题：当前MLLMs在视觉观察不完美时的空间智能鲁棒性如何？为回答这个问题，我们引入SpaceDG，首个大规模退化感知空间理解数据集。它通过物理基础的退化合成引擎将退化形成过程嵌入3D高斯点散布（3DGS）渲染，能够真实模拟九种退化类型。所生成的数据集包含约100万对QA问题，来自近1000个室内场景。我们进一步引入SpaceDG-Bench，一个经人类验证的基准，包含11种推理类别和9种视觉退化类型的1102个问题，产生超过10000个VQA实例。评估25个开源和闭源MLLMs发现，视觉退化一致且显著损害空间推理能力，暴露出关键的鲁棒性差距。最后，我们展示在SpaceDG上微调可显著提高退化鲁棒性，并且在退化条件下甚至可以超越人类性能，而不会在清晰图像上造成性能下降，突显了退化感知训练在鲁棒空间智能方面的潜力。

英文摘要：

Multimodal Large Language Models (MLLMs) have made rapid progress in spatial intelligence, yet existing spatial reasoning benchmarks largely assume pristine visual inputs and overlook the degradations that commonly occur in real-world deployment, such as motion blur, low light, adverse weather, lens distortion, and compression artifacts. This raises a fundamental question: how robust is the spatial intelligence of current MLLMs when visual observations are imperfect? To answer this question, we introduce SpaceDG, the first large-scale dataset for degradation-aware spatial understanding. It is constructed with a physically grounded degradation synthesis engine that embeds degradation formation process into 3D Gaussian Splatting (3DGS) rendering, enabling realistic simulation of nine degradation types. The resulting dataset contains approximately 1M QA pairs from nearly 1,000 indoor scenes. We further introduce SpaceDG-Bench, an human-verified benchmark with 1,102 questions spanning 11 reasoning categories and 9 visual degradation types, yielding over 10K VQA instances. Evaluating 25 open- and closed-source MLLMs reveals that visual degradations consistently and substantially impair spatial reasoning, exposing a critical robustness gap. Finally, we show that finetuning on SpaceDG markedly improves degradation robustness and can even surpass human performance under degraded conditions without any performance drop on clean images, highlighting the promise of degradation-aware training for robust spatial intelligence.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。