几何感知的测试时学习用于定量空间推理

中文摘要 AI 辅助

视觉-语言模型（VLMs）中的定量空间推理旨在从二维图像和自然语言查询中推断三维空间中物体间的空间距离和方向关系。尽管近期取得了进展，但在分布偏移下，VLM的空间推理仍然脆弱，这主要归因于三维监督的高成本。因此，当面对新颖的物体配置或改写后的空间查询时，模型常产生不一致或矛盾的预测，揭示了学习表示与底层几何之间的错位。为解决此问题，我们提出TTL-SR，一种用于定量空间推理的几何感知测试时学习框架，利用几何一致性约束和未标注的测试数据使模型适应目标域。具体而言，TTL-SR用几何耦合的辅助查询增强输入查询，通过自适应几何触发过滤不可靠预测以构建结构化的token级伪标签，并仅使用测试数据在几何感知的多目标损失下更新模型参数。实验结果表明，TTL-SR显著提升了空间推理性能，在Q-Spatial-ScanNet数据集上，对Qwen3-VL-4B-Instruct和SpatialRGPT-VILA-1.5-8B分别带来了6.47%和9.41%的准确率提升。

英文摘要

Quantitative spatial reasoning in visual-language models (VLMs) aims to infer spatial distances and directional relationships among objects in 3D space from a 2D image and a natural language query. Despite recent progress, VLM spatial reasoning remains brittle under distribution shifts, largely due to the high cost of 3D supervision. As a result, models often produce inconsistent or contradictory predictions when faced with novel object configurations or rephrased spatial queries, revealing a misalignment between learned representations and underlying geometry. To address this, we propose TTL-SR, a geometry-aware Test-Time Learning framework for quantitative Spatial Reasoning that leverages geometric consistency constraints and unlabeled test data to adapt models to target domains. Specifically, TTL-SR augments the input query with geometrically coupled auxiliary queries, filters unreliable predictions via adaptive geometric triggering to construct structured token-level pseudo-labels, and updates model parameters under a geometry-aware multi-objective loss using only test data. Experimental results demonstrate that TTL-SR significantly boosts spatial reasoning performance, yielding 6.47% and 9.41% accuracy gains for Qwen3-VL-4B-Instruct and SpatialRGPT-VILA-1.5-8B on Q-Spatial-ScanNet dataset, respectively.

发表机构

• South China University of Technology（华南理工大学）
• Nanyang Technological University（南洋理工大学）
• Guangdong University of Technology（广东工业大学）

机构由 AI 辅助整理，请以论文原文为准。

补充信息

备注

Accepted by ACM MM 2026