SSR：通过理由引导的空间推理增强视觉-语言模型中的深度感知
SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning
AI总结：
提出SSR框架，将深度数据转为文本理由以增强视觉-语言模型的空间推理，并利用知识蒸馏实现高效集成，通过新数据集和基准验证了其有效性。
摘要：
尽管视觉-语言模型（VLMs）在多模态任务中取得了令人瞩目的进展，但它们对RGB输入的依赖限制了精确的空间理解。现有的整合空间线索（如点云或深度）的方法，要么需要专用传感器，要么无法有效利用深度信息进行高阶推理。为此，我们提出了一种新颖的空间感知与推理方法，称为SSR，这是一个将原始深度数据转换为结构化、可解释的文本理由的新框架。这些文本理由作为有意义的中间表示，显著增强了空间推理能力。此外，我们利用知识蒸馏将生成的理由压缩为紧凑的潜在嵌入，这有助于资源高效且即插即用地集成到现有VLMs中，而无需重新训练。为了进行全面评估，我们引入了一个名为SSR-CoT的新数据集，这是一个百万规模的视觉-语言推理数据集，带有中间空间推理注释，并提出了SSRBench，一个全面的多任务基准。在多个基准上的大量实验表明，SSR显著提高了深度利用率和空间推理能力，从而推动VLMs向更类人的多模态理解发展。项目页面：https://yliu-cs.github.io/SSR。
英文摘要：
Despite impressive advancements in Visual-Language Models (VLMs) for multi-modal tasks, their reliance on RGB inputs limits precise spatial understanding. Existing methods for integrating spatial cues, such as point clouds or depth, either require specialized sensors or fail to effectively exploit depth information for higher-order reasoning. To this end, we propose a novel Spatial Sense and Reasoning method, dubbed SSR, a novel framework that transforms raw depth data into structured, interpretable textual rationales. These textual rationales serve as meaningful intermediate representations to significantly enhance spatial reasoning capabilities. Additionally, we leverage knowledge distillation to compress the generated rationales into compact latent embeddings, which facilitate resource-efficient and plug-and-play integration into existing VLMs without retraining. To enable comprehensive evaluation, we introduce a new dataset named SSR-CoT, a million-scale visual-language reasoning dataset enriched with intermediate spatial reasoning annotations, and present SSRBench, a comprehensive multi-task benchmark. Extensive experiments on multiple benchmarks demonstrate SSR substantially improves depth utilization and enhances spatial reasoning, thereby advancing VLMs toward more human-like multi-modal understanding. Project page: https://yliu-cs.github.io/SSR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
