隐私保护下基于深度图的开放词汇3D语义分割：不确定性引导的测试时优化

中文摘要 AI 辅助

隐私保护感知是在真实室内环境中部署3D场景理解系统的关键要求，但在开放词汇3D语义分割中仍未得到充分探索。现有方法通常依赖于从RGB图像中获取丰富的语义线索，这可能会暴露隐私敏感的视觉信息。仅基于深度图的3D几何提供了一种隐私保护的替代方案，但缺乏基于外观的语义线索使得开放词汇预测高度不确定且不可靠。在此设置下，我们提出将不确定性转化为引导信号，以识别不可靠的语义响应，并利用基础模型的语义先验来规范其细化。我们提出了UTTO，一种用于仅深度开放词汇3D语义分割的不确定性引导测试时优化框架。无需额外训练，在ScanNet20、ScanNet40和ScanNet200上的实验表明，UTTO持续改进了仅深度开放词汇3D分割，并在隐私保护条件下优于代表性基线。

英文摘要

Privacy-preserving perception is increasingly important for robotic systems operating in real-world indoor environments, yet it remains underexplored in open-vocabulary 3D semantic segmentation. We study this problem under an RGB-prohibited deployment setting motivated by scene-specific visual information disclosure, where real RGB observations are unavailable during scene acquisition and fusion. To reflect this deployment constraint on existing 3D datasets, we adopt a stricter depth-only evaluation protocol that re-runs scene fusion without RGB and exposes only the resulting depth-derived geometry to the segmentation pipeline. This constraint removes appearance cues that are often critical for open-vocabulary recognition, making depth-only predictions more uncertain and less reliable. To address this challenge, we propose UTTO, a model-agnostic uncertainty-guided test-time optimization framework that uses structured predictive uncertainty as a reliability signal to refine predictions from frozen open-vocabulary 3D backbones. Experiments across ScanNet and Matterport3D demonstrate consistent improvements over multiple depth-only backbones. Privacy recoverability analyses and a real-robot semantic goal grounding case study further support the proposed privacy-constrained setting and applicability.

发表机构

• Humanoid Robots Lab, University of Bonn（波恩大学仿人机器人实验室）
• Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习和人工智能研究所）
• Center for Robotics, Bonn, Germany（波恩机器人中心）

机构由 AI 辅助整理，请以论文原文为准。