从未标注的多视角图像中学习3D表示以提升空间智能
Learning 3D Representations for Spatial Intelligence from Unposed Multi-View Images
AI总结：
本文提出UniSplat框架，通过双掩码策略、高斯溅射策略和姿态条件重校准机制，解决未标注多视角图像中3D表示学习的几何诱导弱、外观细节不足和几何语义不一致问题，实现鲁棒且通用的3D表示。
摘要：
鲁棒的3D表示学习是空间智能的感知基础，使场景理解和具身AI等下游任务成为可能。然而，直接从未标注的多视角图像中学习此类表示仍然具有挑战性。最近的自监督方法试图通过前馈方式统一几何、外观和语义，但常常面临几何诱导弱、外观细节有限以及几何与语义不一致的问题。我们引入UniSplat，一种前馈框架，通过三个互补组件解决这些问题。首先，我们提出双掩码策略，通过掩码编码器和解码器的token，并将解码器掩码定向于几何丰富的区域，迫使模型从不完整的视觉提示中推断结构信息，从而在未标注输入下获得几何感知的表示。其次，我们开发了粗到细的高斯溅射策略，通过逐步细化辐射场来减少外观-语义不一致。最后，为了强制几何-语义一致性，我们引入了姿态条件重校准机制，通过将预测的3D点和语义地图重新投影到图像平面并使用估计的相机参数对齐，确保跨任务的一致性，从而解决几何-语义不匹配问题。这些组件共同产生了鲁棒且通用的3D表示，能够处理未标注、稀疏视角输入，并在多样化的任务中泛化，为空间智能奠定了感知基础。
英文摘要：
Robust 3D representation learning forms the perceptual foundation of spatial intelligence, enabling downstream tasks in scene understanding and embodied AI. However, learning such representations directly from unposed multi-view images remains challenging. Recent self-supervised methods attempt to unify geometry, appearance, and semantics in a feed-forward manner, but they often suffer from weak geometry induction, limited appearance detail, and inconsistencies between geometry and semantics. We introduce UniSplat, a feed-forward framework designed to address these limitations through three complementary components. First, we propose a dual-masking strategy that strengthens geometry induction in the encoder. By masking both encoder and decoder tokens, and targeting decoder masks toward geometry-rich regions, the model is forced to infer structural information from incomplete visual cues, yielding geometry-aware representations even under unposed inputs. Second, we develop a coarse-to-fine Gaussian splatting strategy that reduces appearance-semantics inconsistencies by progressively refining the radiance field. Finally, to enforce geometric-semantic consistency, we introduce a pose-conditioned recalibration mechanism that interrelates the outputs of multiple heads by re-projecting predicted 3D point and semantic maps into the image plane using estimated camera parameters, and aligning them with corresponding RGB and semantic predictions to ensure cross-task consistency, thereby resolving geometry-semantic mismatches. Together, these components yield unified 3D representations that are robust to unposed, sparse-view inputs and generalize across diverse tasks, laying a perceptual foundation for spatial intelligence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
