Gaussian-JEPA：面向3D高斯溅射的联合嵌入预测学习
Gaussian-JEPA: Joint-Embedding Predictive Learning for 3D Gaussian Splats
浏览论文内容
中文总结 AI 辅助
提出Gaussian-JEPA，通过预测高斯令牌块潜在表示实现自监督学习，在多类任务上优于重构预训练，为3D高斯表示提供有效学习目标
中文摘要 AI 辅助
3D高斯溅射（3DGS）通过各向异性基元表示3D内容，这些基元联合编码几何与外观信息。固定预算的编码器会处理高斯资产的采样观测，因此同一物体可能通过不同的基元实现被观测到。现有自监督方法主要对掩码后的高斯属性进行重构，将监督与某一采样实现绑定，且需要输入空间解码器。潜在预测提供了另一种方案，但其应用于高斯令牌时，需要能适配耦合属性与异质空间支撑的目标。我们提出Gaussian-JEPA，它从可见上下文预测未观测到的高斯令牌块的表示。在线编码器处理上下文，而共享的指数移动平均编码器提供停止梯度特征作为多尺度目标。互补的目标投影与特征空间接地提供潜在监督，无需重构高斯属性。我们在高斯重采样、部分观测和可渲染形状补全场景下评估这些特征，同时评估其在部件分割和物体分类任务上的迁移性能。与匹配的重构预训练方法相比，Gaussian-JEPA在重采样输入下表现更一致，在部分观测下保留更多实例信息，且为高斯补全提供更强的冻结特征。这些结果支持潜在预测作为可复用3D高斯表示的有效目标。代码可在项目页面获取：https://this.url
英文摘要
3D Gaussian Splatting (3DGS) represents 3D content with anisotropic primitives that jointly encode geometry and appearance. Fixed-budget encoders consume sampled observations of Gaussian assets, so the same object may be observed through different primitive realizations. Existing self-supervised methods mainly reconstruct masked Gaussian attributes, tying supervision to one sampled realization and requiring an input-space decoder. Latent prediction offers an alternative, but its application to Gaussian tokens requires targets that accommodate coupled attributes and heterogeneous spatial support. We introduce Gaussian-JEPA, which predicts representations of held-out Gaussian token blocks from visible context. An online encoder processes the context, while a shared exponential-moving-average encoder supplies stop-gradient features for multi-scale targets. Complementary target projections and feature-space grounding provide latent supervision without reconstructing Gaussian attributes. We evaluate the features under Gaussian resampling, partial observations, and renderable shape completion, together with transfer to part segmentation and object classification. Compared with matched reconstruction pretraining, Gaussian-JEPA is more consistent across resampled inputs, retains more instance information under partial observations, and provides stronger frozen features for Gaussian completion. These results support latent prediction as an effective objective for reusable 3D Gaussian representations. Code is on the project page (https://amazingren.github.io/Gaussian-JEPA/).
