重新思考3D噪声：通过无优化形态扰动学习3D感知视频先验

中文摘要 AI 辅助

NeRF和3D高斯溅射（3DGS）等3D场景表示在稀疏视图场景中会出现严重伪影。近期生成式3D伪影修复器尝试解决该问题，但依赖成对的损坏与干净渲染图，需要在不同视图配置下进行成本高昂的逐场景重建。2D图像增强可作为即时正则化器，但3D表示不存在显式等效项来保留跨视图的空间一致性，而这是3D感知训练的关键属性。我们提出3D形态扰动作为无优化正则化器，可保留空间一致性。利用显式3DGS，我们将每个高斯视为类似2D像素的基本构建块，并在其形态参数空间中通过缩放、旋转和剪枝应用扰动。我们的方法消除了数据集整理中逐场景3DGS优化循环，同时在轻量视频扩散沙箱上进行的诊断消融实验中，使模型比稀疏视图基线学习到更强的几何先验。通过ControlNet扩展到140亿参数的视频模型后，我们的方法在保持视觉保真度的同时，比最先进的图像到图像3D伪影修复器将平均深度误差降低了12.5%，最终在4个操纵任务中的3个任务上将下游机器人策略成功率提高了多达8.0%。

英文摘要

3D scene representations like NeRF and 3D Gaussian Splatting (3DGS) suffer severe artifacts in sparse-view settings. Recent generative 3D artifact fixers attempt to address this, but rely on paired corrupted and clean renders requiring costly, per-scene reconstructions across varying view configurations. While 2D image augmentations act as instant regularizers, no explicit equivalents exist for 3D representations to preserve spatial consistency across views, an essential property for 3D-aware training. We propose 3D Morphological Perturbations as an optimization-free regularizer that preserves spatial consistency. Leveraging explicit 3DGS, we treat each Gaussian as a fundamental building block - analogous to a 2D pixel - and apply perturbations across its morphological parameter space via scale, rotation, and pruning. Our method eliminates per-scene 3DGS optimization loops from dataset curation while enabling models to learn stronger geometric priors than sparse-view baselines in diagnostic ablations conducted on a lightweight video diffusion sandbox. Scaled to a 14B-parameter video model via ControlNet, our approach maintains visual fidelity while reducing mean depth error by 12.5% over state-of-the-art image-to-image 3D artifact refiners, ultimately boosting downstream robotics policy success rates by up to 8.0% across 3 of 4 manipulation tasks.

发表机构

• Technical University of Munich（慕尼黑工业大学）
• Huawei Heisenberg Research Center（华为海森堡研究中心）
• Tavus（塔沃斯公司）

机构由 AI 辅助整理，请以论文原文为准。