利用二维预训练先验提升零样本三维风格迁移
Boosting Zero-Shot 3D Style Transfer with 2D Pre-trained Priors
AI总结：
提出Data-Sufficient StyleGaussian模型，通过集成大规模2D图像数据集预训练的解码器，结合特征高斯溅射与延迟风格化，在数据稀缺条件下实现零样本3D风格迁移的高质量多视图一致渲染。
摘要：
在这项工作中，我们专注于零样本三维风格迁移，即给定任意风格图像，生成三维场景的多视图一致风格化视图。我们主要解决三维风格迁移中的数据稀缺问题，该问题源于每个模型仅在单个场景上训练，从而限制了可用内容图像的数量。这种稀缺性严重阻碍了风格化性能，因为模型优化依赖于足够数量的内容-风格图像对来提供监督信号。我们的核心思想是将在大规模二维图像数据集上预训练的解码器集成到三维风格迁移流程中，从而利用解码器从大量内容-风格图像对中学习到的先验知识。我们的方法结合了特征高斯溅射和延迟风格化，通过将视图相关操作统一为视图不变过程，在确保视图一致性的同时，利用数据充足的解码器网络实现高质量风格化。实验表明，我们的Data-Sufficient StyleGaussian（DS-StyleGaussian）模型在多个数据集上的视觉质量优于现有的零样本三维风格迁移方法。这项工作也表明，二维预训练可以作为三维任务的强增强手段，弥合二维与三维之间的数据差距。
英文摘要：
In this work, we focus on zero-shot 3D style transfer that can generate multi-view consistent stylized views of the 3D scene given an arbitrary style image. We primarily tackle the issue of data scarcity in 3D style transfer, which arises when each model is trained on only a single scene, thereby limiting the number of available content images. This scarcity significantly hampers stylization performance, as model optimization relies on a sufficient number of content-style image pairs to provide supervisory signals. Our core idea is to integrate a decoder pre-trained on large-scale 2D image datasets into the 3D style transfer pipeline, thereby leveraging the prior knowledge encoded in the decoder from learning over numerous content-style image pairs. Our method combines feature Gaussian splatting and deferred stylization, enabling high-quality stylization with the data-sufficient decoder network while ensuring view consistency by unifying view-dependent operations into a view-invariant process. Experiments demonstrate that our Data-Sufficient StyleGaussian (DS-StyleGaussian) model outperforms existing zero-shot 3D style transfer methods in terms of visual quality across various datasets. This work also suggests that 2D pre-training can serve as a strong enhancement for 3D tasks, bridging the data gap between 2D and 3D.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
