WaSt-3D:基于3D高斯的场景到场景风格化的Wasserstein-2距离
WaSt-3D: Wasserstein-2 Distance for Scene-to-Scene Stylization on 3D Gaussians
AI总结:
本文提出WaSt-3D,利用显式高斯泼溅表示和熵正则化的Wasserstein-2距离直接匹配内容与风格场景间的高斯分布,将风格化重构为显式分布匹配,无需训练即可实现高分辨率3D场景风格化。
摘要:
尽管风格迁移技术在2D图像风格化方面已发展成熟,但这些方法向3D场景的扩展仍相对未被探索。现有方法在迁移颜色和纹理方面表现出色,但在复制场景几何结构时往往存在困难。在我们的工作中,我们利用显式高斯泼溅(GS)表示,并使用推土机距离(EMD)直接匹配风格场景与内容场景之间的高斯分布。通过采用熵正则化的Wasserstein-2距离,我们确保变换保持空间平滑性。此外,我们将场景风格化问题分解为更小的块以提高效率。这种范式转换将风格化从由潜在空间损失驱动的纯生成过程,重新构建为两个高斯表示之间的显式分布匹配。我们的方法通过将3D风格场景的细节忠实地迁移到内容场景上,实现了高分辨率3D风格化。此外,WaSt-3D在多样的内容和风格场景中始终如一地提供结果,而无需任何训练,因为它仅依赖于基于优化的技术。请参阅我们的项目页面以获取更多结果和源代码:$\href{https://compvis.github.io/wast3d/}{https://compvis.github.io/wast3d/}$。
英文摘要:
While style transfer techniques have been well-developed for 2D image stylization, the extension of these methods to 3D scenes remains relatively unexplored. Existing approaches demonstrate proficiency in transferring colors and textures but often struggle with replicating the geometry of the scenes. In our work, we leverage an explicit Gaussian Splatting (GS) representation and directly match the distributions of Gaussians between style and content scenes using the Earth Mover's Distance (EMD). By employing the entropy-regularized Wasserstein-2 distance, we ensure that the transformation maintains spatial smoothness. Additionally, we decompose the scene stylization problem into smaller chunks to enhance efficiency. This paradigm shift reframes stylization from a pure generative process driven by latent space losses to an explicit matching of distributions between two Gaussian representations. Our method achieves high-resolution 3D stylization by faithfully transferring details from 3D style scenes onto the content scene. Furthermore, WaSt-3D consistently delivers results across diverse content and style scenes without necessitating any training, as it relies solely on optimization-based techniques. See our project page for additional results and source code: $\href{https://compvis.github.io/wast3d/}{https://compvis.github.io/wast3d/}$.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。