SSGaussian：语义感知且结构保留的3D风格迁移
SSGaussian: Semantic-Aware and Structure-Preserving 3D Style Transfer
AI总结：
针对现有3D风格迁移方法高层语义提取弱、结构清晰度不足的问题，提出SSGaussian管线，通过跨视角风格对齐与实例级风格迁移，结合2D扩散先验实现更优3D风格化效果，在多类场景中性能超越SOTA。
摘要：
神经表征（如神经辐射场Neural Radiance Fields与3D高斯溅射3D Gaussian Splatting）的最新进展，引发了将风格迁移应用于3D场景的研究兴趣。现有方法虽能将风格图案迁移至具有3D一致性的神经表征上，但难以有效提取并迁移参考风格图像中的高层风格语义。此外，风格化结果往往缺乏结构清晰度与区分度，难以辨别3D场景内的不同实例或物体。针对这些局限，我们提出一种新型3D风格迁移管线，有效整合预训练2D扩散模型的先验知识。该管线包含两个核心阶段：首先，利用扩散先验生成关键视角的风格化渲染图；随后，将风格化关键视角迁移至3D表征上。此过程包含两项创新设计：第一项是跨视角风格对齐，在UNet的最后一个上采样块中插入跨视角注意力，实现多个关键视角间的特征交互，确保扩散模型生成的风格化关键视角同时保持风格保真度与实例级一致性。第二项是实例级风格迁移，有效利用风格化关键视角间的实例级一致性，并将其迁移至3D表征，从而生成更具结构性、视觉连贯性且艺术表现力更强的风格化效果。大量定性与定量实验表明，我们的3D风格迁移管线在从前视场景到具有挑战性的360度环境的各类场景中，性能均显著优于当前最优方法。访问项目页面https://jm-xu.github.io/SSGaussian可查看沉浸式可视化效果。
英文摘要：
Recent advancements in neural representations, such as Neural Radiance Fields and 3D Gaussian Splatting, have increased interest in applying style transfer to 3D scenes. While existing methods can transfer style patterns onto 3D-consistent neural representations, they struggle to effectively extract and transfer high-level style semantics from the reference style image. Additionally, the stylized results often lack structural clarity and separation, making it difficult to distinguish between different instances or objects within the 3D scene. To address these limitations, we propose a novel 3D style transfer pipeline that effectively integrates prior knowledge from pretrained 2D diffusion models. Our pipeline consists of two key stages: First, we leverage diffusion priors to generate stylized renderings of key viewpoints. Then, we transfer the stylized key views onto the 3D representation. This process incorporates two innovative designs. The first is cross-view style alignment, which inserts cross-view attention into the last upsampling block of the UNet, allowing feature interactions across multiple key views. This ensures that the diffusion model generates stylized key views that maintain both style fidelity and instance-level consistency. The second is instance-level style transfer, which effectively leverages instance-level consistency across stylized key views and transfers it onto the 3D representation. This results in a more structured, visually coherent, and artistically enriched stylization. Extensive qualitative and quantitative experiments demonstrate that our 3D style transfer pipeline significantly outperforms state-of-the-art methods across a wide range of scenes, from forward-facing to challenging 360-degree environments. Visit our project page https://jm-xu.github.io/SSGaussian for immersive visualization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
