4DStyleGaussian:基于Gaussian Splatting的零样本4D风格迁移
4DStyleGaussian: Zero-shot 4D Style Transfer with Gaussian Splatting
AI总结:
提出4DStyleGaussian框架,利用嵌入式4D Gaussian Splatting和可逆神经网络预测4D风格变换矩阵,实现高效、零样本且时空一致的动态场景风格迁移。
摘要:
3D神经风格迁移因能提供具空间一致性的用户友好风格化而备受关注。然而,现有3D风格迁移方法在推理效率和泛化能力上常显不足,且难以处理具有时间一致性的动态场景。本文提出4DStyleGaussian,一种新型4D风格迁移框架,旨在对任意风格参考实现实时风格化,同时保持合理的内容亲和力、多视图一致性和时间连贯性。该方法利用嵌入式4D Gaussian Splatting技术,通过可逆神经网络训练以减少特征蒸馏过程中的内容损失。利用4D嵌入高斯,我们预测4D风格变换矩阵,借助Gaussian Splatting实现空间和时间上保持一致的风格迁移。实验表明,该方法能为4D场景实现高质量、零样本风格化,并提升效率和时空一致性。
英文摘要:
3D neural style transfer has gained significant attention for its potential to provide user-friendly stylization with spatial consistency. However, existing 3D style transfer methods often fall short in terms of inference efficiency, generalization ability, and struggle to handle dynamic scenes with temporal consistency. In this paper, we introduce 4DStyleGaussian, a novel 4D style transfer framework designed to achieve real-time stylization of arbitrary style references while maintaining reasonable content affinity, multi-view consistency, and temporal coherence. Our approach leverages an embedded 4D Gaussian Splatting technique, which is trained using a reversible neural network for reducing content loss in the feature distillation process. Utilizing the 4D embedded Gaussians, we predict a 4D style transformation matrix that facilitates spatially and temporally consistent style transfer with Gaussian Splatting. Experiments demonstrate that our method can achieve high-quality and zero-shot stylization for 4D scenarios with enhanced efficiency and spatial-temporal consistency.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。