arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2505.22854cs.CV

CLIPGaussian:基于高斯溅射的通用多模态风格迁移

CLIPGaussian: Universal and Multimodal Style Transfer Based on Gaussian Splatting

Jagiellonian University(雅盖隆大学);IDEAS Research Institute(IDEAS研究院)

Kornel Howil, Joanna Waczyńska, Piotr Borycki, Tadeusz Dziarmaga, Marcin Mazur, Przemysław Spurek

AI总结:

针对高斯溅射表征的复杂风格迁移难题,提出首个支持多模态、文本/图像引导的统一风格迁移框架CLIPGaussian,可插件化集成、无需大模型或重训练,实现色彩几何联合优化与时序一致,风格保真与一致性表现更优。

摘要:

高斯溅射(Gaussian Splatting,GS)近期已成为一种从2D图像渲染3D场景的高效表征方式,且已被拓展应用于图像、视频及动态4D内容领域。然而,针对基于GS的表征开展风格迁移,尤其是实现超越简单色彩变化的风格迁移,仍具挑战性。本研究提出CLIPGaussian,这是首个支持文本与图像引导式风格化的统一风格迁移框架,可覆盖2D图像、视频、3D物体及4D场景等多种模态。该方法直接作用于高斯基元,可作为插件模块集成至现有GS流程中,无需依赖大型生成模型,也无需从头重新训练。CLIPGaussian方法可在3D与4D场景下实现色彩与几何的联合优化,在保证模型规模不变的同时,还能实现视频中的时序一致性。我们在所有任务中均验证了该方法具备更优的风格保真度与一致性,证实CLIPGaussian是多模态风格迁移的通用高效解决方案。

英文摘要:

Gaussian Splatting (GS) has recently emerged as an efficient representation for rendering 3D scenes from 2D images and has been extended to images, videos, and dynamic 4D content. However, applying style transfer to GS-based representations, especially beyond simple color changes, remains challenging. In this work, we introduce CLIPGaussian, the first unified style transfer framework that supports text- and image-guided stylization across multiple modalities: 2D images, videos, 3D objects, and 4D scenes. Our method operates directly on Gaussian primitives and integrates into existing GS pipelines as a plug-in module, without requiring large generative models or retraining from scratch. The CLIPGaussian approach enables joint optimization of color and geometry in 3D and 4D settings, and achieves temporal coherence in videos, while preserving the model size. We demonstrate superior style fidelity and consistency across all tasks, validating CLIPGaussian as a universal and efficient solution for multimodal style transfer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑