Jigsaw3D:通过块打乱和掩码实现解耦的3D风格迁移
Jigsaw3D: Disentangled 3D Style Transfer via Patch Shuffling and Masking
AI总结:
提出Jigsaw3D流水线,通过拼图操作解耦风格与内容,利用多视图扩散模型实现快速且视图一致的3D风格迁移,显著降低延迟并支持多种泛化应用。
摘要:
可控的3D风格迁移旨在对3D资产进行重新风格化,使其纹理匹配参考图像,同时保持完整性和多视图一致性。现有方法通常依赖直接注入参考风格token或从2D扩散模型中进行分数蒸馏,这会导致繁重的单场景优化,且经常将风格与语义内容纠缠在一起。我们提出了Jigsaw3D,一个基于多视图扩散的流水线,它将风格与内容解耦,并实现快速、视图一致的风格化。我们的核心思想是利用拼图操作——对参考块进行空间打乱和随机掩码——来抑制对象语义并分离风格统计信息(调色板、笔触、纹理)。我们通过参考到视图的交叉注意力将这些风格线索集成到多视图扩散模型中,生成以输入网格为条件的视图一致的风格化渲染。然后将渲染结果烘焙到表面上以生成无缝纹理。在标准的3D风格化基准上,Jigsaw3D实现了高风格保真度和多视图一致性,且延迟显著降低,并能泛化到掩码部分参考风格化、多对象场景风格设计和可平铺纹理生成。项目页面:https://babahui.github.io/jigsaw3D.github.io/
英文摘要:
Controllable 3D style transfer seeks to restyle a 3D asset so that its textures match a reference image while preserving the integrity and multi-view consistency. The prevalent methods either rely on direct reference style token injection or score-distillation from 2D diffusion models, which incurs heavy per-scene optimization and often entangles style with semantic content. We introduce Jigsaw3D, a multi-view diffusion based pipeline that decouples style from content and enables fast, view-consistent stylization. Our key idea is to leverage the jigsaw operation - spatial shuffling and random masking of reference patches - to suppress object semantics and isolate stylistic statistics (color palettes, strokes, textures). We integrate these style cues into a multi-view diffusion model via reference-to-view cross-attention, producing view-consistent stylized renderings conditioned on the input mesh. The renders are then style-baked onto the surface to yield seamless textures. Across standard 3D stylization benchmarks, Jigsaw3D achieves high style fidelity and multi-view consistency with substantially lower latency, and generalizes to masked partial reference stylization, multi-object scene styling, and tileable texture generation. Project page is available at: https://babahui.github.io/jigsaw3D.github.io/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。