Tailor3D：基于双侧图像的定制化3D资产编辑与生成
Tailor3D: Customized 3D Assets Editing and Generation with Dual-Side Images
AI总结：
针对3D AIGC难以精准实现3D资产精细化定制编辑的问题，提出Tailor3D流水线，基于双侧图像编辑与Dual-sided LRM拼接特征，实现高效精准的3D生成与编辑，单步编辑仅需数秒。
摘要：
3D AIGC的最新进展展现了从文本和图像直接创建3D对象的潜力，可在动画和产品设计中大幅节约成本。然而，3D资产的精细化编辑与定制仍是一项长期存在的挑战。具体而言，3D生成方法无法像2D图像生成方法那样精准遵循精细的细节指令。试想你通过3D AIGC得到一个玩具，但它带有不符合预期的配饰和着装。为解决这一挑战，我们提出了名为Tailor3D的新型流水线，它可基于可编辑的双侧图像快速生成定制化3D资产。我们旨在模拟裁缝对物体进行局部修改或执行整体风格迁移的能力。与从多视图创建3D资产不同，使用双侧图像可避免编辑单个视图时在重叠区域产生的冲突。具体而言，该流程首先编辑前视图，随后通过多视图扩散生成对象的后视图，接着对后视图进行编辑。最后，我们提出了Dual-sided LRM（双侧大型重建模型），可将前后3D特征无缝拼接，类似于裁缝缝合衣服的前后片。Dual-sided LRM可修正前后视图之间的不完全一致性，增强编辑能力并降低内存负担，同时通过LoRA Triplane Transformer将它们无缝整合为统一的3D表示。实验结果表明，Tailor3D在各类3D生成与编辑任务（包括3D生成式填充和风格迁移）中均表现出有效性。它为3D资产编辑提供了一种用户友好且高效的解决方案，每个编辑步骤仅需数秒即可完成。
英文摘要：
Recent advances in 3D AIGC have shown promise in directly creating 3D objects from text and images, offering significant cost savings in animation and product design. However, detailed edit and customization of 3D assets remains a long-standing challenge. Specifically, 3D Generation methods lack the ability to follow finely detailed instructions as precisely as their 2D image creation counterparts. Imagine you can get a toy through 3D AIGC but with undesired accessories and dressing. To tackle this challenge, we propose a novel pipeline called Tailor3D, which swiftly creates customized 3D assets from editable dual-side images. We aim to emulate a tailor's ability to locally change objects or perform overall style transfer. Unlike creating 3D assets from multiple views, using dual-side images eliminates conflicts on overlapping areas that occur when editing individual views. Specifically, it begins by editing the front view, then generates the back view of the object through multi-view diffusion. Afterward, it proceeds to edit the back views. Finally, a Dual-sided LRM is proposed to seamlessly stitch together the front and back 3D features, akin to a tailor sewing together the front and back of a garment. The Dual-sided LRM rectifies imperfect consistencies between the front and back views, enhancing editing capabilities and reducing memory burdens while seamlessly integrating them into a unified 3D representation with the LoRA Triplane Transformer. Experimental results demonstrate Tailor3D's effectiveness across various 3D generation and editing tasks, including 3D generative fill and style transfer. It provides a user-friendly, efficient solution for editing 3D assets, with each editing step taking only seconds to complete.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
