Make Your MoVe:通过将多视图扩散模型适配于外部编辑来制作你的3D内容
Make Your MoVe: Make Your 3D Contents by Adapting Multi-View Diffusion Models to External Editing
AI总结:
针对2D编辑结果直接用于3D生成导致质量下降的问题,提出免微调的即插即用方案,通过几何保留模块和注入切换器,在单次推理中实现编辑后资产与原始几何的对齐,有效提升了3D资产的多视图一致性与网格质量。
摘要:
随着3D生成技术的不断繁荣,对生成个性化内容的需求正在迅速上升。用户越来越希望应用各种编辑方法来润色生成的3D内容,旨在增强其颜色、风格和光照,同时不损害底层几何结构。然而,大多数现有编辑工具专注于2D领域,直接将其结果输入到3D生成方法(如多视图扩散模型)中会引入信息丢失,降低最终3D资产的质量。在本文中,我们提出了一种免微调、即插即用的方案,能够在单次推理运行中将编辑后的资产与其原始几何结构对齐。我们方法的核心是一个几何保留模块,该模块利用原始输入法线隐变量来指导编辑后的多视图生成。此外,还提出了一个注入切换器,用于专门控制原始法线的监督程度,确保编辑后的颜色视图与法线视图之间的对齐。大量实验表明,在多视图扩散模型和编辑方法的多种组合下,我们的方法持续提升了编辑后3D资产的多视图一致性和网格质量。
英文摘要:
As 3D generation techniques continue to flourish, the demand for generating personalized content is rapidly rising. Users increasingly seek to apply various editing methods to polish generated 3D content, aiming to enhance its color, style, and lighting without compromising the underlying geometry. However, most existing editing tools focus on the 2D domain, and directly feeding their results into 3D generation methods (like multi-view diffusion models) will introduce information loss, degrading the quality of the final 3D assets. In this paper, we propose a tuning-free, plug-and-play scheme that aligns edited assets with their original geometry in a single inference run. Central to our approach is a geometry preservation module that guides the edited multi-view generation with original input normal latents. Besides, an injection switcher is proposed to deliberately control the supervision extent of the original normals, ensuring the alignment between the edited color and normal views. Extensive experiments show that our method consistently improves both the multi-view consistency and mesh quality of edited 3D assets, across multiple combinations of multi-view diffusion models and editing methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。