MagicClay:使用生成式神经场雕刻网格
MagicClay: Sculpting Meshes With Generative Neural Fields
AI总结:
针对神经场缺乏增量控制、网格难以神经优化的问题,提出MagicClay混合框架,协同维持网格与SDF表示一致性,支持文本引导的网格区域雕刻与连续编辑,生成效果优于现有技术。
摘要:
神经场的最新进展为形状生成领域带来了卓越的能力,但它们缺乏关键特性,例如增量控制——这是艺术创作的一项基本要求。另一方面,三角形网格是大多数几何相关任务的首选表示形式,兼具高效性与直观可控性,但不适用于神经优化。为支撑下游任务,现有技术通常采用两步法:首先使用神经场生成形状,随后提取网格以进行后续处理。\n与之不同,本文提出了一种混合方法,可同时维持网格与有符号距离场(Signed Distance Field,SDF)两种表示的一致性。基于该表示,我们推出了MagicClay——一款面向艺术家的友好工具,能够根据文本提示雕刻网格的特定区域,同时保持其他区域不受影响。我们的框架在形状优化的每一步中,都能精细且高效地平衡两种表示之间的一致性与正则化约束;借助网格表示,我们展示了如何以更高分辨率、更快速度渲染SDF。此外,我们采用最新的可微网格重建技术,根据SDF的指示在网格中需要的位置自适应分配三角形。\n通过已实现的原型系统,我们证明该方法生成的几何形状优于当前最优水平,并且实现了全新的一致性控制,首次支持对同一网格进行基于提示的连续编辑。
英文摘要:
The recent developments in neural fields have brought phenomenal capabilities to the field of shape generation, but they lack crucial properties, such as incremental control - a fundamental requirement for artistic work. Triangular meshes, on the other hand, are the representation of choice for most geometry related tasks, offering efficiency and intuitive control, but do not lend themselves to neural optimization. To support downstream tasks, previous art typically proposes a two-step approach, where first a shape is generated using neural fields, and then a mesh is extracted for further processing. Instead, in this paper we introduce a hybrid approach that maintains both a mesh and a Signed Distance Field (SDF) representations consistently. Using this representation, we introduce MagicClay - an artist friendly tool for sculpting regions of a mesh according to textual prompts while keeping other regions untouched. Our framework carefully and efficiently balances consistency between the representations and regularizations in every step of the shape optimization; Relying on the mesh representation, we show how to render the SDF at higher resolutions and faster. In addition, we employ recent work in differentiable mesh reconstruction to adaptively allocate triangles in the mesh where required, as indicated by the SDF. Using an implemented prototype, we demonstrate superior generated geometry compared to the state-of-the-art, and novel consistent control, allowing sequential prompt-based edits to the same mesh for the first time.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。