Coin3D：通过代理引导条件实现可控交互式3D资产生成
Coin3D: Controllable and Interactive 3D Assets Generation with Proxy-Guided Conditioning
AI总结：
针对3D生成可控性与效率不足的问题，提出Coin3D框架，通过粗略几何代理控制生成，结合多项技术实现秒级响应的交互式局部编辑，提升3D资产生成的可控性与灵活性。
摘要：
作为人类，我们渴望创造既能自由构思又易于控制的媒体内容。得益于生成式技术的显著发展，我们如今可以轻松利用2D扩散方法合成由原始草图或指定人体姿态控制的图像，甚至能通过掩码修复逐步编辑/再生局部区域。然而，由于3D生成缺乏可控性与效率，3D建模任务中尚无法实现类似的工作流。本文提出了一种新型可控交互式3D资产建模框架，名为Coin3D。Coin3D允许用户使用由基础形状组装的粗略几何代理控制3D生成，并引入交互式生成工作流，支持无缝的局部部件编辑，同时能在数秒内生成响应式3D对象预览。为此，我们开发了多项技术，包括将体素粗略形状控制应用于扩散模型的3D适配器、用于精确部件编辑的代理约束编辑策略、支持响应式预览的渐进式体素缓存，以及确保一致网格重建的volume-SDS。在多样形状代理上开展的交互式生成与编辑大量实验表明，我们的方法在3D资产生成任务中实现了更出色的可控性与灵活性。
英文摘要：
As humans, we aspire to create media content that is both freely willed and readily controlled. Thanks to the prominent development of generative techniques, we now can easily utilize 2D diffusion methods to synthesize images controlled by raw sketch or designated human poses, and even progressively edit/regenerate local regions with masked inpainting. However, similar workflows in 3D modeling tasks are still unavailable due to the lack of controllability and efficiency in 3D generation. In this paper, we present a novel controllable and interactive 3D assets modeling framework, named Coin3D. Coin3D allows users to control the 3D generation using a coarse geometry proxy assembled from basic shapes, and introduces an interactive generation workflow to support seamless local part editing while delivering responsive 3D object previewing within a few seconds. To this end, we develop several techniques, including the 3D adapter that applies volumetric coarse shape control to the diffusion model, proxy-bounded editing strategy for precise part editing, progressive volume cache to support responsive preview, and volume-SDS to ensure consistent mesh reconstruction. Extensive experiments of interactive generation and editing on diverse shape proxies demonstrate that our method achieves superior controllability and flexibility in the 3D assets generation task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
