MvDrag3D：基于多视图生成-重建先验的拖拽式创意3D编辑
MvDrag3D: Drag-based Creative 3D Editing via Multi-view Generation-Reconstruction Priors
AI总结：
MVDrag3D提出利用多视图扩散生成先验与重建模型实现灵活一致的3D拖拽编辑，并通过视图特定变形网络和多视图评分函数解决视图错位问题，提升编辑质量与多样性。
摘要：
在图像生成模型能力的推动下，拖拽式编辑已在2D内容创作中变得流行。然而，将这一技术扩展到3D仍然是一个挑战。现有的3D拖拽式编辑方法，无论是采用显式空间变换，还是依赖容量有限的3D生成模型中的隐式潜在优化，都难以处理显著的拓扑变化或在多样化物体类别上生成新纹理。为克服这些局限，我们提出了MVDrag3D，这是一个利用多视图生成与重建先验来实现更灵活、更具创意的拖拽式3D编辑的新框架。我们方法的核心是使用多视图扩散模型作为强生成先验，在多个渲染视图上执行一致的拖拽编辑，随后由重建模型重建编辑后物体的3D高斯。虽然初始的3D高斯可能存在不同视图之间的错位，我们通过视图特定的变形网络来调整高斯的位置，使其良好对齐。此外，我们提出了一种多视图评分函数，从多个视图中蒸馏生成先验，以进一步增强视图一致性和视觉质量。大量实验表明，MVDrag3D为3D拖拽式编辑提供了一种精确、生成式且灵活的解决方案，支持跨多种物体类别和3D表示实现更多样化的编辑效果。
英文摘要：
Drag-based editing has become popular in 2D content creation, driven by the capabilities of image generative models. However, extending this technique to 3D remains a challenge. Existing 3D drag-based editing methods, whether employing explicit spatial transformations or relying on implicit latent optimization within limited-capacity 3D generative models, fall short in handling significant topology changes or generating new textures across diverse object categories. To overcome these limitations, we introduce MVDrag3D, a novel framework for more flexible and creative drag-based 3D editing that leverages multi-view generation and reconstruction priors. At the core of our approach is the usage of a multi-view diffusion model as a strong generative prior to perform consistent drag editing over multiple rendered views, which is followed by a reconstruction model that reconstructs 3D Gaussians of the edited object. While the initial 3D Gaussians may suffer from misalignment between different views, we address this via view-specific deformation networks that adjust the position of Gaussians to be well aligned. In addition, we propose a multi-view score function that distills generative priors from multiple views to further enhance the view consistency and visual quality. Extensive experiments demonstrate that MVDrag3D provides a precise, generative, and flexible solution for 3D drag-based editing, supporting more versatile editing effects across various object categories and 3D representations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
