arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.08488cs.CVcs.AIcs.LG
收藏

ICE-G:3D Gaussian Splats 的图像条件编辑

ICE-G: Image Conditional Editing of 3D Gaussian Splats

Georgia Institute of Technology(佐治亚理工学院);Toyota Research Institute(丰田研究所);Stability AI;Google Research(谷歌研究院)

Vishnu Jaganathan, Hannah Hanyun Huang, Muhammad Zubair Irshad, Varun Jampani, Amit Raj, Zsolt Kira

AI总结:

本文提出 ICE-G 框架,利用单张参考图像的分割与 DINO 语义对应,将颜色和纹理编辑传播到多视图并重新训练 3D 场景,实现高质量、细粒度可控的 Gaussian Splats 编辑。

摘要:

最近涌现出许多用于创建高质量 3D 资产和场景的技术。然而,当涉及编辑这些对象时,现有方法要么速度慢、在质量上妥协,要么无法提供足够的定制能力。我们提出了一种新颖方法,可从单个参考视图快速编辑 3D 模型。我们的技术首先对编辑图像进行分割,然后使用 DINO 特征在选定的、已分割的数据集视图之间匹配语义对应区域。随后,来自编辑图像特定区域的颜色或纹理变化能够以语义合理的方式自动应用到其他视图。这些经过编辑的视图充当更新后的数据集,用于进一步训练并重新风格化 3D 场景。因此,最终结果是一个经过编辑的 3D 模型。我们的框架支持多种编辑任务,例如手动局部编辑、基于任意示例图像中对应关系的风格迁移,以及来自多张示例图像的不同风格组合。由于 Gaussian Splats 具有速度快且便于局部编辑的特点,我们将其用作主要 3D 表示,但我们的技术也适用于 NeRF 等其他方法。我们通过多个示例表明,我们的方法能够产生更高质量的结果,同时提供细粒度的编辑控制。项目页面:ice-gaussian.github.io

英文摘要:

Recently many techniques have emerged to create high quality 3D assets and scenes. When it comes to editing of these objects, however, existing approaches are either slow, compromise on quality, or do not provide enough customization. We introduce a novel approach to quickly edit a 3D model from a single reference view. Our technique first segments the edit image, and then matches semantically corresponding regions across chosen segmented dataset views using DINO features. A color or texture change from a particular region of the edit image can then be applied to other views automatically in a semantically sensible manner. These edited views act as an updated dataset to further train and re-style the 3D scene. The end-result is therefore an edited 3D model. Our framework enables a wide variety of editing tasks such as manual local edits, correspondence based style transfer from any example image, and a combination of different styles from multiple example images. We use Gaussian Splats as our primary 3D representation due to their speed and ease of local editing, but our technique works for other methods such as NeRFs as well. We show through multiple examples that our method produces higher quality results while offering fine-grained control of editing. Project page: ice-gaussian.github.io

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑