arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.02034cs.CV

TrAME:轨迹锚定的多视图编辑用于文本引导的三维高斯泼溅操作

TrAME: Trajectory-Anchored Multi-View Editing for Text-Guided 3D Gaussian Splatting Manipulation

School of Information Science and Technology, University of Science and Technology of China(中国科学技术大学信息科学技术学院);Space AI, Li Auto(理想汽车空间人工智能部门);Department of Automation, Tsinghua University(清华大学自动化系)

Chaofan Luo, Donglin Di, Xun Yang, Yongjia Ma, Zhou Xue, Chen Wei, Yebin Liu

AI总结:

针对三维场景编辑中多视图一致性问题,提出基于轨迹锚定方案和双分支机制的渐进式编辑策略,并引入视图一致性注意力控制模块,实现高质量文本引导三维高斯泼溅编辑。

摘要:

尽管三维场景编辑领域取得了显著进展,但当前方法仍面临重大挑战,尤其是在多视图编辑过程中保持三维一致性的问题。为应对这一挑战,我们提出了一种渐进式三维编辑策略,通过轨迹锚定方案(TAS)和双分支编辑机制确保多视图一致性。具体而言,TAS促进了二维视图编辑与三维更新之间的紧密耦合迭代过程,防止了文本到图像过程中产生的误差累积。此外,我们探讨了基于优化的方法与基于重建的方法之间的关系,为选择更优的设计方案提供了统一视角,支持了所设计TAS的合理性。我们进一步提出了一种无需调优的视图一致性注意力控制(VCAC)模块,该模块利用源分支的跨视图语义和几何参考,在二维视图编辑过程中从目标分支生成对齐的视图。为验证我们方法的有效性,我们分析了二维示例,以展示使用VCAC模块后一致性的提升。在文本引导的三维场景编辑中进行的进一步广泛定量和定性结果表明,我们的方法相比最先进的方法实现了更优的编辑质量。我们将在评审过程结束后公开发布完整的代码库。

英文摘要:

Despite significant strides in the field of 3D scene editing, current methods encounter substantial challenge, particularly in preserving 3D consistency in multi-view editing process. To tackle this challenge, we propose a progressive 3D editing strategy that ensures multi-view consistency via a Trajectory-Anchored Scheme (TAS) with a dual-branch editing mechanism. Specifically, TAS facilitates a tightly coupled iterative process between 2D view editing and 3D updating, preventing error accumulation yielded from text-to-image process. Additionally, we explore the relationship between optimization-based methods and reconstruction-based methods, offering a unified perspective for selecting superior design choice, supporting the rationale behind the designed TAS. We further present a tuning-free View-Consistent Attention Control (VCAC) module that leverages cross-view semantic and geometric reference from the source branch to yield aligned views from the target branch during the editing of 2D views. To validate the effectiveness of our method, we analyze 2D examples to demonstrate the improved consistency with the VCAC module. Further extensive quantitative and qualitative results in text-guided 3D scene editing indicate that our method achieves superior editing quality compared to state-of-the-art methods. We will make the complete codebase publicly available following the conclusion of the review process.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑