arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.02514cs.CV

Freditor:通过频率分解实现高保真且可迁移的 NeRF 编辑

Freditor: High-Fidelity and Transferable NeRF Editing by Frequency Decomposition

Alibaba Group(阿里巴巴集团);Fudan University(复旦大学);The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Yisheng He, Weihao Yuan, Siyu Zhu, Zilong Dong, Liefeng Bo, Qixing Huang

AI总结:

针对 NeRF 编辑中二维结果不一致导致的模糊和细节丢失问题,提出 Freditor,在低频特征空间中编辑以保留高频细节,并支持强度控制与场景迁移。

摘要:

本文通过频率分解实现了高保真、可迁移的 NeRF 编辑。近期的 NeRF 编辑流程将二维风格化结果提升到三维场景,但结果模糊,并且由于二维编辑之间的不一致性,无法捕捉细节结构。我们的关键洞察是,与高频部分相比,图像的低频成分在编辑后具有更强的多视角一致性。此外,外观风格主要体现在低频成分中,而内容细节尤其存在于高频部分。这促使我们对低频成分执行编辑,从而产生高保真的编辑场景。另外,由于编辑在低频特征空间中执行,因此能够实现稳定的强度控制和新场景迁移。在照片级真实感数据集上进行的综合实验证明了高保真且可迁移的 NeRF 编辑的优越性能。项目页面位于 https://aigc3d.github.io/freditor。

英文摘要:

This paper enables high-fidelity, transferable NeRF editing by frequency decomposition. Recent NeRF editing pipelines lift 2D stylization results to 3D scenes while suffering from blurry results, and fail to capture detailed structures caused by the inconsistency between 2D editings. Our critical insight is that low-frequency components of images are more multiview-consistent after editing compared with their high-frequency parts. Moreover, the appearance style is mainly exhibited on the low-frequency components, and the content details especially reside in high-frequency parts. This motivates us to perform editing on low-frequency components, which results in high-fidelity edited scenes. In addition, the editing is performed in the low-frequency feature space, enabling stable intensity control and novel scene transfer. Comprehensive experiments conducted on photorealistic datasets demonstrate the superior performance of high-fidelity and transferable NeRF editing. The project page is at \url{https://aigc3d.github.io/freditor}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑