神经视频场编辑
Neural Video Fields Editing
AI总结：
针对文本驱动视频编辑中显存需求高、帧间不一致的问题，提出基于神经视频场的NVEdit框架，结合三平面与稀疏网格高效编码长视频，借助T2I模型和渐进优化实现一致编辑，支持组件替换。
摘要：
扩散模型已经彻底革新了文本驱动的视频编辑领域。但将这类方法应用于现实编辑场景时，会遇到两大显著挑战：（1）GPU显存需求随帧数增长急剧上升；（2）编辑后视频存在帧间不一致问题。为此，我们提出NVEdit，这是一种全新的文本驱动视频编辑框架，旨在降低显存开销并提升现实长视频编辑的一致性。具体而言，我们构建了由三平面（tri-plane）和稀疏网格（sparse grid）支撑的神经视频场，能够以高效利用显存的方式编码包含数百帧的长视频。随后，我们通过现成的文本到图像（Text-to-Image, T2I）模型更新视频场，以实现文本驱动的编辑效果。我们还设计了一种渐进式优化策略，用于保留视频原有的时序先验。值得注意的是，神经视频场和T2I模型均具备可适配性与可替换性，可为后续研究提供启发。实验表明，我们的方法能够编辑数百帧的视频，且帧间一致性表现出色。我们的项目可访问：https://nvedit.github.io/。
英文摘要：
Diffusion models have revolutionized text-driven video editing. However, applying these methods to real-world editing encounters two significant challenges: (1) the rapid increase in GPU memory demand as the number of frames grows, and (2) the inter-frame inconsistency in edited videos. To this end, we propose NVEdit, a novel text-driven video editing framework designed to mitigate memory overhead and improve consistent editing for real-world long videos. Specifically, we construct a neural video field, powered by tri-plane and sparse grid, to enable encoding long videos with hundreds of frames in a memory-efficient manner. Next, we update the video field through off-the-shelf Text-to-Image (T2I) models to impart text-driven editing effects. A progressive optimization strategy is developed to preserve original temporal priors. Importantly, both the neural video field and T2I model are adaptable and replaceable, thus inspiring future research. Experiments demonstrate the ability of our approach to edit hundreds of frames with impressive inter-frame consistency. Our project is available at: https://nvedit.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
