ZeST-NeRF:利用时间聚合实现零样本时序 NeRF
ZeST-NeRF: Using temporal aggregation for Zero-Shot Temporal NeRFs
AI总结:
本文提出 ZeST-NeRF,通过时间聚合、多视角合成与场景流场估计,在不重新训练的情况下为新场景生成时序 NeRF,定量指标提升 15% 并取得更优视觉结果。
摘要:
在媒体制作领域,视频编辑技术发挥着关键作用。近期的方法在静态场景的新视角图像合成方面取得了巨大成功,但引入时间信息增加了一层额外的复杂性。以往的模型侧重于使用 NeRF 隐式表示静态和动态场景。这些模型取得了令人印象深刻的结果,但在训练和推理阶段成本高昂。它们过度拟合一个 MLP,将场景隐式地描述为位置的函数。本文提出了 ZeST-NeRF,这是一种无需重新训练即可为新场景生成时序 NeRF 的新方法。我们仅使用不相关场景进行训练,即可利用多视角合成技术和场景流场估计准确重建新视角。我们展示了来自多个领域的现有最先进方法如何无法充分解决这一新任务,并证明了我们解决方案的有效性。所得到的网络在定量指标上提升了 15%,并产生了显著更好的视觉结果。
英文摘要:
In the field of media production, video editing techniques play a pivotal role. Recent approaches have had great success at performing novel view image synthesis of static scenes. But adding temporal information adds an extra layer of complexity. Previous models have focused on implicitly representing static and dynamic scenes using NeRF. These models achieve impressive results but are costly at training and inference time. They overfit an MLP to describe the scene implicitly as a function of position. This paper proposes ZeST-NeRF, a new approach that can produce temporal NeRFs for new scenes without retraining. We can accurately reconstruct novel views using multi-view synthesis techniques and scene flow-field estimation, trained only with unrelated scenes. We demonstrate how existing state-of-the-art approaches from a range of fields cannot adequately solve this new task and demonstrate the efficacy of our solution. The resulting network improves quantitatively by 15% and produces significantly better visual results.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。