MonoNeRF：从单目视频中学习可泛化的动态辐射场
MonoNeRF: Learning a Generalizable Dynamic Radiance Field from Monocular Videos
AI总结：
针对单目动态 NeRF 泛化困难，本文提出 MonoNeRF，通过 Neural ODE 轨迹建模与基于流的特征聚合实现跨场景端到端学习。
摘要：
本文旨在解决从单目视频中学习可泛化动态辐射场的问题。与大多数基于多视图的现有 NeRF 方法不同，单目视频在每个时间戳仅包含一个视图，因此在估计点特征和场景流时会受到沿视图方向歧义的影响。以往研究（如 DynNeRF）通过位置编码消除点特征的歧义，但这种方式不可迁移，并严重限制了泛化能力。因此，这些方法不得不为每个场景训练一个独立模型，在实际应用中面对不断增加的单目视频时会产生高昂的计算成本。为解决该问题，我们提出 MonoNeRF，利用跨帧的点轨迹和特征对应约束，同时学习点特征与场景流。具体而言，我们学习一个隐式速度场，通过 Neural ODE 根据时间特征估计点轨迹，随后使用基于流的特征聚合模块获取沿点轨迹的空间特征。我们以端到端方式联合优化时间特征和空间特征。实验表明，MonoNeRF 能够从多个场景中学习，并支持场景编辑、未见帧合成以及快速适应新场景等新应用。代码可在 https://github.com/tianfr/MonoNeRF 获取。
英文摘要：
In this paper, we target at the problem of learning a generalizable dynamic radiance field from monocular videos. Different from most existing NeRF methods that are based on multiple views, monocular videos only contain one view at each timestamp, thereby suffering from ambiguity along the view direction in estimating point features and scene flows. Previous studies such as DynNeRF disambiguate point features by positional encoding, which is not transferable and severely limits the generalization ability. As a result, these methods have to train one independent model for each scene and suffer from heavy computational costs when applying to increasing monocular videos in real-world applications. To address this, We propose MonoNeRF to simultaneously learn point features and scene flows with point trajectory and feature correspondence constraints across frames. More specifically, we learn an implicit velocity field to estimate point trajectory from temporal features with Neural ODE, which is followed by a flow-based feature aggregation module to obtain spatial features along the point trajectory. We jointly optimize temporal and spatial features in an end-to-end manner. Experiments show that our MonoNeRF is able to learn from multiple scenes and support new applications such as scene editing, unseen frame synthesis, and fast novel scene adaptation. Codes are available at https://github.com/tianfr/MonoNeRF.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
