用于场景理解的分因子神经表示
Factored Neural Representation for Scene Understanding
AI总结：
提出一种可直接从单目RGB-D视频学习的分因子神经场景表示，实现物体级可解释、可编辑的编码，支持刚体轨迹与非刚体变形，优于现有神经方法。
摘要：
场景理解的一个长期目标是获得可解释且可编辑的表示，这些表示可以直接从原始单目RGB-D视频中构建，而无需专门的硬件设置或先验知识。在存在多个移动和/或变形物体的情况下，该问题变得更具挑战性。传统方法通过简化、场景先验、预训练模板或已知变形模型的组合来处理该设置。神经表示的出现，特别是神经隐式表示和辐射场，开启了端到端优化的可能性，以共同捕捉几何、外观和物体运动。然而，当前方法产生全局场景编码，假设多视角捕获且场景中运动有限或没有运动，并且除了新视角合成外，不便于轻松操作。在这项工作中，我们引入了一种分因子的神经场景表示，可以直接从单目RGB-D视频中学习，以产生具有物体运动（例如，刚体轨迹）和/或变形（例如，非刚体运动）显式编码的物体级神经表示。我们在合成和真实数据上评估了我们的方法，与一组神经方法进行比较，以证明该表示是高效、可解释且可编辑的（例如，改变物体轨迹）。代码和数据可在http://geometry.cs.ucl.ac.uk/projects/2023/factorednerf获取。
英文摘要：
A long-standing goal in scene understanding is to obtain interpretable and editable representations that can be directly constructed from a raw monocular RGB-D video, without requiring specialized hardware setup or priors. The problem is significantly more challenging in the presence of multiple moving and/or deforming objects. Traditional methods have approached the setup with a mix of simplifications, scene priors, pretrained templates, or known deformation models. The advent of neural representations, especially neural implicit representations and radiance fields, opens the possibility of end-to-end optimization to collectively capture geometry, appearance, and object motion. However, current approaches produce global scene encoding, assume multiview capture with limited or no motion in the scenes, and do not facilitate easy manipulation beyond novel view synthesis. In this work, we introduce a factored neural scene representation that can directly be learned from a monocular RGB-D video to produce object-level neural presentations with an explicit encoding of object movement (e.g., rigid trajectory) and/or deformations (e.g., nonrigid movement). We evaluate ours against a set of neural approaches on both synthetic and real data to demonstrate that the representation is efficient, interpretable, and editable (e.g., change object trajectory). Code and data are available at http://geometry.cs.ucl.ac.uk/projects/2023/factorednerf .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
