NeO 360：面向户外场景稀疏视角合成的神经场
NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes
AI总结：
针对现有新视角合成方法需多视角逐场景优化、难以应用于少视角无边界户外场景的问题，提出可泛化的NeO 360方法，通过混合图像条件三平面表示实现单/少视角360°场景重建，性能优于现有方法并支持编辑合成。
摘要：
近期的隐式神经表示在新视角合成任务中已展现出优异效果。但现有方法需要基于大量视角进行成本高昂的逐场景优化，这限制了其在真实世界无边界城市场景中的应用——这类场景中，目标物体或背景仅能从极少视角被观测到。为解决这一难题，我们提出了名为NeO 360（面向户外场景稀疏视角合成的神经场）的新方法。NeO 360是一种可泛化的方法，能够从单张或少量带位姿的RGB图像重建360°场景。该方法的核心在于捕捉复杂真实户外3D场景的分布，并采用一种混合的图像条件三平面表示，可从任意世界坐标点进行查询。这种表示融合了基于体素的表示与鸟瞰图（BEV）表示的优势，比二者单独使用的效果更优、表达能力更强。NeO 360的表示形式使我们能够从大量无边界3D场景集合中学习，同时在推理阶段仅需单张图像即可实现对新视角、新场景的泛化能力。我们在提出的高难度360°无边界数据集NeRDS 360上验证了该方法，结果表明NeO 360的性能优于当前最先进的可泛化新视角合成方法，同时还具备编辑与合成功能。项目主页：https://zubair-irshad.github.io/projects/neo360.html
英文摘要：
Recent implicit neural representations have shown great results for novel view synthesis. However, existing methods require expensive per-scene optimization from many views hence limiting their application to real-world unbounded urban settings where the objects of interest or backgrounds are observed from very few views. To mitigate this challenge, we introduce a new approach called NeO 360, Neural fields for sparse view synthesis of outdoor scenes. NeO 360 is a generalizable method that reconstructs 360° scenes from a single or a few posed RGB images. The essence of our approach is in capturing the distribution of complex real-world outdoor 3D scenes and using a hybrid image-conditional triplanar representation that can be queried from any world point. Our representation combines the best of both voxel-based and bird's-eye-view (BEV) representations and is more effective and expressive than each. NeO 360's representation allows us to learn from a large collection of unbounded 3D scenes while offering generalizability to new views and novel scenes from as few as a single image during inference. We demonstrate our approach on the proposed challenging 360° unbounded dataset, called NeRDS 360, and show that NeO 360 outperforms state-of-the-art generalizable methods for novel view synthesis while also offering editing and composition capabilities. Project page: https://zubair-irshad.github.io/projects/neo360.html
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
