arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.06233cs.GR
收藏

视图对应网络用于隐式光场表示

View Correspondence Network for Implicit Light Field Representation

Süleyman Aslan, Brandon Yushan Feng, Amitabh Varshney

AI总结:

提出视图对应网络(VICON),通过立体匹配和自动微分实现隐式光场表示,在连续视点下高质量重建并超越相机视场。

摘要:

我们提出了一种新颖的技术,用于在连续定义的视点下以高质量和高保真度对光场进行隐式神经表示。我们的隐式神经表示将定义光场双平面参数化的4D坐标映射到相应的颜色值。我们利用周期激活函数来实现高表达能力和对复杂数据流形的精确重建,同时保持低存储和低推理时间需求。然而,朴素训练的非3D结构化网络不能充分满足多视图一致性;相反,它们执行附近视点的alpha混合。相比之下,我们的视图对应网络(VICON)利用立体匹配、对输入空间进行自动微分的优化以及多视图像素对应,提供了一种新颖的隐式光场表示,对训练期间未见的新视图保持忠实。实验结果表明,VICON在定性和定量上都优于最先进的非3D隐式光场表示。此外,我们的隐式表示捕获了更大的视场(FoV),超越了地面真值渲染中相机可观察场景的范围。

英文摘要:

We present a novel technique for implicit neural representation of light fields at continuously defined viewpoints with high quality and fidelity. Our implicit neural representation maps 4D coordinates defining two-plane parameterization of the light fields to the corresponding color values. We leverage periodic activations to achieve high expressivity and accurate reconstruction for complex data manifolds while keeping low storage and inference time requirements. However, naïvely trained non-3D structured networks do not adequately satisfy the multi-view consistency; instead, they perform alpha blending of nearby viewpoints. In contrast, our View Correspondence Network, or VICON, leverages stereo matching, optimization by automatic differentiation with respect to the input space, and multi-view pixel correspondence to provide a novel implicit representation of the light fields faithful to the novel views that are unseen during the training. Experimental results show VICON superior to the state-of-the-art non-3D implicit light field representations both qualitatively and quantitatively. Moreover, our implicit representation captures a larger field of view (FoV), surpassing the extent of the observable scene by the cameras of the ground truth renderings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑