arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.07408cs.CV
收藏

宽基线极端稀疏光场的学习动态插值

Learning Dynamic Interpolation for Extremely Sparse Light Fields with Wide Baselines

City University of Hong Kong(香港城市大学)

Mantang Guo, Jing Jin, Hui Liu, Junhui Hou

摘要:

本文解决了从宽基线稀疏采样光场重建稠密光场的问题,提出了一种可学习的模型,即动态插值,以代替常用的几何扭曲操作。具体来说,与输入视图之间的几何关系估计,我们首先构建一个轻量级的神经网络动态学习权值插值相邻像素从输入视图独立合成每个像素的新的视图。与几何变形操作中采用的固定和内容无关的权重相比,学习的插值权重隐含地包含源视图和新视图之间的对应关系,并适应于不同的图像内容信息。然后,我们恢复独立合成的像素之间的空间相关性的每个新的视图通过参考输入视图使用基于几何的空间细化模块。我们还约束的角度之间的相关性的新的意见,通过一个取向的LF结构损失。在宽基线LF数据集上的实验结果表明,与现有方法相比,重建的LF具有更高的PSNR/SSIM,并且更好地保留了LF视差结构。源代码可在https://github.com/MantangGuo/DI4SLF上公开获得。

英文摘要:

In this paper, we tackle the problem of dense light field (LF) reconstruction from sparsely-sampled ones with wide baselines and propose a learnable model, namely dynamic interpolation, to replace the commonly-used geometry warping operation. Specifically, with the estimated geometric relation between input views, we first construct a lightweight neural network to dynamically learn weights for interpolating neighbouring pixels from input views to synthesize each pixel of novel views independently. In contrast to the fixed and content-independent weights employed in the geometry warping operation, the learned interpolation weights implicitly incorporate the correspondences between the source and novel views and adapt to different image content information. Then, we recover the spatial correlation between the independently synthesized pixels of each novel view by referring to that of input views using a geometry-based spatial refinement module. We also constrain the angular correlation between the novel views through a disparity-oriented LF structure loss. Experimental results on LF datasets with wide baselines show that the reconstructed LFs achieve much higher PSNR/SSIM and preserve the LF parallax structure better than state-of-the-art methods. The source code is publicly available at https://github.com/MantangGuo/DI4SLF.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑