全光函数的群体采样
Crowdsampling the Plenoptic Function
摘要:
许多受欢迎的旅游地标都被大量的在线公开照片所捕捉。这些照片表示特定场景的全光函数的稀疏和非结构化采样。在本文中,我们提出了一种新的方法,根据这些数据的时变照明下的新的视图合成。我们的方法建立在最近的多平面图像(MPI)的格式表示在固定的观看条件下的局部光场。我们引入了一种新的DeepMPI表示,其动机是对全光函数的稀疏结构的观察,允许实时合成在空间和照明变化中连续的真实感视图。我们的方法可以合成与以前的MPI方法相同的引人注目的视差和视图相关的效果,同时沿着反射率和照明随时间的变化进行插值。我们展示了如何从非结构化的照片集合中以无监督的方式学习这些效果的模型,而无需时间注册,这表明神经渲染的近期工作有了显着的改进。更多信息可以在crowdsampling.io上找到。
英文摘要:
Many popular tourist landmarks are captured in a multitude of online, public photos. These photos represent a sparse and unstructured sampling of the plenoptic function for a particular scene. In this paper,we present a new approach to novel view synthesis under time-varying illumination from such data. Our approach builds on the recent multi-plane image (MPI) format for representing local light fields under fixed viewing conditions. We introduce a new DeepMPI representation, motivated by observations on the sparsity structure of the plenoptic function, that allows for real-time synthesis of photorealistic views that are continuous in both space and across changes in lighting. Our method can synthesize the same compelling parallax and view-dependent effects as previous MPI methods, while simultaneously interpolating along changes in reflectance and illumination with time. We show how to learn a model of these effects in an unsupervised way from an unstructured collection of photos without temporal registration, demonstrating significant improvements over recent work in neural rendering. More information can be found crowdsampling.io.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。