arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2308.01262cs.CVeess.IV
收藏

将季节和太阳特异性融入基于卫星图像的NeRF架构渲染结果中

Incorporating Season and Solar Specificity into Renderings made by a NeRF Architecture using Satellite Images

Michael Gableman, Avinash Kak

AI总结:

本研究拓展Shadow NeRF与Sat-NeRF,通过引入时间输入变量和损失函数额外项,构建了可分离季节特征与阴影的NeRF架构,在8个Maxar卫星图像区域验证了新视角渲染、高度图生成等性能,消融实验验证了设计合理性。

摘要:

Shadow NeRF与Sat-NeRF的出现,使得在基于神经辐射场(Neural Radiance Field, NeRF)的框架中,利用卫星图像训练并从新视角渲染场景时能够纳入太阳角度因素。我们的工作拓展了这些成果,展示了如何实现具备季节特异性的渲染。我们面临的核心挑战是构建一个NeRF,使其既能独立于视角和太阳角度渲染季节特征,又仍能渲染阴影。我们通过引入一个额外的输入变量——一年中的时间,来训练网络渲染季节特征。然而,卫星图像典型的小训练数据集可能会产生歧义:当某个季节的所有图像中同一位置都存在阴影时,网络可能会混淆季节特征与阴影。我们在损失函数中加入额外项,避免网络将阴影归因于季节特征。我们在8个包含Maxar WorldView-3卫星捕获图像的感兴趣区域(Areas of Interest)上验证了网络性能,评估内容包括测试框架准确渲染新视角、生成高度图、预测阴影以及将季节特征与阴影分离的能力。我们的消融实验也验证了网络设计参数选择的合理性。

英文摘要:

As a result of Shadow NeRF and Sat-NeRF, it is possible to take the solar angle into account in a NeRF-based framework for rendering a scene from a novel viewpoint using satellite images for training. Our work extends those contributions and shows how one can make the renderings season-specific. Our main challenge was creating a Neural Radiance Field (NeRF) that could render seasonal features independently of viewing angle and solar angle while still being able to render shadows. We teach our network to render seasonal features by introducing one more input variable -- time of the year. However, the small training datasets typical of satellite imagery can introduce ambiguities in cases where shadows are present in the same location for every image of a particular season. We add additional terms to the loss function to discourage the network from using seasonal features for accounting for shadows. We show the performance of our network on eight Areas of Interest containing images captured by the Maxar WorldView-3 satellite. This evaluation includes tests measuring the ability of our framework to accurately render novel views, generate height maps, predict shadows, and specify seasonal features independently from shadows. Our ablation studies justify the choices made for network design parameters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑