arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.15487cs.RO
收藏

AgriNeRF:面向挑战性光照条件的农业神经辐射场

AgriNeRF: Neural Radiance Fields for Agriculture in Challenging Lighting Conditions

University of Pittsburgh(匹兹堡大学);University of Pennsylvania(宾夕法尼亚大学)

Samarth Chopra, Fernando Cladera, Varun Murali, Vijay Kumar

AI总结:

本研究提出AgriNeRF,通过融合RGB、事件和热成像相机构建多模态神经辐射场,解决了传统NeRF在农业挑战性光照下鲁棒性差的问题,显著提升了场景重建质量与下游果实检测性能。

摘要:

神经辐射场(Neural Radiance Fields, NeRFs)在3D场景重建和新视角合成中展现出显著潜力。在农业场景中,NeRFs可作为数字孪生,为农民提供关于果实检测的关键信息,用于产量估计及其他重要指标。然而,传统NeRFs对挑战性光照条件(如低光照、极端强光和变化光照)不够鲁棒。为解决这些问题,本研究利用三种不同的传感器:RGB相机、事件相机和热成像相机。我们的RGB场景重建在PSNR和SSIM上分别提升了+2.06 dB和+8.3%。我们的跨光谱场景重建将下游果实检测在mAP50上提升了+43.0%,在mAP50-95上提升了+61.1%。额外传感器的集成使得NeRF更加鲁棒且信息更丰富。我们证明,我们的多模态系统能够在各种树冠覆盖下和一天中的不同时间生成高质量、照片级真实的重建结果。本研究开发了一种弹性NeRF,能够在视觉退化场景中表现良好,并构建了一种学习到的跨光谱表示,用于自动化果实检测。

英文摘要:

Neural Radiance Fields (NeRFs) have shown significant promise in 3D scene reconstruction and novel view synthesis. In agricultural settings, NeRFs can serve as digital twins, providing critical information about fruit detection for yield estimation and other important metrics for farmers. However, traditional NeRFs are not robust to challenging lighting conditions, such as low-light, extreme bright light and varying lighting. To address these issues, this work leverages three different sensors: an RGB camera, an event camera and a thermal camera. Our RGB scene reconstruction shows an improvement in PSNR and SSIM by +2.06 dB and +8.3% respectively. Our cross-spectral scene reconstruction enhances downstream fruit detection by +43.0% in mAP50 and +61.1% increase in mAP50-95. The integration of additional sensors leads to a more robust and informative NeRF. We demonstrate that our multi-modal system yields high quality photo-realistic reconstructions under various tree canopy covers and at different times of the day. This work results in the development of a resilient NeRF, capable of performing well in visibly degraded scenarios, as well as a learnt cross-spectral representation, that is used for automated fruit detection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑