arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.07578cs.CVcs.LGeess.IV
收藏

E2V-SDE:通过神经随机微分方程从异步事件到快速连续视频重建

E2V-SDE: From Asynchronous Events to Fast and Continuous Video Reconstruction via Neural Stochastic Differential Equations

Jongwan Kim, DongJin Lee, Byunggook Na, Seongsik Park, Jeonghee Jo, Sungroh Yoon

摘要:

事件摄像机对场景中的亮度变化做出异步响应,并独立于每个像素。由于这些属性,这些相机具有独特的功能:高动态范围(HDR),高时间分辨率和低功耗。然而,事件相机的结果应该被处理成计算机视觉任务的替代表示。此外,它们通常很吵,在事件很少的区域性能很差。近年来,许多研究人员试图从事件中重建视频。然而,由于缺乏来自不规则和不连续数据的时间信息,它们不能提供高质量的视频。为了克服这些困难,我们引入了一个E2 V-E2 V,其动力学是由随机微分方程(E2 V)的潜空间。因此,E2 V-SVM可以在任意时间步长快速重建图像,并对看不见的数据进行现实的预测。此外,我们成功地采用了各种图像合成技术,提高图像清晰度和时间一致性。通过对模拟和真实场景数据集进行广泛的实验,我们验证了我们的模型在各种视频重建设置下的性能优于最先进的方法。在图像质量方面,LPIPS评分提高了12%,重建速度比ET-Net高出87%。

英文摘要:

Event cameras respond to brightness changes in the scene asynchronously and independently for every pixel. Due to the properties, these cameras have distinct features: high dynamic range (HDR), high temporal resolution, and low power consumption. However, the results of event cameras should be processed into an alternative representation for computer vision tasks. Also, they are usually noisy and cause poor performance in areas with few events. In recent years, numerous researchers have attempted to reconstruct videos from events. However, they do not provide good quality videos due to a lack of temporal information from irregular and discontinuous data. To overcome these difficulties, we introduce an E2V-SDE whose dynamics are governed in a latent space by Stochastic differential equations (SDE). Therefore, E2V-SDE can rapidly reconstruct images at arbitrary time steps and make realistic predictions on unseen data. In addition, we successfully adopted a variety of image composition techniques for improving image clarity and temporal consistency. By conducting extensive experiments on simulated and real-scene datasets, we verify that our model outperforms state-of-the-art approaches under various video reconstruction settings. In terms of image quality, the LPIPS score improves by up to 12% and the reconstruction speed is 87% higher than that of ET-Net.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑