arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.02376cs.CVcs.LG
收藏

基于事件的混合网络合成孔径成像

Event-based Synthetic Aperture Imaging with a Hybrid Network

Wuhan University(武汉大学)

Xiang Zhang, Wei Liao, Lei Yu, Wen Yang, Gui-Song Xia

摘要:

合成孔径成像(SAI)通过模糊掉离焦的前景遮挡,从多视点图像中重建出聚焦的遮挡目标,从而达到透视效果。然而,非常密集的遮挡和极端光照条件可能会对基于传统帧的相机的SAI带来显著干扰,导致性能退化。为了解决这些问题,我们提出了一种新的SAI系统的基础上的事件相机,可以产生异步事件具有极低的延迟和高动态范围。因此,它可以通过几乎连续的视图测量来消除密集遮挡的干扰,同时解决过/欠曝光问题。为了重建被遮挡的目标,我们提出了一个混合的编码器-解码器网络组成的脉冲神经网络(SNNs)和卷积神经网络(CNN)。在混合网络中,收集到的事件的时空信息首先通过SNN层编码,然后通过风格转换CNN解码器转换为被遮挡目标的视觉图像。实验结果表明,该方法在处理密集遮挡和极端光照条件下表现出了良好的性能,能够利用纯事件数据重建出高质量的视觉图像。

英文摘要:

Synthetic aperture imaging (SAI) is able to achieve the see through effect by blurring out the off-focus foreground occlusions and reconstructing the in-focus occluded targets from multi-view images. However, very dense occlusions and extreme lighting conditions may bring significant disturbances to the SAI based on conventional frame-based cameras, leading to performance degeneration. To address these problems, we propose a novel SAI system based on the event camera which can produce asynchronous events with extremely low latency and high dynamic range. Thus, it can eliminate the interference of dense occlusions by measuring with almost continuous views, and simultaneously tackle the over/under exposure problems. To reconstruct the occluded targets, we propose a hybrid encoder-decoder network composed of spiking neural networks (SNNs) and convolutional neural networks (CNNs). In the hybrid network, the spatio-temporal information of the collected events is first encoded by SNN layers, and then transformed to the visual image of the occluded targets by a style-transfer CNN decoder. Through experiments, the proposed method shows remarkable performance in dealing with very dense occlusions and extreme lighting conditions, and high quality visual images can be reconstructed using pure event data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑