arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.12768cs.CV
收藏

DA4Event:使用领域自适应弥合事件摄像机的模拟与真实差距

DA4Event: towards bridging the Sim-to-Real Gap for Event Cameras using Domain Adaptation

Politecnico di Torino(都灵理工大学);Istituto Italiano di Tecnologia(意大利技术研究院);Politecnico di Milano(米兰理工大学)

Mirco Planamente, Chiara Plizzari, Marco Cannici, Marco Ciccone, Francesco Strada, Andrea Bottino, Matteo Matteucci, Barbara Caputo

摘要:

事件相机是一种新型的生物传感器,它以“事件”的形式异步捕获像素级的强度变化。与标准设备相比,它们获取数据的创新方式具有多项优势,尤其是在光线不足和高速运动的条件下。然而,这些传感器的新颖性导致缺乏能够充分释放其潜力的大量训练数据。研究人员解决这个问题的最常见方法是利用模拟事件数据。然而,这种方法带来了一个开放的研究问题:模拟数据如何推广到真实数据?为了回答这个问题,我们建议利用,在基于事件的背景下,最近的领域自适应(DA)在传统的计算机视觉的进展,表明DA技术应用于事件数据有助于减少模拟到真实的差距。为此,我们提出了一种新的架构,我们称之为多视图DA 4 E(MV-DA 4 E),更好地利用基于帧的事件表示的特点,同时也促进域不变特征。通过大量的实验,我们证明了DA方法和MV-DA 4 E在N-Caltech 101上的有效性。此外,我们通过对流行的RGB-D对象数据集(ROD)进行跨域分析,验证了它们在现实世界中的可靠性,并将其扩展到事件模态(RGB-E)。

英文摘要:

Event cameras are novel bio-inspired sensors, which asynchronously capture pixel-level intensity changes in the form of "events". The innovative way they acquire data presents several advantages over standard devices, especially in poor lighting and high-speed motion conditions. However, the novelty of these sensors results in the lack of a large amount of training data capable of fully unlocking their potential. The most common approach implemented by researchers to address this issue is to leverage simulated event data. Yet, this approach comes with an open research question: how well simulated data generalize to real data? To answer this, we propose to exploit, in the event-based context, recent Domain Adaptation (DA) advances in traditional computer vision, showing that DA techniques applied to event data help reduce the sim-to-real gap. To this purpose, we propose a novel architecture, which we call Multi-View DA4E (MV-DA4E), that better exploits the peculiarities of frame-based event representations while also promoting domain invariant characteristics in features. Through extensive experiments, we prove the effectiveness of DA methods and MV-DA4E on N-Caltech101. Moreover, we validate their soundness in a real-world scenario through a cross-domain analysis on the popular RGB-D Object Dataset (ROD), which we extended to the event modality (RGB-E).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑