arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.09124eess.AScs.SD
收藏

基于变换器的多传感器融合多视角多模态事件检测

Multi-view and Multi-modal Event Detection Utilizing Transformer-based Multi-sensor fusion

NTT Corporation(NTT公司)

Masahiro Yasuda, Yasunori Ohishi, Shoichiro Saito, Noboru Harada

摘要:

我们面临着一项具有挑战性的任务:多视图和多模态事件检测,通过利用来自分布式摄像机和麦克风及其弱标签的数据来检测大范围真实环境中的事件。在这项任务中,分布式传感器被互补地用于捕获难以用单个传感器捕获的事件,例如在复杂房间中移动的人的一系列动作,或者在房间中相距很远的人之间的通信。为了使传感器在这种情况下有效合作,系统应该能够在传感器之间交换信息,并以互补的方式组合对识别事件有用的信息。对于这样一个机制,我们提出了一个基于变换器的多传感器融合(MultiTrans),它结合了多传感器数据的基础上,不同的观点和方式的功能之间的关系。在使用为此任务新收集的数据集的实验中,我们提出的使用MultiTrans的方法提高了事件检测性能并优于比较。

英文摘要:

We tackle a challenging task: multi-view and multi-modal event detection that detects events in a wide-range real environment by utilizing data from distributed cameras and microphones and their weak labels. In this task, distributed sensors are utilized complementarily to capture events that are difficult to capture with a single sensor, such as a series of actions of people moving in an intricate room, or communication between people located far apart in a room. For sensors to cooperate effectively in such a situation, the system should be able to exchange information among sensors and combines information that is useful for identifying events in a complementary manner. For such a mechanism, we propose a Transformer-based multi-sensor fusion (MultiTrans) which combines multi-sensor data on the basis of the relationships between features of different viewpoints and modalities. In the experiments using a dataset newly collected for this task, our proposed method using MultiTrans improved the event detection performance and outperformed comparatives.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑