RFMask:一种简单的无线电信号人体轮廓分割基线
RFMask: A Simple Baseline for Human Silhouette Segmentation with Radio Signals
摘要:
人体轮廓分割最初是在计算机视觉中定义的,它在理解人体活动方面取得了很好的效果。然而,物理限制使得基于光学相机的现有系统在低照明、烟雾和/或不透明障碍条件下遭受严重的性能降级。为了克服这些局限性,在本文中,我们提出利用无线电信号,它可以穿越障碍物,不受光照条件的影响,实现轮廓分割。建议的RFMask框架由三个模块组成。它首先将毫米波雷达在两个平面上捕获的RF信号转换到空间域,并与信号处理模块一起抑制干扰。然后利用人体检测模块对射频帧上的人体反射进行定位,并从周围信号中提取特征。最后,从RF帧中提取的特征与基于注意力的掩模生成模块聚合。为了验证我们提出的框架,我们收集了一个包含804,760个无线电帧和402,380个摄像机帧的数据集,其中包含各种场景下的人类活动。实验结果表明,该框架可以实现令人印象深刻的人体轮廓分割,即使在具有挑战性的情况下(如低光和遮挡的情况下),传统的基于光学相机的方法失败。据我们所知,这是第一次研究对分割人体轮廓的基础上毫米波信号。我们希望我们的工作可以作为一个基线,并激发进一步的研究,用无线电信号执行视觉任务。数据集和代码将公开。
英文摘要:
Human silhouette segmentation, which is originally defined in computer vision, has achieved promising results for understanding human activities. However, the physical limitation makes existing systems based on optical cameras suffer from severe performance degradation under low illumination, smoke, and/or opaque obstruction conditions. To overcome such limitations, in this paper, we propose to utilize the radio signals, which can traverse obstacles and are unaffected by the lighting conditions to achieve silhouette segmentation. The proposed RFMask framework is composed of three modules. It first transforms RF signals captured by millimeter wave radar on two planes into spatial domain and suppress interference with the signal processing module. Then, it locates human reflections on RF frames and extract features from surrounding signals with human detection module. Finally, the extracted features from RF frames are aggregated with an attention based mask generation module. To verify our proposed framework, we collect a dataset containing 804,760 radio frames and 402,380 camera frames with human activities under various scenes. Experimental results show that the proposed framework can achieve impressive human silhouette segmentation even under the challenging scenarios(such as low light and occlusion scenarios) where traditional optical-camera-based methods fail. To the best of our knowledge, this is the first investigation towards segmenting human silhouette based on millimeter wave signals. We hope that our work can serve as a baseline and inspire further research that perform vision tasks with radio signals. The dataset and codes will be made in public.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。