一种用于自动多模态人类活动识别系统的视觉与惯性传感器数据双流决策级融合新方法
A Novel Two Stream Decision Level Fusion of Vision and Inertial Sensors Data for Automatic Multimodal Human Activity Recognition System
AI总结：
本文提出一种基于视觉与惯性传感器数据的双流决策级融合多模态人类活动识别系统，通过结合CNN与正则化LSTM提取时空特征，在四个基准数据集上取得了优于现有方法的准确率。
摘要：
本文提出了一种新颖的多模态人类活动识别系统。该系统采用了视觉与惯性传感器的双流决策级融合。在第一个流中，原始RGB帧被传递给基于部分亲和场的姿态估计网络，以检测用户的关键点。这些关键点随后被预处理，并以滑动窗口的方式输入到一个专门设计的卷积神经网络中进行空间特征提取，接着使用正则化LSTM来计算时间特征。LSTM网络的输出随后被输入到全连接层进行分类。在第二个流中，从惯性传感器获取的数据被预处理并输入到正则化LSTM中进行特征提取，随后通过全连接层进行分类。在此阶段，两个流的SoftMax分数通过决策级融合进行合并，从而给出最终的预测。本文进行了广泛的实验以评估性能。实验使用了四个多模态标准基准数据集（UP-Fall detection、UTD-MHAD、Berkeley-MHAD和C-MHAD）。所提出的系统在UP-Fall Detection、UTD-MHAD、Berkeley-MHAD和C-MHAD数据集上分别获得了96.9%、97.6%、98.7%和95.9%的准确率。这些结果远优于当前最先进的方法。
英文摘要：
This paper presents a novel multimodal human activity recognition system. It uses a two-stream decision level fusion of vision and inertial sensors. In the first stream, raw RGB frames are passed to a part affinity field-based pose estimation network to detect the keypoints of the user. These keypoints are then pre-processed and inputted in a sliding window fashion to a specially designed convolutional neural network for the spatial feature extraction followed by regularized LSTMs to calculate the temporal features. The outputs of LSTM networks are then inputted to fully connected layers for classification. In the second stream, data obtained from inertial sensors are pre-processed and inputted to regularized LSTMs for the feature extraction followed by fully connected layers for the classification. At this stage, the SoftMax scores of two streams are then fused using the decision level fusion which gives the final prediction. Extensive experiments are conducted to evaluate the performance. Four multimodal standard benchmark datasets (UP-Fall detection, UTD-MHAD, Berkeley-MHAD, and C-MHAD) are used for experimentations. The accuracies obtained by the proposed system are 96.9 %, 97.6 %, 98.7 %, and 95.9 % respectively on the UP-Fall Detection, UTDMHAD, Berkeley-MHAD, and C-MHAD datasets. These results are far superior than the current state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
