用于聚焦身体重复行为检测的深度多模态可穿戴传感器融合

中文摘要 AI 辅助

聚焦身体的重复行为（如拔毛、抠皮肤）是常与强迫症及焦虑障碍相关的强迫性动作，这类动作较为细微且与普通非病理性手势重叠，导致早期客观检测仍存在困难。我们开发并评估了一种多模态深度学习框架，用于从腕戴式传感器数据中检测和分类这些行为。数据由儿童心理研究所（Child Mind Institute）使用Helios腕戴设备采集，整合了惯性测量单元、热堆传感器和飞行时间传感器的数据，分别捕捉运动学、热学和邻近度信息。该框架结合了卷积神经网络与门控循环单元，以及模态特定的自动编码器和晚期融合分类器，以利用时间和空间动态信息。在二元检测任务中，该模型区分这些行为与其他活动时，F1分数达0.985，受试者工作特征曲线下面积达0.997；在区分各类单独行为与单一分组非目标类别的九分类方案中，其宏平均F1分数为0.700，曲线下面积为0.963，优于单模态基线模型。基于Shapley加性解释的事后可解释性分析显示，飞行时间和惯性模态通过捕捉空间邻近度和动态运动主导了判别能力，而层次聚类表明，误分类主要由手势的解剖区域驱动。这些发现证明，多模态传感器融合可实现准确、客观且连续的行为监测，为生物医学研究和临床护理中的实时可穿戴辅助心理健康诊断及个性化干预奠定了基础。

英文摘要

Body-focused repetitive behaviors, such as hair pulling and skin picking, are compulsive motor actions commonly associated with obsessive-compulsive and anxiety disorders. Their early, objective detection remains difficult because the movements are subtle and overlap with ordinary, non-pathological gestures. We developed and evaluated a multimodal deep learning framework to detect and classify these behaviors from wrist-worn sensor data. The data, collected by the Child Mind Institute using the Helios wrist-worn device, combine inertial measurement units, thermopile sensors, and time-of-flight sensors, capturing kinematic, thermal, and proximity information. The framework combined a convolutional neural network with a gated recurrent unit, alongside modality-specific autoencoders and a late-fusion classifier, to exploit temporal and spatial dynamics. It achieved an F1 score of 0.985 and an area under the receiver operating characteristic curve of 0.997 for binary detection, distinguishing these behaviors from other activities, and a macro-averaged F1 score of 0.700 with an area under the curve of 0.963 across a nine-class scheme that distinguished each individual behavior from a single grouped Non-Target class, improving over single-modality baselines. Post-hoc interpretability based on Shapley additive explanations showed that the time-of-flight and inertial modalities dominated discriminative power by capturing spatial proximity and dynamic movement, while hierarchical clustering indicated that misclassifications were driven primarily by the anatomical region of the gesture. These findings demonstrate that multimodal sensor fusion enables accurate, objective, and continuous behavioral monitoring. This work establishes a foundation for real-time, wearable-assisted mental health diagnostics and personalized interventions in biomedical research and clinical care.

发表机构

• George Mason University（乔治梅森大学）

机构由 AI 辅助整理，请以论文原文为准。