InMyFace：基于惯性传感与肌机械图（肌音）融合的可穿戴面部活动识别
InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition
AI总结：
针对可穿戴场景中摄像头不便的问题，本文提出融合惯性、压力与肌音传感器的面部活动识别系统，实验显示跨用户F1达79%，性能与最先进非摄像头方案相当。
摘要：
面部活动识别是一个已被充分理解但并非平凡（即具有一定难度）的计算机视觉问题。然而，可靠的解决方案通常需要摄像头能够良好地捕捉面部，而这在可穿戴环境中往往难以实现。此外，在可穿戴应用中，系统伴随用户进行日常活动，持续运行的摄像头可能引发隐私（及法律）方面的问题。本研究提出了一种替代方案，该方案基于可穿戴惯性传感器、平面压力传感器以及声学肌机械图（肌肉声音）的融合。传感器被隐蔽地放置在运动帽中，以监测与面部表情相关的面部肌肉活动。我们展示了集成的可穿戴传感器系统，描述了数据融合与分析方法，并在一个包含来自不同文化背景（八个国家）及不同性别（六名女性和七名男性）的十三名受试者的实验中对该系统进行了评估。在“每用户一模型”方案下，采用晚期融合方法，当所有感知模态组合时，系统取得了平均F1分数85.00%。在跨用户验证及“所有用户共用一个模型”方案下，十三名参与者（六名女性和七名男性）获得了F1分数79.00%。此外，在混合融合（跨用户）方法及六类分类任务中，八名用户获得了平均F1分数82.00%。这些结果与最先进的非摄像头解决方案在跨用户研究中具有竞争力。此外，我们独特的参与者群体证明了该方法的包容性和泛化能力。
英文摘要：
Recognizing facial activity is a well-understood (but non-trivial) computer vision problem. However, reliable solutions require a camera with a good view of the face, which is often unavailable in wearable settings. Furthermore, in wearable applications, where systems accompany users throughout their daily activities, a permanently running camera can be problematic for privacy (and legal) reasons. This work presents an alternative solution based on the fusion of wearable inertial sensors, planar pressure sensors, and acoustic mechanomyography (muscle sounds). The sensors were placed unobtrusively in a sports cap to monitor facial muscle activities related to facial expressions. We present our integrated wearable sensor system, describe data fusion and analysis methods, and evaluate the system in an experiment with thirteen subjects from different cultural backgrounds (eight countries) and both sexes (six women and seven men). In a one-model-per-user scheme and using a late fusion approach, the system yielded an average F1 score of 85.00% for the case where all sensing modalities are combined. With a cross-user validation and a one-model-for-all-user scheme, an F1 score of 79.00% was obtained for thirteen participants (six females and seven males). Moreover, in a hybrid fusion (cross-user) approach and six classes, an average F1 score of 82.00% was obtained for eight users. The results are competitive with state-of-the-art non-camera-based solutions for a cross-user study. In addition, our unique set of participants demonstrates the inclusiveness and generalizability of the approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
