IMUFace:基于耳机惯性传感的常开式3D面部重建研究
IMUFace: Towards Always-On 3D Facial Reconstruction via Earphone Inertial Sensing
AI总结:
针对现有耳戴式面部重建方案美观性与功耗不足的问题,本文提出基于耳机IMU的IMUFace系统,结合IMUTwinTrans模型实现低功耗常开3D面部重建,实验验证了其精度与实用价值。
摘要:
面部表情重建技术应用潜力巨大,可覆盖人机交互、情感计算、虚拟现实等多个领域。近期已有研究提出利用耳戴设备实现面部表情重建,以解决传统基于摄像头的方法存在的环境限制与隐私隐患问题,但这类方案在美观性和功耗方面仍有待提升。本文提出名为IMUFace的系统,该系统利用嵌入无线耳机的惯性测量单元(IMU)检测面部肌肉活动引发的细微耳部运动,可实现隐蔽、低功耗的面部重建。研究开展了包含12名参与者的用户实验,同时提出名为IMUTwinTrans的深度学习模型。实验结果显示,仅需5分钟训练数据,IMUFace就能以2.21毫米的精度准确预测用户面部关键点,预测得到的关键点可用于重建三维面部模型。IMUFace运行采样率为30Hz,功耗仅为58mW,相对较低。本研究的结果验证了IMUFace的实际应用可行性,同时指出了后续推动其落地应用的潜在研究方向。
英文摘要:
The potential of facial expression reconstruction technology is significant, with applications in various fields such as human-computer interaction, affective computing, and virtual reality. Recent studies have proposed using ear-worn devices for facial expression reconstruction to address the environmental limitations and privacy concerns associated with traditional camera-based methods. However, these approaches still require improvements in terms of aesthetics and power consumption. This paper introduces a system called IMUFace. It uses inertial measurement units (IMUs) embedded in wireless earphones to detect subtle ear movements caused by facial muscle activities, allowing for covert and low-power facial reconstruction. A user study involving 12 participants was conducted, and a deep learning model named IMUTwinTrans was proposed. The results show that IMUFace can accurately predict users' facial landmarks with a precision of 2.21 mm, using only five minutes of training data. The predicted landmarks can be utilized to reconstruct a three-dimensional facial model. IMUFace operates at a sampling rate of 30 Hz with a relatively low power consumption of 58 mW. The findings presented in this study demonstrate the real-world applicability of IMUFace and highlight potential directions for further research to facilitate its practical adoption.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。