arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.15239cs.HCcs.AI
收藏

在腕戴式设备上实现手势自定义

Enabling hand gesture customization on wrist-worn devices

Apple Inc.(苹果公司);CMU(卡内基梅隆大学)
Xuhai Xu, Jun Gong, Carolina Brum, Lilian Liang, Bongsoo Suh, Kumar Gupta, Yash Agarwal, Laurence Lindsey, Runchang Kang, Behrooz Shahsavari, Tu Nguyen, Heriber… 展开作者

Xuhai Xu, Jun Gong, Carolina Brum, Lilian Liang, Bongsoo Suh, Kumar Gupta, Yash Agarwal, Laurence Lindsey, Runchang Kang, Behrooz Shahsavari, Tu Nguyen, Heriberto Nieto, Scott E. Hudson, Charlie Maalouf, Seyed Mousavi, Gierad Laput

摘要:

我们提出了一个框架,手势定制需要最少的例子,从用户,所有没有降低现有的手势集的性能。为了实现这一目标,我们首先部署了一个大规模的研究(N=500+)来收集数据和训练一个加速度计-陀螺仪识别模型,在日常非手势数据上测试时,该模型的跨用户准确率为95.7%,误报率为每小时0.6。接下来,我们设计了一个少样本学习框架,该框架从我们的预训练模型中派生出一个轻量级模型,从而在不降低性能的情况下实现知识转移。我们通过一项用户研究(N=20)验证了我们的方法,该研究从12个新手势中检查设备上的自定义,在添加新手势时使用一个,三个或五个镜头的平均准确率分别为55.3%,83.1%和87.2%,同时保持与现有手势集相同的识别准确率和误报率。我们进一步评估我们的实时实现的可用性与用户体验研究(N=20)。我们的研究结果突出了我们的定制框架的有效性,可学习性和可用性。我们的方法为未来铺平了道路,用户不再受现有手势的束缚,可以根据自己的偏好和能力创造性地引入新的手势。

英文摘要:

We present a framework for gesture customization requiring minimal examples from users, all without degrading the performance of existing gesture sets. To achieve this, we first deployed a large-scale study (N=500+) to collect data and train an accelerometer-gyroscope recognition model with a cross-user accuracy of 95.7% and a false-positive rate of 0.6 per hour when tested on everyday non-gesture data. Next, we design a few-shot learning framework which derives a lightweight model from our pre-trained model, enabling knowledge transfer without performance degradation. We validate our approach through a user study (N=20) examining on-device customization from 12 new gestures, resulting in an average accuracy of 55.3%, 83.1%, and 87.2% on using one, three, or five shots when adding a new gesture, while maintaining the same recognition accuracy and false-positive rate from the pre-existing gesture set. We further evaluate the usability of our real-time implementation with a user experience study (N=20). Our results highlight the effectiveness, learnability, and usability of our customization framework. Our approach paves the way for a future where users are no longer bound to pre-existing gestures, freeing them to creatively introduce new gestures tailored to their preferences and abilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑