指尖检测与跟踪在视频手写体识别中的应用
Fingertip Detection and Tracking for Recognition of Air-Writing in Videos
摘要：
空气书写是在没有任何手持设备的帮助下，使用手指或手的运动在自由空间中书写字符或单词的过程。在这项工作中，我们解决了使用网络摄像头视频作为输入的半空中手指书写的问题。尽管最近在目标检测和跟踪方面取得了进展，但指尖的准确且鲁棒的检测和跟踪仍然是一项具有挑战性的任务，这主要是由于指尖的尺寸小。此外，由于缺乏任何标准的划界准则，空中手指书写的初始化和终止也具有挑战性。为了解决这些问题，我们提出了一种新的书写手姿态检测算法，用于使用Faster R-CNN框架进行空气书写的初始化，以进行准确的手检测，然后进行手分割，最后根据手的几何特性计算抬起的手指的数量。此外，我们提出了一个强大的指尖检测和跟踪方法，使用一个新的签名函数称为距离加权曲率熵。最后，基于指尖速度的终止准则被用作标记空气书写手势完成的标志。实验表明，所提出的指尖检测和跟踪算法的优越性，在国家的最先进的方法，平均精度为73.1%，同时实现实时性能在18.5 fps，这是至关重要的条件，空气书写。字符识别实验给出了一个平均准确率为96.11%，使用建议的空气书写系统，这是一个结果，这是现有的手写字符识别系统。
英文摘要：
Air-writing is the process of writing characters or words in free space using finger or hand movements without the aid of any hand-held device. In this work, we address the problem of mid-air finger writing using web-cam video as input. In spite of recent advances in object detection and tracking, accurate and robust detection and tracking of the fingertip remains a challenging task, primarily due to small dimension of the fingertip. Moreover, the initialization and termination of mid-air finger writing is also challenging due to the absence of any standard delimiting criterion. To solve these problems, we propose a new writing hand pose detection algorithm for initialization of air-writing using the Faster R-CNN framework for accurate hand detection followed by hand segmentation and finally counting the number of raised fingers based on geometrical properties of the hand. Further, we propose a robust fingertip detection and tracking approach using a new signature function called distance-weighted curvature entropy. Finally, a fingertip velocity-based termination criterion is used as a delimiter to mark the completion of the air-writing gesture. Experiments show the superiority of the proposed fingertip detection and tracking algorithm over state-of-the-art approaches giving a mean precision of 73.1 % while achieving real-time performance at 18.5 fps, a condition which is of vital importance to air-writing. Character recognition experiments give a mean accuracy of 96.11 % using the proposed air-writing system, a result which is comparable to that of existing handwritten character recognition systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
