arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.03407cs.CL
收藏

基于Transformer与自然语言处理的在线手势识别

Online Gesture Recognition using Transformer and Natural Language Processing

University College Dublin(都柏林大学学院);University of Pisa(比萨大学)

G. C. M. Silvestre, F. Balado, O. Akinremi, M. Ramo

AI总结:

本研究提出基于Transformer的在线手写手势识别方法,结合注意力机制、BPE大解码空间与多语言迁移学习,在新数据集上实现英法德94%-96%的识别准确率。

摘要:

研究表明,Transformer架构为对应自然语言句子字形笔画的在线手写手势提供了强大的机器转导框架。注意力机制被成功用于构建端到端编码器-解码器模型的潜在表示,在解决多级分割问题的同时还能学习部分语言特征和句法规则。研究证实,额外采用带有学习得到的字节对编码(BPE)的大解码空间,可提升对消融输入和句法规则的鲁棒性。编码器栈直接接收时空数据词元,这些词元可构成无限大的输入词表,该方法的应用场景可拓展至本研究之外。研究还验证了编码器在多种语言上的迁移学习能力,可实现更快的优化与参数共享。本研究使用一个适用于通用手写识别任务的新型在线手写手势有监督数据集,成功训练出一个小型Transformer模型,其在英文或德文句子上的平均归一化Levenshtein准确率达96%,在法文上达94%。

英文摘要:

The Transformer architecture is shown to provide a powerful machine transduction framework for online handwritten gestures corresponding to glyph strokes of natural language sentences. The attention mechanism is successfully used to create latent representations of an end-to-end encoder-decoder model, solving multi-level segmentation while also learning some language features and syntax rules. The additional use of a large decoding space with some learned Byte-Pair-Encoding (BPE) is shown to provide robustness to ablated inputs and syntax rules. The encoder stack was directly fed with spatio-temporal data tokens potentially forming an infinitely large input vocabulary, an approach that finds applications beyond that of this work. Encoder transfer learning capabilities is also demonstrated on several languages resulting in faster optimisation and shared parameters. A new supervised dataset of online handwriting gestures suitable for generic handwriting recognition tasks was used to successfully train a small transformer model to an average normalised Levenshtein accuracy of 96% on English or German sentences and 94% in French.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑