硕士论文：学习Tokenization的神经手语翻译
Master Thesis: Neural Sign Language Translation by Learning Tokenization
摘要：
在这篇论文中，我们提出了一种基于多任务学习的方法来改进神经手语翻译（NMT），该方法包括两个部分，一个标记层和神经机器翻译（NMT）。标记化部分侧重于如何表示手语（SL）视频以将其输入另一部分。它还没有被仔细研究，而NMT研究吸引了一些研究人员，贡献了巨大的进步。到目前为止，有两个主要的输入标记化级别，即帧级和注释级标记化。注释是类似于世界的中间表达，是SL所特有的。因此，我们的目标是开发一个通用的符号级标记化层，以便它适用于其他领域，而无需进一步的努力。我们首先研究当前的标记化方法，并通过几个实验解释它们的弱点。为了提供解决方案，我们将迁移学习、多任务学习和无监督领域自适应应用到这项研究中，以利用额外的监督。我们成功地实现了SL之间的知识转移，并在BLEU-4中提高了5分，在ROUGE中提高了8分。其次，我们通过大量的实验，在所有的标记化方法的身体部位的效果。除此之外，我们还采用3D-CNN来提高时间和空间方面的效率。最后，我们讨论了符号级标记化相对于注释级标记化的优势。综上所述，我们提出的方法消除了光泽度注释，以获得更高的分数，通过利用弱监督源提供额外的监督。
英文摘要：
In this thesis, we propose a multitask learning based method to improve Neural Sign Language Translation (NSLT) consisting of two parts, a tokenization layer and Neural Machine Translation (NMT). The tokenization part focuses on how Sign Language (SL) videos should be represented to be fed into the other part. It has not been studied elaborately whereas NMT research has attracted several researchers contributing enormous advancements. Up to now, there are two main input tokenization levels, namely frame-level and gloss-level tokenization. Glosses are world-like intermediate presentation and unique to SLs. Therefore, we aim to develop a generic sign-level tokenization layer so that it is applicable to other domains without further effort. We begin with investigating current tokenization approaches and explain their weaknesses with several experiments. To provide a solution, we adapt Transfer Learning, Multitask Learning and Unsupervised Domain Adaptation into this research to leverage additional supervision. We succeed in enabling knowledge transfer between SLs and improve translation quality by 5 points in BLEU-4 and 8 points in ROUGE scores. Secondly, we show the effects of body parts by extensive experiments in all the tokenization approaches. Apart from these, we adopt 3D-CNNs to improve efficiency in terms of time and space. Lastly, we discuss the advantages of sign-level tokenization over gloss-level tokenization. To sum up, our proposed method eliminates the need for gloss level annotation to obtain higher scores by providing additional supervision by utilizing weak supervision sources.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
