In many languages like Arabic, diacritics are used to specify pronunciations as well as meanings. Such diacritics are often omitted in written text, increasing the number of possible pronunciations and meanings for a word. This results in a more ambiguous text making computational processing on such text more difficult. Diacritic restoration is the task of restoring missing diacritics in the written text. Most state-of-the-art diacritic restoration models are built on character level information which helps generalize the model to unseen data, but presumably lose useful information at the word level. Thus, to compensate for this loss, we investigate the use of multi-task learning to jointly optimize diacritic restoration with related NLP problems namely word segmentation, part-of-speech tagging, and syntactic diacritization. We use Arabic as a case study since it has sufficient data resources for tasks that we consider in our joint modeling. Our joint models significantly outperform the baselines and are comparable to the state-of-the-art models that are more complex relying on morphological analyzers and/or a lot more data (e.g. dialectal data).
Efficient Convolutional Neural Networks for Diacritic Restoration
AWS, Amazon AI（亚马逊人工智能 AWS）；The George Washington University（乔治华盛顿大学）；Princess Nourah Bint Abdul Rahman University（诺拉·宾特·阿卜杜勒拉赫曼公主大学）
随着机器理解书面文本的需求不断增长，变音符号恢复变得越来越重要。该任务通常被建模为序列标记问题，目前双向长短期记忆（BiLSTM）模型提供了最先进的结果。最近，Bai et al.（2018）展示了时间卷积神经网络（TCN）在性能和计算资源方面优于递归神经网络（RNN）的序列建模优势。由于变音恢复的好处，从以前的以及随后的时间步，我们进一步应用和评估的TCN的一个变种，tuminusal TCN（A-TCN），它结合了上下文中的两个方向（以前和未来），而不是严格纳入以前的上下文的情况下，TCN。A-TCN在三种不同语言（阿拉伯语、约鲁巴语和越南语）的变音符号化方面比TCN有显著的改进。此外，A-TCN和BiLSTM具有相当的性能，使A-TCN成为BiLSTM的有效替代方案，因为卷积可以并行训练。A-TCN在推理时间上明显快于BiLSTM（每分钟变音文本量提高了270%-334%）。
