arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2002.01207cs.CLcs.LG
收藏

使用具有丰富数据的biLSTM模型恢复阿拉伯语变音符号

Arabic Diacritic Recovery Using a Feature-Rich biLSTM Model

Qatar Computing Research Institute(卡塔尔计算研究所);Hamad Bin Khalifa University(哈马德·本·哈利法大学)

Kareem Darwish, Ahmed Abdelali, Hamdy Mubarak, Mohamed Eldesouki

摘要:

变音符号(短元音)通常在书写阿拉伯语文本时被省略,读者必须重新引入它们才能正确发音。阿拉伯语的变音符号有两种类型:第一种是核心词变音符号(CW),它指定词汇选择;第二种是格尾(CE),它通常出现在词干的末尾,通常指定它们的句法角色。恢复CE比恢复核心词变音符号相对更难,因为词间依赖性通常是遥远的。在本文中,我们使用一个功能丰富的递归神经网络模型,使用各种语言和表面水平的功能,以恢复核心词的变音符号和情况下结束。我们的模型超越了所有以前的国家的最先进的系统与CW出错率(CWER)的2.86\%和CE出错率(CEER)的3.7%的现代标准阿拉伯语(MSA)和2.2%的CWER和2.5%的古典阿拉伯语(CA)的CEER。当将变音词核与格尾相结合时,MSA和CA的词出错率分别为6.0%和4.3%。这突出了这种深度神经模型的特征工程的有效性。

英文摘要:

Diacritics (short vowels) are typically omitted when writing Arabic text, and readers have to reintroduce them to correctly pronounce words. There are two types of Arabic diacritics: the first are core-word diacritics (CW), which specify the lexical selection, and the second are case endings (CE), which typically appear at the end of the word stem and generally specify their syntactic roles. Recovering CEs is relatively harder than recovering core-word diacritics due to inter-word dependencies, which are often distant. In this paper, we use a feature-rich recurrent neural network model that uses a variety of linguistic and surface-level features to recover both core word diacritics and case endings. Our model surpasses all previous state-of-the-art systems with a CW error rate (CWER) of 2.86\% and a CE error rate (CEER) of 3.7% for Modern Standard Arabic (MSA) and CWER of 2.2% and CEER of 2.5% for Classical Arabic (CA). When combining diacritized word cores with case endings, the resultant word error rate is 6.0% and 4.3% for MSA and CA respectively. This highlights the effectiveness of feature engineering for such deep neural models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑