复合词切分与惯用语复合词检测的联合方法
A Joint Approach to Compound Splitting and Idiomatic Compound Detection
摘要:
诸如机器翻译、语音识别和信息检索之类的应用需要有效地处理名词复合词,因为它们是词汇表外(OOV)词的可能来源之一。对名词复合词的深入处理不仅需要将它们分解成更小的成分(甚至词根),而且还需要识别那些不应该被分解的实例,因为它们具有惯用的性质。我们为德语开发了一种基于双重深度学习的名词复合词拆分和惯用复合词检测方法,我们使用新收集的注释德语复合词语料库进行训练。我们的神经名词复合拆分器在子单词级别上运行,比当前最先进的拆分器性能高出约5%。
英文摘要:
Applications such as machine translation, speech recognition, and information retrieval require efficient handling of noun compounds as they are one of the possible sources for out-of-vocabulary (OOV) words. In-depth processing of noun compounds requires not only splitting them into smaller components (or even roots) but also the identification of instances that should remain unsplitted as they are of idiomatic nature. We develop a two-fold deep learning-based approach of noun compound splitting and idiomatic compound detection for the German language that we train using a newly collected corpus of annotated German compounds. Our neural noun compound splitter operates on a sub-word level and outperforms the current state of the art by about 5%.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。