评估15种欧盟官方语言的语言工具
Evaluating Language Tools for Fifteen EU-official Under-resourced Languages
摘要:
本文介绍了15个欧盟官方资源不足的语言可用的语言工具的评估活动的结果。该评估是在MSC ITN CLEOPATRA行动中进行的,该行动旨在在至少24种欧盟官方语言的语言处理链(LPC)应用程序的基础上建立以跨语言事件为中心的知识处理。在这次活动中,我们集中在三个现有的NLP平台(Stanford CoreNLP,NLP Cube,UDPipe)上,它们都为资源不足的语言提供了模型,在第一次运行中,我们涵盖了15种资源不足的语言。我们提出的评估活动的设计,并提出的结果,以及讨论它们。我们认为报告结果和我们检测结果之间的差异在单个百分点内,在可接受的公差范围内,因此认为该结果具有重现性。然而,对于一些语言,结果低于文献中报道的,在某些情况下,我们的测试结果甚至比以前报道的更好。特别成问题的是对全国教育资源中心系统的评价。原因之一是缺乏通用或跨语言适用的命名实体分类方案,该方案将在不同语言中服务于NERC任务,类似于解析任务中的通用依赖方案。建立这样的方案已成为我们未来的研究方向之一。
英文摘要:
This article presents the results of the evaluation campaign of language tools available for fifteen EU-official under-resourced languages. The evaluation was conducted within the MSC ITN CLEOPATRA action that aims at building the cross-lingual event-centric knowledge processing on top of the application of linguistic processing chains (LPCs) for at least 24 EU-official languages. In this campaign, we concentrated on three existing NLP platforms (Stanford CoreNLP, NLP Cube, UDPipe) that all provide models for under-resourced languages and in this first run we covered 15 under-resourced languages for which the models were available. We present the design of the evaluation campaign and present the results as well as discuss them. We considered the difference between reported and our tested results within a single percentage point as being within the limits of acceptable tolerance and thus consider this result as reproducible. However, for a number of languages, the results are below what was reported in the literature, and in some cases, our testing results are even better than the ones reported previously. Particularly problematic was the evaluation of NERC systems. One of the reasons is the absence of universally or cross-lingually applicable named entities classification scheme that would serve the NERC task in different languages analogous to the Universal Dependency scheme in parsing task. To build such a scheme has become one of our the future research directions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。