arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1810.03450cs.CLcs.AI
收藏

面向自然语言理解新领域的主动学习

Active Learning for New Domains in Natural Language Understanding

Alexa Machine Learning, Amazon.com(亚马逊Alexa机器学习团队(Amazon Alexa Machine Learning),亚马逊公司(Amazon.com))

Stanislav Peshterliev, John Kearney, Abhyuday Jagannatha, Imre Kiss, Spyros Matsoukas

摘要:

我们探索主动学习(AL),以提高自然语言理解(NLU)系统中新领域的准确性。我们提出了一种算法,称为多数-CRF,使用分类模型的合奏,以指导相关的话语的选择,以及一个序列标记模型,以帮助优先信息的例子。三个领域的实验表明,多数CRF实现了6.6%-9%的相对出错率减少随机抽样与相同的注释预算相比,统计上显着的改善相比,其他AL方法。此外,六个新领域的人类在环AL案例研究显示,现有NLU系统的性能提高了4.6%-9%。

英文摘要:

We explore active learning (AL) for improving the accuracy of new domains in a natural language understanding (NLU) system. We propose an algorithm called Majority-CRF that uses an ensemble of classification models to guide the selection of relevant utterances, as well as a sequence labeling model to help prioritize informative examples. Experiments with three domains show that Majority-CRF achieves 6.6%-9% relative error rate reduction compared to random sampling with the same annotation budget, and statistically significant improvements compared to other AL approaches. Additionally, case studies with human-in-the-loop AL on six new domains show 4.6%-9% improvement on an existing NLU system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑