LEAN-LIFE:一个从解释中学习的标签有效标注框架
LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation
摘要:
成功训练深度神经网络需要大量的标记数据。然而,每个标签只提供有限的信息来学习,并且收集必要数量的标签需要大量的人力。在这项工作中,我们介绍了LEAN-LIFE,一个基于Web的,标签高效的序列标签和分类任务的注释框架,具有易于使用的UI,不仅允许注释器为任务提供所需的标签,而且还可以为每个标签决策提供学习。这样的解释使我们能够从未标记的实例中生成有用的附加标记数据,从而支持可用的训练数据池。在三个流行的NLP任务(命名实体识别,关系抽取,情感分析)中,我们发现使用这种增强的监督可以使我们的模型超过竞争基准F1分数超过5-10个百分点,同时使用2倍更少的标记实例。我们的框架是第一个利用这种增强的监督技术,并这样做了三个重要的任务-从而为用户提供改进的注释建议,并能够构建(数据,标签,解释)三元组而不是常规(数据,标签)对的数据集。
英文摘要:
Successfully training a deep neural network demands a huge corpus of labeled data. However, each label only provides limited information to learn from and collecting the requisite number of labels involves massive human effort. In this work, we introduce LEAN-LIFE, a web-based, Label-Efficient AnnotatioN framework for sequence labeling and classification tasks, with an easy-to-use UI that not only allows an annotator to provide the needed labels for a task, but also enables LearnIng From Explanations for each labeling decision. Such explanations enable us to generate useful additional labeled data from unlabeled instances, bolstering the pool of available training data. On three popular NLP tasks (named entity recognition, relation extraction, sentiment analysis), we find that using this enhanced supervision allows our models to surpass competitive baseline F1 scores by more than 5-10 percentage points, while using 2X times fewer labeled instances. Our framework is the first to utilize this enhanced supervision technique and does so for three important tasks -- thus providing improved annotation recommendations to users and an ability to build datasets of (data, label, explanation) triples instead of the regular (data, label) pair.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。