自然语言推理的多源主动学习研究
Investigating Multi-source Active Learning for Natural Language Inference
摘要：
近年来，主动学习已成功地应用于一系列的NLP任务。然而，以前的工作通常假设训练和测试数据来自相同的分布。这是有问题的，因为在现实生活中，数据可能来自不同相关性和质量的几个来源。我们发现，四个流行的主动学习计划未能优于随机选择时，适用于由多个数据源组成的未标记池的任务上的自然语言推理。我们发现，基于不确定性的策略表现不佳，由于收购集体离群值，即，难以学习的实例，妨碍学习和概括。当离群值被移除时，发现策略可以恢复并优于随机基线。在进一步的分析中，我们发现集体离群值在不同来源之间的形式不同，并表明难以学习的数据并不总是绝对有害的。最后，我们利用数据集制图来引入难度分层测试，并发现不同的策略受到示例可学习性和难度的不同影响。
英文摘要：
In recent years, active learning has been successfully applied to an array of NLP tasks. However, prior work often assumes that training and test data are drawn from the same distribution. This is problematic, as in real-life settings data may stem from several sources of varying relevance and quality. We show that four popular active learning schemes fail to outperform random selection when applied to unlabelled pools comprised of multiple data sources on the task of natural language inference. We reveal that uncertainty-based strategies perform poorly due to the acquisition of collective outliers, i.e., hard-to-learn instances that hamper learning and generalization. When outliers are removed, strategies are found to recover and outperform random baselines. In further analysis, we find that collective outliers vary in form between sources, and show that hard-to-learn data is not always categorically harmful. Lastly, we leverage dataset cartography to introduce difficulty-stratified testing and find that different strategies are affected differently by example learnability and difficulty.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
