自然语言任务中的多领域主动学习
Active Learning Over Multiple Domains in Natural Language Tasks
摘要:
主动学习的研究传统上假设目标数据和源数据来自单一领域。然而,在实际应用中,从业者通常需要使用多个分布外数据源进行主动学习,其中先验地不清楚哪些数据源将帮助或损害目标域。我们调查了各种各样的技术在主动学习(AL),域偏移检测(DS),和多域采样,以检查这个具有挑战性的设置为问答和情感分析。我们要问:(1)什么样的方法对这项任务有效?以及,(2)所选的例子和域的哪些属性实现了强结果?在来自4个方法家族的18个采集函数中,我们发现H-Divergence方法,特别是我们提出的变体DAL-E,产生了有效的结果,平均比随机基线提高了2-3%。我们还显示了一个不同的分配域的重要性,以及现有的方法域和例子选择的改进空间。我们的研究结果产生了第一个全面的分析现有的和新的方法,从业者面临多领域的主动学习自然语言任务。
英文摘要:
Studies of active learning traditionally assume the target and source data stem from a single domain. However, in realistic applications, practitioners often require active learning with multiple sources of out-of-distribution data, where it is unclear a priori which data sources will help or hurt the target domain. We survey a wide variety of techniques in active learning (AL), domain shift detection (DS), and multi-domain sampling to examine this challenging setting for question answering and sentiment analysis. We ask (1) what family of methods are effective for this task? And, (2) what properties of selected examples and domains achieve strong results? Among 18 acquisition functions from 4 families of methods, we find H-Divergence methods, and particularly our proposed variant DAL-E, yield effective results, averaging 2-3% improvements over the random baseline. We also show the importance of a diverse allocation of domains, as well as room-for-improvement of existing methods on both domain and example selection. Our findings yield the first comprehensive analysis of both existing and novel methods for practitioners faced with multi-domain active learning for natural language tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。