arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.07499cs.CLcs.AI
收藏

NLP中有限数据学习的数据增强研究

An Empirical Survey of Data Augmentation for Limited Data Learning in NLP

Georgia Institute of Technology(佐治亚理工学院);UNC Chapel Hill(北卡罗来纳大学教堂山分校)

Jiaao Chen, Derek Tam, Colin Raffel, Mohit Bansal, Diyi Yang

摘要:

在过去的十年中,通过使用神经模型和大型标记数据集,NLP取得了很大的进步。对大量数据的依赖阻止了NLP模型应用于低资源环境或新任务,这些任务需要大量的时间,金钱或专业知识来标记大量的文本数据。最近,数据增强方法已经被探索作为提高NLP中的数据效率的手段。到目前为止,还没有系统的经验概述数据增强的NLP在有限的标记数据设置,很难理解哪些方法在哪些设置工作。在本文中,我们提供了一个实证调查的最新进展数据增强的NLP在有限的标记数据设置,总结了景观的方法(包括标记级增强,扩展级增强,对抗性增强,隐藏空间增强),并进行了实验,11个数据集涵盖主题/新闻分类,推理任务,释义任务,单句任务。基于这些结果,我们得出了几个结论,以帮助从业者在不同的环境中选择适当的增强,并讨论了NLP中有限数据学习的当前挑战和未来方向。

英文摘要:

NLP has achieved great progress in the past decade through the use of neural models and large labeled datasets. The dependence on abundant data prevents NLP models from being applied to low-resource settings or novel tasks where significant time, money, or expertise is required to label massive amounts of textual data. Recently, data augmentation methods have been explored as a means of improving data efficiency in NLP. To date, there has been no systematic empirical overview of data augmentation for NLP in the limited labeled data setting, making it difficult to understand which methods work in which settings. In this paper, we provide an empirical survey of recent progress on data augmentation for NLP in the limited labeled data setting, summarizing the landscape of methods (including token-level augmentations, sentence-level augmentations, adversarial augmentations, and hidden-space augmentations) and carrying out experiments on 11 datasets covering topics/news classification, inference tasks, paraphrasing tasks, and single-sentence tasks. Based on the results, we draw several conclusions to help practitioners choose appropriate augmentations in different settings and discuss the current challenges and future directions for limited data learning in NLP.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑