arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.08826cs.CLcs.AI
收藏

GPT3Mix:利用大规模语言模型进行文本增强

GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation

NAVER AI Lab;NAVER Clova AI

Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Woo Lee, Woomyeong Park

摘要:

像GPT-3这样的大规模语言模型是优秀的Few-Shot学习器,允许通过自然文本提示来控制它们。最近的研究报告,基于直接分类消除了微调的需要,但缺乏数据和推理的可扩展性。本文提出了一种新的数据增强技术,利用大规模的语言模型,从真实样本的混合生成逼真的文本样本。我们还建议利用软标签预测的语言模型,有效地从大规模的语言模型中提取知识,同时创建文本扰动。我们在不同的分类任务上进行了数据增强实验,并表明我们的方法大大优于现有的文本增强方法。消融研究和定性分析为我们的方法提供了更多见解。

英文摘要:

Large-scale language models such as GPT-3 are excellent few-shot learners, allowing them to be controlled via natural text prompts. Recent studies report that prompt-based direct classification eliminates the need for fine-tuning but lacks data and inference scalability. This paper proposes a novel data augmentation technique that leverages large-scale language models to generate realistic text samples from a mixture of real samples. We also propose utilizing soft-labels predicted by the language models, effectively distilling knowledge from the large-scale language models and creating textual perturbations simultaneously. We perform data augmentation experiments on diverse classification tasks and show that our method hugely outperforms existing text augmentation methods. Ablation studies and a qualitative analysis provide more insights into our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑