arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.18845cs.CL
收藏

面向大型语言模型的文本数据增强:方法、挑战与机遇的全面综述

Text Data Augmentation for Large Language Models: A Comprehensive Survey of Methods, Challenges, and Opportunities

Lingnan University(岭南大学);School of Data Science, Lingnan University(岭南大学数据科学学院)

Yaping Chai, Haoran Xie, Joe S. Qin

AI总结:

本综述全面分析了大型语言模型(LLMs)的文本数据增强方法,将其分为简单增强、基于提示、基于检索和混合增强四类,并总结了后处理技术、评估指标及未来挑战与机遇。

摘要:

预训练语言模型规模和复杂性的不断增加在许多应用中展现出了卓越的性能,但它们通常需要大型训练数据集才能得到充分训练。训练集不足可能会意外导致模型过拟合,并使其无法应对复杂任务。在广泛语料库上训练的大型语言模型(LLMs)具有突出的文本生成能力,这提高了数据的质量和数量,并在数据增强中发挥着至关重要的作用。具体而言,在个性化任务中会给出独特的提示模板,以引导LLMs生成所需内容。近期有前景的基于检索的技术通过引入外部知识,进一步提升了LLMs在数据增强中的表达能力,使其能够产生更多基于事实的数据。本综述对LLMs中的数据增强进行了深入分析,将技术分为简单增强、基于提示的增强、基于检索的增强和混合增强。我们总结了数据增强中的后处理方法,这对于细化增强数据并使模型能够过滤掉不忠实内容具有重要意义。然后,我们提供了常见任务和评估指标。最后,我们介绍了现有挑战和未来机遇,这些挑战和机遇可能会为数据增强带来进一步的改进。

英文摘要:

The increasing size and complexity of pre-trained language models have demonstrated superior performance in many applications, but they usually require large training datasets to be adequately trained. Insufficient training sets could unexpectedly make the model overfit and fail to cope with complex tasks. Large language models (LLMs) trained on extensive corpora have prominent text generation capabilities, which improve the quality and quantity of data and play a crucial role in data augmentation. Specifically, distinctive prompt templates are given in personalised tasks to guide LLMs in generating the required content. Recent promising retrieval-based techniques further improve the expressive performance of LLMs in data augmentation by introducing external knowledge to enable them to produce more grounded-truth data. This survey provides an in-depth analysis of data augmentation in LLMs, classifying the techniques into Simple Augmentation, Prompt-based Augmentation, Retrieval-based Augmentation and Hybrid Augmentation. We summarise the post-processing approaches in data augmentation, which contributes significantly to refining the augmented data and enabling the model to filter out unfaithful content. Then, we provide the common tasks and evaluation metrics. Finally, we introduce existing challenges and future opportunities that could bring further improvement to data augmentation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑