The increasing size and complexity of pre-trained language models have demonstrated superior performance in many applications, but they usually require large training datasets to be adequately trained. Insufficient training sets could unexpectedly make the model overfit and fail to cope with complex tasks. Large language models (LLMs) trained on extensive corpora have prominent text generation capabilities, which improve the quality and quantity of data and play a crucial role in data augmentation. Specifically, distinctive prompt templates are given in personalised tasks to guide LLMs in generating the required content. Recent promising retrieval-based techniques further improve the expressive performance of LLMs in data augmentation by introducing external knowledge to enable them to produce more grounded-truth data. This survey provides an in-depth analysis of data augmentation in LLMs, classifying the techniques into Simple Augmentation, Prompt-based Augmentation, Retrieval-based Augmentation and Hybrid Augmentation. We summarise the post-processing approaches in data augmentation, which contributes significantly to refining the augmented data and enabling the model to filter out unfaithful content. Then, we provide the common tasks and evaluation metrics. Finally, we introduce existing challenges and future opportunities that could bring further improvement to data augmentation.
Use Random Selection for Now: Investigation of Few-Shot Selection Strategies in LLM-based Text Augmentation for Classification
Faculty of Information Technology, Brno University of Technology(布尔诺理工大学信息技术学院);Kempelen Institute of Intelligent Technologies(肯佩伦智能技术研究所);University of Pittsburgh(匹兹堡大学)
CoUDA: Coherence Evaluation via Unified Data Augmentation
Institute of Artificial Intelligence, Soochow University(苏州大学人工智能研究院);School of Computer Science, Peking University(北京大学计算机学院);Jiangsu Normal University(江苏师范大学)
Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challenges
LMU Munich & Munich Center for Machine Learning(慕尼黑大学及慕尼黑机器学习中心);University of Cologne(科隆大学);University of Augsburg(奥格斯堡大学);Bosch Center for Artificial Intelligence(博世人工智能中心);Columbia University(哥伦比亚大学);University of Maryland, College Park(马里兰大学学院市分校)
Open Foundation Models in Healthcare: Challenges, Paradoxes, and Opportunities with GenAI Driven Personalized Prescription
Qatar University(卡塔尔大学);Birmingham City University(伯明翰城市大学);Hamad Bin Khalifa University (HBKU)(哈马德·本·哈利法大学);University of the West of England (UWE)(西英格兰大学)
ZJUKLAB at SemEval-2021 Task 4: Negative Augmentation with Language Model for Reading Comprehension of Abstract Meaning
Zhejiang University & AZFT Joint Lab for Knowledge Engine(浙江大学与AZFT联合知识引擎实验室);Hangzhou Innovation Center, Zhejiang University(浙江大学杭州创新中心);Microsoft(微软)