This paper embarks on an exploration into the Large Language Model (LLM) datasets, which play a crucial role in the remarkable advancements of LLMs. The datasets serve as the foundational infrastructure analogous to a root system that sustains and nurtures the development of LLMs. Consequently, examination of these datasets emerges as a critical topic in research. In order to address the current lack of a comprehensive overview and thorough analysis of LLM datasets, and to gain insights into their current status and future trends, this survey consolidates and categorizes the fundamental aspects of LLM datasets from five perspectives: (1) Pre-training Corpora; (2) Instruction Fine-tuning Datasets; (3) Preference Datasets; (4) Evaluation Datasets; (5) Traditional Natural Language Processing (NLP) Datasets. The survey sheds light on the prevailing challenges and points out potential avenues for future investigation. Additionally, a comprehensive review of the existing available dataset resources is also provided, including statistics from 444 datasets, covering 8 language categories and spanning 32 domains. Information from 20 dimensions is incorporated into the dataset statistics. The total data size surveyed surpasses 774.5 TB for pre-training corpora and 700M instances for other datasets. We aim to present the entire landscape of LLM text datasets, serving as a comprehensive reference for researchers in this field and contributing to future studies. Related resources are available at: https://github.com/lmmlzn/Awesome-LLMs-Datasets.
A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models
Amirkabir University of Technology(阿米尔卡比尔理工大学);Stanford University(斯坦福大学);Massachusetts Institute of Technology(麻省理工学院);Columbia University(哥伦比亚大学);Amazon GenAI(亚马逊生成式AI部门)
A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(莫扎德大学人工智能学院);Fraunhofer Institute for Open Communication Systems (FOKUS)(弗劳恩霍夫开放通信系统研究所);Technical University of Munich(慕尼黑技术大学);Technical University of Darmstadt(达姆施塔特技术大学);University of Toronto(多伦多大学)
A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery
University of Washington(华盛顿大学);Texas A&M University(德克萨斯农工大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Los Angeles(加利福尼亚大学洛杉矶分校)
A Comprehensive Survey of Foundation Models in Medicine
Herbert Wertheim College of Engineering, University of Florida(佛罗里达大学赫伯特·沃特海姆工程学院);Norman Fixel Institute for Neurological Diseases, University of Florida(佛罗里达大学诺曼·菲克塞尔神经疾病研究所);University of North Carolina at Charlotte(北卡罗来纳大学夏洛特分校);McKnight Brain Institute, University of Florida(佛罗里达大学麦克奈特脑科学研究所)
A Comprehensive Survey on Aspect Based Sentiment Analysis
Vellore Institute Of Technology(韦洛尔理工大学);School of Computer Science and Engineering- SCOPE(计算机科学与工程学院-SCOPE)
基于方面的情感分析(Aspect Based Sentiment Analysis,ABSA)是自然语言处理的一个分支,它主要是将数据分成方面,并最终提取情感信息。已知ABSA比一般的情感分析提供更多关于上下文的信息。在这项研究中,我们的目的是探索各种方法,同时执行ABSA,并提供一个比较研究。这份调查报告深入讨论了各种解决方案,并对它们进行了比较。并方便地分为几个部分,以获得对该过程的整体看法。
Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学);McGill University(麦吉尔大学);Tencent Company(腾讯公司);Artificial Intelligence Research Institute, iFLYTEK Co., Ltd(人工智能研究院,iFLYTEK公司)