探究公开微调数据集：从构建视角对当前实践的综合性综述
Investigating Public Fine-Tuning Datasets: A Complex Review of Current Practices from a Construction Perspective
AI总结：
本文从构建视角综述公开微调数据集，梳理其演进与分类，详述大语言模型微调数据集的构建技术，总结不同数据准备阶段的特征，为未来微调数据集的构建与发展提供参考。
摘要：
随着大模型领域的快速发展，微调作为大模型训练流程的组成部分，其相关研究也同步取得了显著进展。数据工程在模型训练过程中发挥着基础性作用，涵盖数据基础设施、数据处理等环节，而微调阶段的数据同样是大模型的基础。为充分发挥微调数据集的价值并探索其新的可能性，本文从数据构建的视角对当前公开微调数据集展开综述。\n本综述从演进历程与分类体系两个维度对公开微调数据集进行概述，旨在梳理其发展轨迹。文章按照上述分类，具体围绕示范类、对比类和通用类三大类别，详细阐述了大语言模型（LLMs）公开微调数据集的构建技术与方法，包括数据生成、数据增强等内容。此外，本综述还提炼出数据生成技术的分类树，以帮助研究者从构建维度更深入地理解微调数据集。\n本综述还总结了当前该领域实践中不同数据准备阶段的构建特征，旨在提供全面的概览并为未来研究提供参考。文章还从构建视角讨论了涵盖多种数据模态的微调数据集实践。在文末，我们针对微调数据集未来的构建与发展提出了见解与思考方向。
英文摘要：
With the rapid development of the large model domain, research related to fine-tuning has concurrently seen significant advancement, given that fine-tuning is a constituent part of the training process for large-scale models. Data engineering plays a fundamental role in the training process of models, which includes data infrastructure, data processing, etc. Data during fine-tuning likewise forms the base for large models. In order to embrace the power and explore new possibilities of fine-tuning datasets, this paper reviews current public fine-tuning datasets from the perspective of data construction. An overview of public fine-tuning datasets from two sides: evolution and taxonomy, is provided in this review, aiming to chart the development trajectory. Construction techniques and methods for public fine-tuning datasets of Large Language Models (LLMs), including data generation and data augmentation among others, are detailed. This elaboration follows the aforementioned taxonomy, specifically across demonstration, comparison, and generalist categories. Additionally, a category tree of data generation techniques has been abstracted in our review to assist researchers in gaining a deeper understanding of fine-tuning datasets from the construction dimension. Our review also summarizes the construction features in different data preparation phases of current practices in this field, aiming to provide a comprehensive overview and inform future research. Fine-tuning dataset practices, encompassing various data modalities, are also discussed from a construction perspective in our review. Towards the end of the article, we offer insights and considerations regarding the future construction and developments of fine-tuning datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
