arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.15151cs.CLcs.AI
收藏

语言模型作为持续自我进化的数据工程师

Language Models as Continuous Self-Evolving Data Engineers

School of Computer Science and Engineering, Northeastern University(东北大学计算机科学与工程学院);China Mobile Internet Company Limited(中国移动互联网有限公司);Alibaba Group(阿里巴巴集团)

Peidong Wang, Ming Wang, Zhiming Ma, Xiaocui Yang, Shi Feng, Daling Wang, Yifei Zhang, Kaisong Song

AI总结:

针对LLMs因缺乏高质量训练数据及依赖专家标注受限的问题,提出LANCE范式让LLMs自主生成、清洗、审查并标注数据自我训练,经Qwen2系列模型验证,多任务性能提升,减少数据构建成本与对专家依赖。

摘要:

大型语言模型(LLMs)在各类任务上展现出卓越能力,但因缺乏高质量训练数据而限制了进一步进化。此外,传统训练方法过度依赖专家标注数据,为LLMs的性能设置了上限。为解决该问题,我们提出一种名为LANCE(语言模型作为持续自我进化的数据工程师)的新型范式,使LLMs能通过自主生成、清洗、审查并标注含偏好信息的数据来自我训练。我们的方法表明,LLMs可作为持续自我进化的数据工程师,显著减少训练后数据构建的时间和成本。通过对Qwen2系列模型的迭代微调,我们验证了LANCE在多任务中的有效性,显示其能维持高质量数据生成并持续提升模型性能。在多基准维度上,LANCE使Qwen2-7B的平均得分提升3.64,Qwen2-7B-Instruct提升1.75。这种自主数据构建的训练范式不仅减少了对人类专家或外部模型的依赖,还确保数据符合人类偏好,为未来超越人类能力的超级智能系统发展铺路。代码可在https://github.com/Control-derek/LANCE获取。

英文摘要:

Large Language Models (LLMs) have demonstrated remarkable capabilities on various tasks, while the further evolvement is limited to the lack of high-quality training data. In addition, traditional training approaches rely too much on expert-labeled data, setting a ceiling on the performance of LLMs. To address this issue, we propose a novel paradigm named LANCE (LANguage models as Continuous self-Evolving data engineers) that enables LLMs to train themselves by autonomously generating, cleaning, reviewing, and annotating data with preference information. Our approach demonstrates that LLMs can serve as continuous self-evolving data engineers, significantly reducing the time and cost of the post-training data construction. Through iterative fine-tuning on Qwen2 series models, we validate the effectiveness of LANCE across various tasks, showing that it can maintain high-quality data generation and continuously improve model performance. Across multiple benchmark dimensions, LANCE results in an average score enhancement of 3.64 for Qwen2-7B and 1.75 for Qwen2-7B-Instruct. This training paradigm with autonomous data construction not only reduces the reliance on human experts or external models but also ensures that the data aligns with human preferences, paving the way for the development of future superintelligent systems that can exceed human capabilities. Codes are available at: https://github.com/Control-derek/LANCE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑