arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.11275cs.CL
收藏

通过知识检测对大型语言模型进行自训练

Self-training Large Language Models through Knowledge Detection

Nanyang Technological University(南洋理工大学);Institute of High Performance Computing (IHPC)(高性能计算研究所)

Wei Jie Yeo, Teddy Ferdinan, Przemyslaw Kazienko, Ranjan Satapathy, Erik Cambria

AI总结:

本文提出一种基于无参考一致性方法识别未知数据的自训练范式,LLM自主生成标签并选择性训练,显著减少了生成幻觉并缓解了灾难性遗忘,降低了对标注数据的依赖。

摘要:

大型语言模型(LLMs)通常需要大量标注数据集和训练计算资源,才能在下游任务中取得令人瞩目的性能。本文探索了一种自训练范式,其中LLM自主整理自己的标签,并通过一种无参考的一致性方法识别未知数据样本,对其进行选择性训练。实证评估表明,该方法在多个学科中显著减少了生成过程中的幻觉现象。此外,该选择性训练框架缓解了分布外基准测试中的灾难性遗忘问题,解决了训练LLMs的一个关键局限。我们的研究结果表明,这种方法可以大幅减少对大型标注数据集的依赖,为更具可扩展性和成本效益的语言模型训练铺平道路。

英文摘要:

Large language models (LLMs) often necessitate extensive labeled datasets and training compute to achieve impressive performance across downstream tasks. This paper explores a self-training paradigm, where the LLM autonomously curates its own labels and selectively trains on unknown data samples identified through a reference-free consistency method. Empirical evaluations demonstrate significant improvements in reducing hallucination in generation across multiple subjects. Furthermore, the selective training framework mitigates catastrophic forgetting in out-of-distribution benchmarks, addressing a critical limitation in training LLMs. Our findings suggest that such an approach can substantially reduce the dependency on large labeled datasets, paving the way for more scalable and cost-effective language model training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑