arXiv 2505.220422025-05-28cs.LGcs.AI

无需重新训练即可估计大型语言模型的样本训练顺序效应

Estimating the Effects of Sample Training Orders for Large Language Models without Retraining

Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学耿丽人工智能学院）；Center for Data Science, Peking University（北京大学数据科学中心）；School of Engineering Mathematics and Technology, University of Bristol（布里斯托大学工程数学与技术学院）；Department of Machine Learning, Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学机器学习系）

Hao Yang, Haoxuan Li, Mengyue Yang, Xu Chen, Mingming Gong

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究提出一种无需重新训练的框架，利用泰勒展开近似Adam优化器并存储中间检查点，高效估计不同训练样本顺序对LLMs的影响，并应用于课程设计与记忆泛化分析。

摘要：

训练样本的顺序在大型语言模型（LLMs）中起着至关重要的作用，对其外部性能和内部学习动态均有显著影响。研究这一效应的传统方法通常需要使用不同的样本顺序重新训练模型，这对于LLMs来说在计算上是不可行的。在本研究中，我们通过设计一个无需重新训练的框架来改进传统方法。通过使用一阶和二阶泰勒展开近似Adam优化器更新，并利用随机投影方法存储中间检查点，我们的框架能够高效估计任意训练样本顺序下的模型参数。接下来，我们将该框架应用于两个下游研究问题：（1）LLMs的训练课程设计——我们基于该无需重新训练的框架提出了一种新颖的课程学习策略，通过估计的模型性能来增强课程方案，从而实现更明智的样本调度。（2）LLMs的记忆与泛化效应分析——我们使用该框架估计训练样本的位置如何影响LLMs的记忆和泛化能力。我们进行了广泛的实验，验证了该框架在重现真实模型性能方面的有效性，并进一步展示了其在优化LLM训练课程和分析LLMs记忆与泛化效应方面的潜力。

英文摘要：

The order of training samples plays a crucial role in large language models (LLMs), significantly impacting both their external performance and internal learning dynamics. Traditional methods for investigating this effect generally require retraining the model with various sample orders, which is computationally infeasible for LLMs. In this work, we improve traditional methods by designing a retraining-free framework. By approximating Adam optimizer updates with first- and second-order Taylor expansions and utilizing random projection methods to store intermediate checkpoints, our framework can efficiently estimate model parameters for arbitrary training sample orders. Next, we apply our framework to two downstream research problems: (1) Training curriculum design for LLMs -- we base our retraining-free framework to propose a novel curriculum learning strategy that augments curriculum proposals with estimated model performances, enabling more informed sample scheduling. (2) LLMs' memorization and generalization effect analysis -- we use our retraining-free framework to estimate how the positions of training samples influence LLMs' capacity for memorization and generalization. We conduct extensive experiments to validate the effectiveness of our retraining-free framework in reproducing the true model performances, and further demonstrate its potential in optimizing LLM training curricula and analyzing the memorization and generalization effects of LLMs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。