arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.03560cs.LGstat.ML
收藏

In2Core:利用影响函数进行大语言模型指令微调中的核心集选择

In2Core: Leveraging Influence Functions for Coreset Selection in Instruction Finetuning of Large Language Models

Institute for Infocomm Research (IR), A*STAR(新加坡科技研究局信息通信研究院);CNRS(法国国家科学研究中心);IRIT(图卢兹计算机科学、建模与信息工程研究所);National University of Singapore(新加坡国立大学);Centre for Frontier AI Research (CFAR), A*STAR(新加坡科技研究局前沿人工智能研究中心)

Ayrton San Joaquin, Bin Wang, Zhengyuan Liu, Nicholas Asher, Brian Lim, Philippe Muller, Nancy F. Chen

AI总结:

本文提出In2Core算法,通过分析训练与评估样本间的梯度相关性进行核心集选择,并优化影响函数计算效率,使大语言模型指令微调仅用50%数据即可达到相似性能。

摘要:

尽管已有诸多进展,大语言模型(LLMs)的微调仍然成本高昂,原因在于参数数量庞大以及模型泛化所需的数据量巨大。计算资源的可及性对开源社区而言仍是一道障碍。为应对这一挑战,我们提出了In2Core算法,该算法通过使用已训练模型分析训练样本与评估样本之间的相关性来选择核心集(coreset)。值得注意的是,我们评估模型的内部梯度以估计这种关系,旨在对每个训练点的贡献进行排序。为提升效率,我们提出一种优化方法,在减少层数的情况下计算影响函数(influence functions),同时保持相似的准确率。将我们的算法应用于大语言模型的指令微调数据后,仅使用50%的训练数据即可达到相似的性能。同时,利用影响函数分析模型对某些测试样本的覆盖情况,可以为训练集对这些测试点的覆盖程度提供可靠且可解释的信号。

英文摘要:

Despite advancements, fine-tuning Large Language Models (LLMs) remains costly due to the extensive parameter count and substantial data requirements for model generalization. Accessibility to computing resources remains a barrier for the open-source community. To address this challenge, we propose the In2Core algorithm, which selects a coreset by analyzing the correlation between training and evaluation samples with a trained model. Notably, we assess the model's internal gradients to estimate this relationship, aiming to rank the contribution of each training point. To enhance efficiency, we propose an optimization to compute influence functions with a reduced number of layers while achieving similar accuracy. By applying our algorithm to instruction fine-tuning data of LLMs, we can achieve similar performance with just 50% of the training data. Meantime, using influence functions to analyze model coverage to certain testing samples could provide a reliable and interpretable signal on the training set's coverage of those test points.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑