arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.00705cs.CLcs.LG
收藏

SHED:基于沙普利值的指令微调自动化数据集精炼方法

SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning

University of Maryland(马里兰大学);Clemson University(克莱姆森大学);Rutgers University(罗格斯大学)

Yexiao He, Ziyao Wang, Zheyu Shen, Guoheng Sun, Yucong Dai, Yongkai Wu, Hongyi Wang, Ang Li

AI总结:

针对指令微调数据冗余问题,提出无需人工与商用LLM的基于沙普利值的自动化数据集精炼框架SHED,其选出的10%数据性能可比肩甚至超越全量数据,且具备跨模型迁移性。

摘要:

预训练大语言模型(LLMs)可通过微调适配众多下游任务,并被调整以符合人类偏好。近期研究发现,LLMs仅需少量高质量数据即可达到理想性能,这表明这些大规模数据集中有大量数据是冗余甚至有害的。从海量数据中识别高质量数据以构建小而有效的数据集,已成为一项关键挑战。本文提出SHED——一种基于沙普利值(Shapley value)的指令微调自动化数据集精炼框架。SHED无需人工干预,也无需使用商用LLMs。此外,通过SHED构建的数据集具有可迁移性,可在不同LLMs间复用并始终保持高性能。我们开展了大量实验评估SHED构建的数据集,结果表明,在各类任务和LLMs上,SHED的表现均优于现有最优方法;值得注意的是,SHED选出的仅占原始数据10%的数据集,其性能与完整数据集相当甚至更优。

英文摘要:

The pre-trained Large Language Models (LLMs) can be adapted for many downstream tasks and tailored to align with human preferences through fine-tuning. Recent studies have discovered that LLMs can achieve desirable performance with only a small amount of high-quality data, suggesting that a large amount of the data in these extensive datasets is redundant or even harmful. Identifying high-quality data from vast datasets to curate small yet effective datasets has emerged as a critical challenge. In this paper, we introduce SHED, an automated dataset refinement framework based on Shapley value for instruction fine-tuning. SHED eliminates the need for human intervention or the use of commercial LLMs. Moreover, the datasets curated through SHED exhibit transferability, indicating they can be reused across different LLMs with consistently high performance. We conduct extensive experiments to evaluate the datasets curated by SHED. The results demonstrate SHED's superiority over state-of-the-art methods across various tasks and LLMs; notably, datasets comprising only 10% of the original data selected by SHED achieve performance comparable to or surpassing that of the full datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑