arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.14536cs.CLcs.AI
收藏

可解释的token级噪声过滤用于LLM微调数据集

Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets

The State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全全国重点实验室);Hangzhou HighTech Zone (Binjiang) Blockchain and Data Security Research Institute(杭州高新区(滨江)区块链与数据安全研究院)

Yuchen Yang, Wenze Lin, Enhao Huang, Zhixuan Chu, Hongbin Zhou, Lan Tao, Yiming Li, Zhan Qin, Kui Ren

AI总结:

本文提出XTF框架,通过分解token级数据对微调过程的贡献,提升LLM微调性能,实验表明在数学、代码和医学任务中性能提升达13.7%。

摘要:

大型语言模型(LLMs)在多种应用中取得了显著进展。微调是将LLMs适应特定下游任务的重要步骤,通常涉及在相应数据集上进一步训练。然而,当前微调数据集与LLM的token级优化机制之间存在根本性差异:大多数数据集是在句子级别设计的,引入token级噪声,从而对最终性能产生负面影响。本文提出XTF,一种可解释的token级噪声过滤框架。XTF将token级数据对微调过程的复杂和微妙贡献分解为三个不同的显性属性(推理重要性、知识新颖性和任务相关性),并通过评分方法进行评估,然后根据所选噪声token的梯度进行掩码,以优化微调LLM的性能。我们在7种主流LLM上进行了广泛的实验,涉及三个代表性的下游任务(数学、代码和医学)。结果表明,XTF相比常规微调可显著提升下游性能,最高提升达13.7%。我们的工作强调了token级数据集优化的重要性,并展示了基于属性分解的策略在解释复杂训练机制方面的潜力。

英文摘要:

Large Language Models (LLMs) have seen remarkable advancements, achieving state-of-the-art results in diverse applications. Fine-tuning, an important step for adapting LLMs to specific downstream tasks, typically involves further training on corresponding datasets. However, a fundamental discrepancy exists between current fine-tuning datasets and the token-level optimization mechanism of LLMs: most datasets are designed at the sentence-level, which introduces token-level noise, causing negative influence to final performance. In this paper, we propose XTF, an explainable token-level noise filtering framework. XTF decomposes the complex and subtle contributions of token-level data to the fine-tuning process into three distinct and explicit attributes (reasoning importance, knowledge novelty, and task relevance), which can be assessed using scoring methods, and then masks the gradients of selected noisy tokens accordingly to optimize the performance of fine-tuned LLMs. We conduct extensive experiments on three representative downstream tasks (math, code and medicine) across 7 mainstream LLMs. The results demonstrate that XTF can significantly improve downstream performance by up to 13.7% compared to regular fine-tuning. Our work highlights the importance of token-level dataset optimization, and demonstrates the potential of strategies based on attribute decomposition for explaining complex training mechanisms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑