arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.04197cs.CL
收藏

DICE:检测数学推理中LLM微调阶段的分布内污染

DICE: Detecting In-distribution Contamination in LLM's Fine-tuning Phase for Math Reasoning

Tsinghua University(清华大学);Beihang University(北京航空航天大学)

Shangqing Tu, Kejian Zhu, Yushi Bai, Zijun Yao, Lei Hou, Juanzi Li

AI总结:

提出DICE方法,利用LLM内部状态定位并检测数学推理微调阶段的分布内数据污染,发现其会导致OOD性能下降,且DICE预测与模型实际性能高度相关。

摘要:

大型语言模型(LLMs)的进步依赖于公共基准的评估,但数据污染会导致性能被高估。以往的研究侧重于通过确定模型在训练期间是否见过完全相同的数据来检测污染。此外,先前的工作已经表明,即使在类似于基准数据的数据上进行训练也会提升性能,即分布内污染。在这项工作中,我们认为分布内污染会导致OOD基准上的性能下降。为了有效检测分布内污染,我们提出了DICE,这是一种利用LLMs内部状态来定位并检测污染的新方法。DICE首先识别对污染最敏感的层,然后基于该层的内部状态训练一个分类器。实验表明,DICE在各种LLMs和数学推理数据集上检测分布内污染的准确率很高。我们还展示了训练好的DICE检测器的泛化能力,它能够检测具有相似分布的多个基准中的污染。此外,我们发现DICE的预测与我们或其他组织微调的LLMs的性能相关,决定系数(R^2)在0.61到0.75之间。代码和数据可在https://github.com/THU-KEG/DICE获取。

英文摘要:

The advancement of large language models (LLMs) relies on evaluation using public benchmarks, but data contamination can lead to overestimated performance. Previous researches focus on detecting contamination by determining whether the model has seen the exact same data during training. Besides, prior work has already shown that even training on data similar to benchmark data inflates performance, namely \emph{In-distribution contamination}. In this work, we argue that in-distribution contamination can lead to the performance drop on OOD benchmarks. To effectively detect in-distribution contamination, we propose DICE, a novel method that leverages the internal states of LLMs to locate-then-detect the contamination. DICE first identifies the most sensitive layer to contamination, then trains a classifier based on the internal states of that layer. Experiments reveal DICE's high accuracy in detecting in-distribution contamination across various LLMs and math reasoning datasets. We also show the generalization capability of the trained DICE detector, which is able to detect contamination across multiple benchmarks with similar distributions. Additionally, we find that DICE's predictions correlate with the performance of LLMs fine-tuned by either us or other organizations, achieving a coefficient of determination ($R^2$) between 0.61 and 0.75. The code and data are available at https://github.com/THU-KEG/DICE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑