The advancement of large language models (LLMs) relies on evaluation using public benchmarks, but data contamination can lead to overestimated performance. Previous researches focus on detecting contamination by determining whether the model has seen the exact same data during training. Besides, prior work has already shown that even training on data similar to benchmark data inflates performance, namely \emph{In-distribution contamination}. In this work, we argue that in-distribution contamination can lead to the performance drop on OOD benchmarks. To effectively detect in-distribution contamination, we propose DICE, a novel method that leverages the internal states of LLMs to locate-then-detect the contamination. DICE first identifies the most sensitive layer to contamination, then trains a classifier based on the internal states of that layer. Experiments reveal DICE's high accuracy in detecting in-distribution contamination across various LLMs and math reasoning datasets. We also show the generalization capability of the trained DICE detector, which is able to detect contamination across multiple benchmarks with similar distributions. Additionally, we find that DICE's predictions correlate with the performance of LLMs fine-tuned by either us or other organizations, achieving a coefficient of determination ($R^2$) between 0.61 and 0.75. The code and data are available at https://github.com/THU-KEG/DICE.
Toward Automated Robustness Evaluation of Mathematical Reasoning
Shanghai University of Finance and Economics(上海金融学院);Ant Group(蚂蚁集团);Southern University of Science and Technology(南方科技大学);MoE Key Laboratory of Interdisciplinary Research of Computation and Economics(计算与经济学交叉学科研究教育部重点实验室)
RePro: Proof-Verified Benchmark Rewriting for Reliable Evaluation of LLM Mathematical Problem Solving
Nanyang Technological University(南洋理工大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));INSAIT;Sofia University “St. Kliment Ohridski”(索非亚大学“圣·克利门特·奥赫里德斯基”);Shenzhen Loop Area Institute(深圳河套学院);AIRS