通过翻译掩盖数据污染：来自阿拉伯语语料库的证据
Obscuring Data Contamination Through Translation: Evidence from Arabic Corpora
AI总结：
本研究通过阿拉伯语翻译实验发现，数据污染在多语言环境下表现复杂，提出翻译感知污染检测方法以提升LLM评估的公平性与透明度。
摘要：
数据污染会削弱大语言模型评估的有效性，因为它使模型依赖于记忆的基准内容而不是真正的泛化。尽管先前的工作提出了污染检测方法，但这些方法大多局限于英语基准，导致多语言污染理解不足。在本工作中，我们通过在不同比例的阿拉伯语语料上微调几种开放式权重的LLMs，并在原始英语基准上进行评估，研究多语言环境中的污染动态。为了检测记忆，我们扩展了测试槽猜测方法，加入选择重排策略，并结合Min-K%概率分析，捕捉行为和分布污染信号。我们的结果表明，阿拉伯语翻译会抑制传统污染指标，但模型仍从接触污染数据中受益，特别是那些阿拉伯语能力较强的模型。这种效果在污染水平增加时，始终反映在Mink%分数上升和跨语言回答一致性增加上。为了解决这一盲点，我们提出了一种翻译感知的污染检测方法，该方法通过比较多个翻译基准变体的信号来识别污染，而不是仅使用英语。翻译感知的污染检测在英语方法失效时仍能可靠地揭示污染。共同，我们的发现突显了需要多语言、翻译感知的评估流程来确保LLMs的公平、透明和可重复评估。
英文摘要：
Data contamination can invalidate benchmark evaluation when a model benefits from memorized evaluation content rather than genuine generalization. Yet contamination is difficult to audit when the exposed content differs in language from the evaluation benchmark. We study this failure mode by deliberately exposing four open-weight instruction-tuned LLMs to Arabic translations of MMLU and XQuAD evaluation items at increasing exposure levels, then evaluating them on the original English tasks. This controlled setup is a proxy for contamination rather than a reconstruction of real-world pretraining leakage. We first test two English-centric post-hoc probes, TS-Guessing and Min-K%++, and find that their signals largely disappear under translated exposure: TS-Guessing remains weak except for model-specific positional recall on MMLU, while Min-K%++ stays at or below chance. At the same time, English MMLU performance increases with Arabic exposure, showing that the absence of an English contamination signal does not imply the absence of an exposure effect. We then introduce Translation-Aware Contamination Detection (TACD), a training-data-free diagnostic based on cross-lingual prediction consistency and choice reordering. Cross-lingual consistency is substantially higher than an independence baseline and generally increases relative to the clean condition, although its magnitude is model-dependent and not strictly monotonic. These results show that translation can conceal contamination-related effects from English-only probes and motivate multilingual diagnostics that are explicitly framed as evidence of contamination-consistent behavior rather than definitive membership tests.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
