大型语言模型中的预训练数据暴露：成员推断、数据污染及安全影响综述
Pretraining Data Exposure in Large Language Models: A Survey of Membership Inference, Data Contamination, and Security Implications
AI总结：
本文首次统一综述了大型语言模型中的预训练数据暴露问题，涵盖成员推断和数据污染，形式化定义了暴露级别，回顾了攻击与防御方法，并总结了实证发现及未来研究方向。
摘要：
大型语言模型（LLMs）已成为NLP中的主导范式，推动了研究和工业的发展。随着模型规模和预训练数据的增长，由于训练数据集的规模和不可见性，对预训练数据暴露（PDE）的担忧也在增加。PDE指的是确定特定数据是否出现在LLM的预训练语料库中。它对于确保评估完整性和保护隐私至关重要，涉及两个关键领域：数据污染和成员推断。尽管概念上相关，但这些领域通常被孤立研究。本文首次在PDE框架下对两者进行了统一综述。我们形式化了跨暴露级别的PDE，回顾了攻击和防御方法，综合了实证发现，并强调了开放的挑战和未来的研究方向。
英文摘要：
Large Language Models (LLMs) have become the predominant paradigm in NLP, advancing both research and industry. As model sizes and pretraining data grow, concerns about Pretraining Data Exposure (PDE) increase due to the scale and opacity of training datasets. PDE refers to determining whether specific data appeared in an LLM's pretraining corpus. It is critical for ensuring evaluation integrity and protecting privacy, intersecting two key areas: data contamination and membership inference. Though conceptually related, these areas have often been studied in isolation. This paper offers the first unified survey of both under the PDE framework. We formalize PDE across exposure levels, review attack and defense methods, synthesize empirical findings, and highlight open challenges and future research directions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
