通过训练动态理解代码模型中的隐私风险：一种因果方法
Understanding Privacy Risks in Code Models Through Training Dynamics: A Causal Approach
AI总结：
本研究通过因果方法分析代码模型中不同PII类型的泄露风险差异，揭示了泄露风险与可学习性之间的因果关系，并为开发类型感知的防御策略提供依据。
摘要：
大型代码语言模型（LLM4Code）极大地提高了开发人员的生产力，但它们依赖于包含大量个人身份信息（PII）的开源仓库，也引发了隐私问题。先前的研究表明，商业模型可以复现敏感的PII，但现有研究大多将PII视为单一类别，并忽视了不同类型之间的异质性风险。我们研究了不同类型的PII在被LLM4Code学习和泄露的可能性是否不同，以及这种关系是否具有因果性。我们的方法包括构建包含多种PII类型的数据库，微调不同规模的代表性模型，对真实PII数据计算训练动态，并构建结构因果模型以估计可学习性对泄露的因果效应。结果表明，不同PII类型之间的泄露风险差异显著，并且与它们的训练动态相关：易于学习的实例如IP地址表现出更高的泄露，而更难学习的类型如密钥和密码泄露频率较低。模糊类型表现出混合行为。本研究提供了首次因果证据，表明泄露风险是类型依赖的，并为开发类型意识和可学习性意识的防御方法提供了指导。
英文摘要：
Large language models for code (LLM4Code) have greatly improved developer productivity but also raise privacy concerns due to their reliance on open-source repositories containing abundant personally identifiable information (PII). Prior work shows that commercial models can reproduce sensitive PII, yet existing studies largely treat PII as a single category and overlook the heterogeneous risks among different types. We investigate whether distinct PII types vary in their likelihood of being learned and leaked by LLM4Code, and whether this relationship is causal. Our methodology includes building a dataset with diverse PII types, fine-tuning representative models of different scales, computing training dynamics on real PII data, and formulating a structural causal model to estimate the causal effect of learnability on leakage. Results show that leakage risks differ substantially across PII types and correlate with their training dynamics: easy-to-learn instances such as IP addresses exhibit higher leakage, while harder types such as keys and passwords leak less frequently. Ambiguous types show mixed behaviors. This work provides the first causal evidence that leakage risks are type-dependent and offers guidance for developing type-aware and learnability-aware defenses for LLM4Code.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
