SERSEM:基于代码语言模型的成员推断的选性熵加权评分
SERSEM: Selective Entropy-Weighted Scoring for Membership Inference in Code Language Models
AI总结:
SERSEM提出一种新的白盒攻击框架,通过抑制无信息的语法模板来增强特定记忆信号,通过双信号方法提升成员推断的准确性,实验表明其在代码记忆检测中表现优于传统基线方法。
摘要:
随着大型语言模型(LLMs)在代码领域越来越多地使用大规模且非许可的 datasets,通过成员推断攻击(MIAs)评估数据污染变得至关重要。我们提出 SERSEM(选性熵加权评分用于成员推断),一种新的白盒攻击框架,通过静态抽象语法树(AST)分析、基于拼写检查的多语言逻辑检测和离线 linting 推导出连续字符级权重掩码,然后利用这些启发式权重池化内部 transformer 激活并校准 token 级 Z 分数。在 25,000 样本平衡数据集上评估,SERSEM 在 StarCoder2-3B 模型上达到全局 AUC-ROC 0.7913,在 StarCoder2-7B 模型上达到 0.7867,始终优于实现的概率基线方法 Loss、Min-K% Prob 和 PAC。我们的发现表明,关注人类中心的编码异常提供了比序列级概率平均显著更稳健的verbatim记忆指示器。
英文摘要:
As Large Language Models (LLMs) for code increasingly utilize massive, often non-permissively licensed datasets, evaluating data contamination through Membership Inference Attacks (MIAs) has become critical. We propose SERSEM (Selective Entropy-Weighted Scoring for Membership Inference), a novel white-box attack framework that suppresses uninformative syntactical boilerplate to amplify specific memorization signals. SERSEM utilizes a dual-signal methodology: first, a continuous character-level weight mask is derived through static Abstract Syntax Tree (AST) analysis, spellchecking-based multilingual logic detection, and offline linting. Second, these heuristic weights are used to pool internal transformer activations and calibrate token-level Z-scores from the output logits. Evaluated on a 25,000-sample balanced dataset, SERSEM achieves a global AUC-ROC of 0.7913 on the StarCoder2-3B model and 0.7867 on the StarCoder2-7B model, consistently outperforming the implemented probability-based baselines Loss, Min-K% Prob, and PAC. Our findings demonstrate that focusing on human-centric coding anomalies provides a significantly more robust indicator of verbatim memorization than sequence-level probability averages.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。