DistillGuard: 评估对抗大语言模型知识蒸馏的防御措施
DistillGuard: Evaluating Defenses Against LLM Knowledge Distillation
AI总结：
DistillGuard通过系统评估发现，现有输出层面防御措施在对抗大语言模型知识蒸馏时效果有限，尤其在数学推理任务中表现突出，揭示了防御措施的依赖性和任务特定性。
摘要：
从专有大语言模型（LLM）API中进行知识蒸馏对模型提供者构成日益增长的威胁，但针对这种攻击的防御措施仍然碎片化且未经评估。我们提出了DistillGuard，一个用于系统评估输出层面防御措施的框架。我们引入了三种防御类别——输出扰动、数据污染和信息 throttling——并通过标准化流程使用Qwen3-14B作为教师模型和Qwen2.5-7B-Instruct作为学生模型，在三个基准（MATH-500、HumanEval+、MT-Bench）上评估了九种防御配置。我们的结果表明，在针对 naive 攻击者的同一家族蒸馏设置中，大多数输出层面的防御措施出人意料地无效：基于改写扰动的措施几乎不会降低蒸馏学生质量，数据污染主要损害对话流畅性，而保留任务特定能力。只有消除推理链的措施显著损害数学推理（31.4% vs. 67.8% 基线），尽管代码生成不受影响。这些发现表明，蒸馏防御的有效性高度依赖于任务，而当前的输出层面方法无法广泛防止知识窃取。
英文摘要：
Knowledge distillation from proprietary LLM APIs poses a growing threat to model providers, yet defenses against this attack remain fragmented and unevaluated. We present DistillGuard, a framework for systematically evaluating output-level defenses against LLM knowledge distillation. We introduce a taxonomy of three defense categories -- output perturbation, data poisoning, and information throttling -- and evaluate nine defense configurations using a standardized pipeline with Qwen3-14B as teacher and Qwen2.5-7B-Instruct as student across three benchmarks (MATH-500, HumanEval+, MT-Bench). Our results reveal that, in a same-family distillation setting against a naive attacker, most output-level defenses are surprisingly ineffective: paraphrasing-based perturbation barely degrades distilled student quality, and data poisoning primarily impairs conversational fluency while leaving task-specific capabilities intact. Only chain-of-thought removal substantially impairs mathematical reasoning (31.4\% vs.\ 67.8\% baseline), though code generation remains unaffected. These findings demonstrate that the effectiveness of distillation defenses is highly task-dependent and that current output-level approaches are insufficient to broadly prevent knowledge theft.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
