arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.10869cs.CR
收藏

代理知识蒸馏:自主训练小型语言模型用于短信威胁检测

Agentic Knowledge Distillation: Autonomous Training of Small Language Models for SMS Threat Detection

University of Kent(肯特大学)

Adel ElZemity, Joshua Sylvester, Budi Arief, Rogério De Lemos

AI总结:

本研究提出代理知识蒸馏方法,通过自主训练小型语言模型实现高效的短信威胁检测,实验显示教师LLM的选择对性能影响显著。

摘要:

基于短信的钓鱼攻击(smishing)已显著增加,但训练有效的设备端检测器需要标记的威胁数据,这些数据很快会过时。为了解决这个问题,我们提出了代理知识蒸馏,其中一个强大的LLM充当自主教师,微调一个较小的学生SLM,可用于安全任务而无需人工干预。教师LLM自主生成合成数据,并迭代优化较小的设备端学生模型,直到性能达到平台期。我们在此教师角色上比较了四种LLM(Claude Opus 4.5、GPT 5.2 Codex、Gemini 3 Pro和DeepSeek V3.2)在短信垃圾/钓鱼检测中的表现,与两种学生SLM(Qwen2.5-0.5B和SmolLM2-135M)。我们的结果表明,性能在很大程度上取决于教师LLM的选择,最佳配置实现了94.31%的准确率和96.25%的召回率。我们还与一个直接偏好优化(DPO)基线进行了比较,该基线使用相同的合成知识和LoRA设置,但没有迭代反馈或针对性优化;代理知识蒸馏显著优于它(例如,86-94% vs 50-80%准确率),表明闭环反馈和针对性优化是关键。这些发现表明,代理知识蒸馏可以快速产生有效的安全分类器用于边缘部署,但结果在很大程度上取决于所使用的教师LLM。

英文摘要:

SMS-based phishing (smishing) attacks have surged, yet training effective on-device detectors requires labelled threat data that quickly becomes outdated. To deal with this issue, we present Agentic Knowledge Distillation, which consists of a powerful LLM acts as an autonomous teacher that fine-tunes a smaller student SLM, deployable for security tasks without human intervention. The teacher LLM autonomously generates synthetic data and iteratively refines a smaller on-device student model until performance plateaus. We compare four LLMs in this teacher role (Claude Opus 4.5, GPT 5.2 Codex, Gemini 3 Pro, and DeepSeek V3.2) on SMS spam/smishing detection with two student SLMs (Qwen2.5-0.5B and SmolLM2-135M). Our results show that performance varies substantially depending on the teacher LLM, with the best configuration achieving 94.31% accuracy and 96.25% recall. We also compare against a Direct Preference Optimisation (DPO) baseline that uses the same synthetic knowledge and LoRA setup but without iterative feedback or targeted refinement; agentic knowledge distillation substantially outperforms it (e.g. 86-94% vs 50-80% accuracy), showing that closed-loop feedback and targeted refinement are critical. These findings demonstrate that agentic knowledge distillation can rapidly yield effective security classifiers for edge deployment, but outcomes depend strongly on which teacher LLM is used.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑