arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.13660cs.CL
收藏

大型语言模型中的木马检测:来自木马检测挑战赛的洞察

Trojan Detection in Large Language Models: Insights from The Trojan Detection Challenge

Narek Maloyan, Ekansh Verma, Bulat Nutfullin, Bislan Ashinov

AI总结:

本文基于TDC2023挑战赛,研究了大型语言模型的木马检测问题,发现实现高Recall比高REASR更难,揭示了非预期触发器区分的困难,为未来LLM安全研究奠定基础。

摘要:

大型语言模型(LLMs)在各个领域展现出了卓越的能力,但其对木马或后门攻击的脆弱性带来了重大的安全风险。本文探讨了从Trojan Detection Competition 2023(TDC2023)中获得的挑战与洞察,该竞赛专注于识别和评估针对LLMs的木马攻击。我们研究了区分预期触发器与非预期触发器的难度,以及在真实场景中对木马进行逆向工程的可行性。我们对各种木马检测方法的比较分析表明,实现高Recall分数比获得高Reverse-Engineering Attack Success Rate(REASR)分数要困难得多。竞赛中表现最佳的方法取得了约0.16的Recall分数,这与从类似于给定训练前缀的分布中随机采样句子的简单基线相当。这一发现提出了一个问题:在仅给定有害目标的情况下,插入模型的木马的可检测性与可恢复性如何。尽管未能完全解决该问题,但竞赛带来了关于木马检测可行性的有趣观察,以及优化LLM输入提示词的改进技术。非预期触发器的现象以及将其与预期触发器区分开来的困难,突显了进一步研究LLM鲁棒性和可解释性的必要性。TDC2023为LLM木马检测相关的挑战和机遇提供了宝贵见解,为该领域的未来研究奠定了基础,以确保其在实际应用中的安全性和可靠性。

英文摘要:

Large Language Models (LLMs) have demonstrated remarkable capabilities in various domains, but their vulnerability to trojan or backdoor attacks poses significant security risks. This paper explores the challenges and insights gained from the Trojan Detection Competition 2023 (TDC2023), which focused on identifying and evaluating trojan attacks on LLMs. We investigate the difficulty of distinguishing between intended and unintended triggers, as well as the feasibility of reverse engineering trojans in real-world scenarios. Our comparative analysis of various trojan detection methods reveals that achieving high Recall scores is significantly more challenging than obtaining high Reverse-Engineering Attack Success Rate (REASR) scores. The top-performing methods in the competition achieved Recall scores around 0.16, comparable to a simple baseline of randomly sampling sentences from a distribution similar to the given training prefixes. This finding raises questions about the detectability and recoverability of trojans inserted into the model, given only the harmful targets. Despite the inability to fully solve the problem, the competition has led to interesting observations about the viability of trojan detection and improved techniques for optimizing LLM input prompts. The phenomenon of unintended triggers and the difficulty in distinguishing them from intended triggers highlights the need for further research into the robustness and interpretability of LLMs. The TDC2023 has provided valuable insights into the challenges and opportunities associated with trojan detection in LLMs, laying the groundwork for future research in this area to ensure their safety and reliability in real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑