arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.01644cs.CLcs.AIcs.CY
收藏

机器学习用于新型LLM越狱的检测与分析

Machine Learning for Detection and Analysis of Novel LLM Jailbreaks

Centre for Artificial Intelligence and Innovation(人工智能与创新中心);Pingla Institute(平拉研究所);Transitional Artificial Intelligence Research Group(过渡人工智能研究组);UNSW(新南威尔士大学)

John Hawkins, Aditya Pramar, Rodney Beard, Rohitash Chandra

AI总结:

本研究分析多种机器学习模型检测新型LLM越狱提示的能力,发现端到端微调BERT模型性能最佳,并揭示提示中的显式反身性可作为越狱意图的信号。

摘要:

大型语言模型(LLMs)存在一系列漏洞,使得恶意用户能够通过操纵输入文本诱导模型产生不良回应。这些所谓的越狱提示旨在欺骗LLM绕过为保持回应符合开发者政策而设置的安全护栏。在本研究中,我们分析了不同机器学习模型区分越狱提示与正常使用的能力,包括识别采用先前未见策略的越狱的能力。我们的结果表明,在使用当前数据集的情况下,最佳性能是通过端到端微调一个基于Transformer的双向编码器表示(BERT)模型来识别越狱而实现的。我们可视化了区分越狱与正常提示的关键词,并得出结论:提示结构中显式的反身性可能是越狱意图的一个信号。

英文摘要:

Large Language Models (LLMs) suffer from a range of vulnerabilities that allow malicious users to solicit undesirable responses through manipulation of the input text. These so-called jailbreak prompts are designed to trick the LLM into circumventing the safety guardrails put in place to keep responses acceptable to the developer's policies. In this study, we analyse the ability of different machine learning models to distinguish jailbreak prompts from genuine uses, including looking at our ability to identify jailbreaks that use previously unseen strategies. Our results indicate that using current datasets the best performance is achieved by fine tuning a Bidirectional Encoder Representations from Transformers (BERT) model end-to-end for identifying jailbreaks. We visualise the keywords that distinguish jailbreak from genuine prompts and conclude that explicit reflexivity in prompt structure could be a signal of jailbreak intention.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑