Sentinel：防御提示注入攻击的SOTA模型
Sentinel: SOTA model to protect against prompt injections
AI总结：
本文提出基于ModernBERT-large微调的提示注入检测模型Sentinel，通过融合多样攻击与良性指令数据集训练，在内部测试和公开基准上均取得SOTA性能。
摘要：
大型语言模型（LLMs）日益强大，但仍易受提示注入攻击的影响，恶意输入会使模型偏离其预期指令。本文介绍了Sentinel，一种基于answerdotai/ModernBERT-large架构的新型检测模型，项目地址为qualifire/prompt-injection-sentinel。通过利用ModernBERT的先进特性，并在由若干开源和私有数据集组成的广泛且多样化的数据集上进行微调，Sentinel实现了最先进的性能。该数据集融合了多种攻击类型，从角色扮演和指令劫持到生成有偏见内容的尝试，以及广泛的良性指令，其中私有数据集专门针对细微的错误纠正和现实世界中的误分类。在一个全面的、未见过的内部测试集上，Sentinel展示了0.987的平均准确率和0.980的F1分数。此外，在公开基准测试上评估时，它始终优于protectai/deberta-v3-base-prompt-injection-v2等强基线模型。本文详细介绍了Sentinel的架构、精细的数据集构建、训练方法以及全面的评估，突出了其卓越的检测能力。
英文摘要：
Large Language Models (LLMs) are increasingly powerful but remain vulnerable to prompt injection attacks, where malicious inputs cause the model to deviate from its intended instructions. This paper introduces Sentinel, a novel detection model, qualifire/prompt-injection-sentinel, based on the \answerdotai/ModernBERT-large architecture. By leveraging ModernBERT's advanced features and fine-tuning on an extensive and diverse dataset comprising a few open-source and private collections, Sentinel achieves state-of-the-art performance. This dataset amalgamates varied attack types, from role-playing and instruction hijacking to attempts to generate biased content, alongside a broad spectrum of benign instructions, with private datasets specifically targeting nuanced error correction and real-world misclassifications. On a comprehensive, unseen internal test set, Sentinel demonstrates an average accuracy of 0.987 and an F1-score of 0.980. Furthermore, when evaluated on public benchmarks, it consistently outperforms strong baselines like protectai/deberta-v3-base-prompt-injection-v2. This work details Sentinel's architecture, its meticulous dataset curation, its training methodology, and a thorough evaluation, highlighting its superior detection capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
