arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.05445cs.CRcs.LG
收藏

基于知识的多轮对抗攻击大语言模型

Knowledge-Driven Multi-Turn Jailbreaking on Large Language Models

Southeast University(东南大学);Nanyang Technological University(南洋理工大学);OPPO Research Institute(OPPO研究院)

Songze Li, Ruishi He, Xiaojun Jia, Jun Wang, Zhihui Fu

AI总结:

Mastermind通过动态知识库和分层规划架构,实现对大语言模型的高效多轮对抗攻击,显著提升攻击效果和防御韧性。

摘要:

大型语言模型(LLMs)面临多轮对抗攻击的重大威胁,其中攻击者逐步引导对话以获取有害输出。然而,现有攻击的实际有效性受到几个关键限制的阻碍:它们难以在长时间互动中保持一致的进展,经常失去对已完成任务和剩余任务的跟踪;它们依赖于刚性或预定义的模式,无法适应LLM动态且不可预测的对话状态。为了解决这些不足,我们引入了Mastermind,一种多轮对抗攻击框架,采用动态和自我改进的方法。Mastermind operates在规划、执行和反思的闭环中,使其能够通过交互自主构建和改进对模型漏洞的知识。它采用分层规划架构,将高层攻击目标与低层战术执行解耦,确保长期关注和一致性。这种规划由一个知识库引导,该知识库通过反思交互经验自主发现和优化有效的攻击模式。Mastermind利用积累的知识动态重组和适应攻击向量,显著提高了有效性和韧性。我们对最先进的模型进行了全面实验,包括GPT-5和Claude 3.7 Sonnet。结果表明,Mastermind在现有基准上表现显著优于,实现了显著更高的攻击成功率和有害性评分。此外,我们的框架在多个高级防御机制面前表现出显著的韧性。

英文摘要:

Large Language Models (LLMs) face a significant threat from multi-turn jailbreak attacks, where adversaries progressively steer conversations to elicit harmful outputs. However, the practical effectiveness of existing attacks is undermined by several critical limitations: they struggle to maintain a coherent progression over long interactions, often losing track of what has been accomplished and what remains to be done; they rely on rigid or pre-defined patterns, and fail to adapt to the LLM's dynamic and unpredictable conversational state. To address these shortcomings, we introduce Mastermind, a multi-turn jailbreak framework that adopts a dynamic and self-improving approach. Mastermind operates in a closed loop of planning, execution, and reflection, enabling it to autonomously build and refine its knowledge of model vulnerabilities through interaction. It employs a hierarchical planning architecture that decouples high-level attack objectives from low-level tactical execution, ensuring long-term focus and coherence. This planning is guided by a knowledge repository that autonomously discovers and refines effective attack patterns by reflecting on interactive experiences. Mastermind leverages this accumulated knowledge to dynamically recombine and adapt attack vectors, dramatically improving both effectiveness and resilience. We conduct comprehensive experiments against state-of-the-art models, including GPT-5 and Claude 3.7 Sonnet. The results demonstrate that Mastermind significantly outperforms existing baselines, achieving substantially higher attack success rates and harmfulness ratings. Moreover, our framework exhibits notable resilience against multiple advanced defense mechanisms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑