arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.14866cs.CLcs.CRcs.LG
收藏

对齐大语言模型上的对抗后缀迁移学习研究进展

Advancing Adversarial Suffix Transfer Learning on Aligned Large Language Models

National University of Singapore(新加坡国立大学)

Hongfu Liu, Yuxi Xie, Ye Wang, Michael Shieh

AI总结:

针对大语言模型对抗后缀搜索算法GCG效率低的问题,提出两阶段迁移学习框架DeGCG及其交错变体i-DeGCG,通过解耦搜索流程提升效率,在HarmBench基准上实现了攻击成功率的显著提升。

摘要:

大语言模型(LLMs)因存在被恶意用户滥用的可能性而面临安全隐患。近期的红队研究已发现,利用基于梯度的搜索算法Greedy Coordinate Gradient(GCG,贪心坐标梯度)生成的对抗后缀能够破解LLMs的安全防护。但GCG存在计算效率低下的问题,限制了针对后缀在不同模型与数据间的可迁移性及可扩展性的进一步研究。\n本研究打通了搜索效率与后缀可迁移性之间的关联,提出了两阶段迁移学习框架DeGCG,将搜索过程解耦为与行为无关的预搜索阶段和与行为相关的后搜索阶段。具体而言,我们在预搜索阶段采用直接首个目标词元优化来推进搜索进程,并将该方法应用于跨模型、跨数据以及自迁移场景。此外,我们还提出了该方法的交错变体i-DeGCG,通过迭代利用自迁移性来加速搜索过程。\n在HarmBench基准上的实验验证了我们的方法在不同模型与领域中的效率。值得注意的是,在Llama2-chat-7b模型上,我们的i-DeGCG性能优于基线,其在验证集和测试集上的攻击成功率(ASR)分别为43.9(提升22.2)和39.0(提升19.5)。针对跨模型迁移的进一步分析表明,首个目标词元优化在利用后缀可迁移性实现高效搜索方面发挥着关键作用。

英文摘要:

Language Language Models (LLMs) face safety concerns due to potential misuse by malicious users. Recent red-teaming efforts have identified adversarial suffixes capable of jailbreaking LLMs using the gradient-based search algorithm Greedy Coordinate Gradient (GCG). However, GCG struggles with computational inefficiency, limiting further investigations regarding suffix transferability and scalability across models and data. In this work, we bridge the connection between search efficiency and suffix transferability. We propose a two-stage transfer learning framework, DeGCG, which decouples the search process into behavior-agnostic pre-searching and behavior-relevant post-searching. Specifically, we employ direct first target token optimization in pre-searching to facilitate the search process. We apply our approach to cross-model, cross-data, and self-transfer scenarios. Furthermore, we introduce an interleaved variant of our approach, i-DeGCG, which iteratively leverages self-transferability to accelerate the search process. Experiments on HarmBench demonstrate the efficiency of our approach across various models and domains. Notably, our i-DeGCG outperforms the baseline on Llama2-chat-7b with ASRs of $43.9$ ($+22.2$) and $39.0$ ($+19.5$) on valid and test sets, respectively. Further analysis on cross-model transfer indicates the pivotal role of first target token optimization in leveraging suffix transferability for efficient searching.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑