Kov：基于马尔可夫决策过程与树搜索的可迁移、自然化黑盒大语言模型攻击
Kov: Transferable and Naturalistic Black-Box LLM Attacks using Markov Decision Processes and Tree Search
AI总结：
该研究提出Kov算法，将红队问题建模为MDP并结合蒙特卡洛树搜索，通过白盒LLM优化token级对抗提示后缀并加入自然化损失，仅用10次查询即可越狱GPT-3.5，验证了新模型对token级攻击的更强鲁棒性，相关代码已开源。
摘要：
诱导大语言模型（LLM）产生有害行为是确保模型正确对齐与安全性的重要任务。通常在训练LLM时会遵循伦理准则，但对齐失效仍可通过红队对抗攻击被发现。本研究将红队问题建模为马尔可夫决策过程（MDP），并使用蒙特卡洛树搜索来发现黑盒、闭源LLM的有害行为。我们在白盒LLM上针对目标有害行为优化token级提示后缀，并加入自然化损失项对数困惑度（log-perplexity），以生成更自然的语言攻击，提升可解释性。所提出的算法Kov在白盒LLM上训练以优化对抗攻击，并定期评估黑盒LLM的响应，引导搜索朝着更具危害性的黑盒行为方向推进。在我们的初步研究中，结果表明仅需10次查询即可越狱GPT-3.5等黑盒模型，但对GPT-4无效——这可能表明更新的模型对token级攻击具有更强的鲁棒性。所有可复现这些结果的工作均已开源（https://github.com/sisl/Kov.jl）。
英文摘要：
Eliciting harmful behavior from large language models (LLMs) is an important task to ensure the proper alignment and safety of the models. Often when training LLMs, ethical guidelines are followed yet alignment failures may still be uncovered through red teaming adversarial attacks. This work frames the red-teaming problem as a Markov decision process (MDP) and uses Monte Carlo tree search to find harmful behaviors of black-box, closed-source LLMs. We optimize token-level prompt suffixes towards targeted harmful behaviors on white-box LLMs and include a naturalistic loss term, log-perplexity, to generate more natural language attacks for better interpretability. The proposed algorithm, Kov, trains on white-box LLMs to optimize the adversarial attacks and periodically evaluates responses from the black-box LLM to guide the search towards more harmful black-box behaviors. In our preliminary study, results indicate that we can jailbreak black-box models, such as GPT-3.5, in only 10 queries, yet fail on GPT-4$-$which may indicate that newer models are more robust to token-level attacks. All work to reproduce these results is open sourced (https://github.com/sisl/Kov.jl).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
