arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.12460cs.CRcs.LG
收藏

TrojanPraise: 通过良性微调劫持LLM

TrojanPraise: Jailbreak LLMs via Benign Fine-Tuning

Nanyang Technological University(南洋理工大学)

Zhixin Xie, Xurui Song, Jun Luo

AI总结:

TrojanPraise通过良性微调利用过滤通过的数据,使LLM在无意识中顺从有害概念,成功率达95.88%

摘要:

定制化大型语言模型(LLM)的需求促使商业LLM提供黑盒微调API,但这种便利性引入了关键的安全漏洞:攻击者可以通过使用恶意数据微调LLM来劫持LLM。尽管这一安全问题最近已被曝光,但此类攻击的可行性存疑,因为恶意训练数据被认为可以被如Llama-Guard-3等审核模型检测到。在本文中,我们提出TrojanPraise,一种基于微调的攻击方法,利用良性且通过过滤的数据。基本上,TrojanPraise微调模型以使一个精心设计的词(例如“bruaf”)与无害的含义相关联,然后使用这个词来称赞有害的概念,从而微妙地使LLM从拒绝转变为顺从。为了解释攻击,我们将LLM对查询的内部表示分解为知识和态度两个维度。我们证明,成功的劫持需要改变态度而避免知识变化,这是模型对概念理解的扭曲。为了验证这一攻击,我们在严格的黑盒设置下对五个开源LLM和两个商业LLM进行了实验。结果表明,TrojanPraise在逃避审核的情况下实现了95.88%的最大攻击成功率。

英文摘要:

The demand of customized large language models (LLMs) has led to commercial LLMs offering black-box fine-tuning APIs, yet this convenience introduces a critical security loophole: attackers could jailbreak the LLMs by fine-tuning them with malicious data. Though this security issue has recently been exposed, the feasibility of such attacks is questionable as malicious training dataset is believed to be detectable by moderation models such as Llama-Guard-3. In this paper, we propose TrojanPraise, a novel finetuning-based attack exploiting benign and thus filter-approved data. Basically, TrojanPraise fine-tunes the model to associate a crafted word (e.g., "bruaf") with harmless connotations, then uses this word to praise harmful concepts, subtly shifting the LLM from refusal to compliance. To explain the attack, we decouple the LLM's internal representation of a query into two dimensions of knowledge and attitude. We demonstrate that successful jailbreak requires shifting the attitude while avoiding knowledge shift, a distortion in the model's understanding of the concept. To validate this attack, we conduct experiments on five opensource LLMs and two commercial LLMs under strict black-box settings. Results show that TrojanPraise achieves a maximum attack success rate of 95.88% while evading moderation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑