表示加噪:一种抵御有害微调的防御机制
Representation Noising: A Defence Mechanism Against Harmful Finetuning
AI总结:
针对大语言模型易遭有害微调的风险,提出防御机制RepNoise,通过移除有害表示信息抵御攻击,可泛化到同分布未知有害子集,不损害模型通用能力与无害任务训练能力,同时指出其局限性以指导后续研究。
摘要:
开源大语言模型(LLM)的发布存在两用风险,因为不良行为者可以轻松地对这些模型进行微调以实现有害目的。即便没有公开发布权重,权重窃取与微调API也会让闭源模型易受有害微调攻击(HFA)的影响。虽然防止越狱、改进安全护栏等安全措施十分重要,但这类措施很容易通过微调被逆转。在本研究中,我们提出了表示加噪(RepNoise)——一种即便攻击者能获取模型权重也能生效的防御机制。RepNoise的工作原理是移除有害表示的相关信息,使其在微调过程中难以被恢复。重要的是,只要未见过的有害子集与攻击集来自同一分布,我们的防御方法还能泛化到这些防御过程中未见过的不同有害子集上。我们的方法不会降低LLM的通用能力,且保留了模型在无害任务上的训练能力。我们提供了实证证据,表明该防御的有效性在于其“深度”:即有害表示的信息在LLM所有层中的移除程度。我们还发现了RepNoise仍存在失效的领域,并指出这些局限性可为未来研究提供参考。
英文摘要:
Releasing open-source large language models (LLMs) presents a dual-use risk since bad actors can easily fine-tune these models for harmful purposes. Even without the open release of weights, weight stealing and fine-tuning APIs make closed models vulnerable to harmful fine-tuning attacks (HFAs). While safety measures like preventing jailbreaks and improving safety guardrails are important, such measures can easily be reversed through fine-tuning. In this work, we propose Representation Noising (RepNoise), a defence mechanism that operates even when attackers have access to the weights. RepNoise works by removing information about harmful representations such that it is difficult to recover them during fine-tuning. Importantly, our defence is also able to generalize across different subsets of harm that have not been seen during the defence process as long as they are drawn from the same distribution of the attack set. Our method does not degrade the general capability of LLMs and retains the ability to train the model on harmless tasks. We provide empirical evidence that the efficacy of our defence lies in its ``depth'': the degree to which information about harmful representations is removed across all layers of the LLM. We also find areas where RepNoise still remains ineffective and highlight how those limitations can inform future research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。