覆盖开源模型的安全保护机制
Overriding Safety protections of Open-source Models
AI总结:
本文研究了微调数据对开源大语言模型安全保护的影响,发现有害数据微调可使ASR增加35%并导致知识漂移,而安全数据微调可使ASR降低51.68%。
摘要:
大语言模型(LLMs)如今已被广泛用作解决各领域/任务问题的工具。由于这些模型容易产生有害或有毒的结果,并面临推理时对抗性攻击,因此它们会经过安全对齐训练和红队测试,以设置安全防护栏。在使用这些模型时,通常会对模型进行微调以使其对齐所需任务,这可以使模型更加对齐,但如果用有害数据进行微调,也会使其更容易产生不安全响应。本文研究了在微调中引入有害数据能产生多大影响,以及它是否能够覆盖这些模型的安全保护。相反地,本文也探索了如果模型在安全数据上进行微调,是否能使模型产生更安全的响应。此外,我们进一步探索了用有害数据微调模型是否会因模型不确定性增加导致知识漂移,从而使模型变得不那么有用或不那么可信。我们广泛的实验结果表明,开源模型中的安全保护可以被覆盖,当用有害数据微调时,观察到ASR相比基础模型的ASR增加了35%。同时,观察到用有害数据微调模型会使有害微调模型高度不确定,出现巨大的知识漂移,并且其响应的真实性更低。此外,对于安全微调模型,ASR相比基础模型下降了51.68%,安全模型在不确定性和真实性方面相比基础模型也表现出轻微下降。本文代码可在以下地址获取:https://github.com/techsachinkr/Overriding_Model_Safety_Protections
英文摘要:
LLMs(Large Language Models) nowadays have widespread adoption as a tool for solving issues across various domain/tasks. These models since are susceptible to produce harmful or toxic results, inference-time adversarial attacks, therefore they do undergo safety alignment training and Red teaming for putting in safety guardrails. For using these models, usually fine-tuning is done for model alignment on the desired tasks, which can make model more aligned but also make it more susceptible to produce unsafe responses, if fine-tuned with harmful data.In this paper, we study how much of impact introduction of harmful data in fine-tuning can make, and if it can override the safety protection of those models. Conversely,it was also explored that if model is fine-tuned on safety data can make the model produce more safer responses. Further we explore if fine-tuning the model on harmful data makes it less helpful or less trustworthy because of increase in model uncertainty leading to knowledge drift. Our extensive experimental results shown that Safety protection in an open-source can be overridden, when fine-tuned with harmful data as observed by ASR increasing by 35% when compared to basemodel's ASR. Also, as observed, fine-tuning a model with harmful data made the harmful fine-tuned model highly uncertain with huge knowledge drift and less truthfulness in its responses. Furthermore, for the safe fine-tuned model, ASR decreases by 51.68% as compared to the basemodel, and Safe model also shown in minor drop in uncertainty and truthfulness as compared to basemodel. This paper's code is available at: https://github.com/techsachinkr/Overriding_Model_Safety_Protections
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。