DESTEIN:通过通用引导对与逐头激活融合实现语言模型去毒化导航
DESTEIN: Navigating Detoxification of Language Models via Universal Steering Pairs and Head-wise Activation Fusion
AI总结:
本文提出DeStein方法,通过激活空间中的通用引导对推导去毒化向量,并以逐头方式融合原始表征,从而在低资源成本下实现大语言模型去毒化,实验证明其性能优于现有方法。
摘要:
尽管语言模型(LMs)在广泛任务中取得了显著成就,但其生成有毒输出的倾向仍然是一个普遍关注的问题。当前涉及微调或辅助模型的解决方案通常需要大量计算资源,阻碍了它们在大语言模型(LLMs)中的实用性。在本文中,我们提出了DeStein,这是一种新颖的方法,通过在激活空间中应用表征工程,以较低的资源成本和时间成本对LMs进行去毒化。具体来说,我们通过激活空间中的算术运算,从自诱导的通用引导对中推导出去毒化向量。在推理过程中,通过以逐头(head-wise)方式将去毒化向量与原始表征进行融合来实现去毒化。实验结果表明,我们的方法在各种指标上显著优于先前的最先进方法,同时还保持了令人满意的生成质量和多样性。我们进一步通过一系列白盒LLMs验证了DeStein的实用性和可扩展性。该方法已在 https://github.com/LizLizLi/DeStein 开源。警告:一些示例模型输出可能包含高度冒犯性或令人不安的文本。
英文摘要:
Despite the remarkable achievements of language models (LMs) across a broad spectrum of tasks, their propensity for generating toxic outputs remains a prevalent concern. Current solutions involving finetuning or auxiliary models usually require extensive computational resources, hindering their practicality in large language models (LLMs). In this paper, we propose DeStein, a novel method that detoxifies LMs by applying representation engineering in activation spaces with lower resource and time costs. Specifically, we derive detoxification vectors from self-induced, universal steering pairs through arithmetic operations in activation spaces. During inference, detoxification is achieved by fusing the detoxification vectors with the original representations in a head-wise manner. Empirical results demonstrate that our method significantly outperforms previous state-of-the-art approaches on various metrics, while also maintaining satisfactory generation quality and diversity. We further validate the practicality and scalability of DeStein with a series of white-box LLMs. The method is open-sourced at https://github.com/LizLizLi/DeStein. Warning: Some example model outputs may contain highly offensive or disturbing text.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。