arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.10464cs.CLcs.AI
收藏

DESTEIN:通过通用引导对与逐头激活融合实现语言模型去毒化导航

DESTEIN: Navigating Detoxification of Language Models via Universal Steering Pairs and Head-wise Activation Fusion

Tongji University(同济大学)

Yu Li, Han Jiang, Chuanyang Gong, Zhihua Wei

AI总结:

本文提出DeStein方法,通过激活空间中的通用引导对推导去毒化向量,并以逐头方式融合原始表征,从而在低资源成本下实现大语言模型去毒化,实验证明其性能优于现有方法。

摘要:

尽管语言模型(LMs)在广泛任务中取得了显著成就,但其生成有毒输出的倾向仍然是一个普遍关注的问题。当前涉及微调或辅助模型的解决方案通常需要大量计算资源,阻碍了它们在大语言模型(LLMs)中的实用性。在本文中,我们提出了DeStein,这是一种新颖的方法,通过在激活空间中应用表征工程,以较低的资源成本和时间成本对LMs进行去毒化。具体来说,我们通过激活空间中的算术运算,从自诱导的通用引导对中推导出去毒化向量。在推理过程中,通过以逐头(head-wise)方式将去毒化向量与原始表征进行融合来实现去毒化。实验结果表明,我们的方法在各种指标上显著优于先前的最先进方法,同时还保持了令人满意的生成质量和多样性。我们进一步通过一系列白盒LLMs验证了DeStein的实用性和可扩展性。该方法已在 https://github.com/LizLizLi/DeStein 开源。警告:一些示例模型输出可能包含高度冒犯性或令人不安的文本。

英文摘要:

Despite the remarkable achievements of language models (LMs) across a broad spectrum of tasks, their propensity for generating toxic outputs remains a prevalent concern. Current solutions involving finetuning or auxiliary models usually require extensive computational resources, hindering their practicality in large language models (LLMs). In this paper, we propose DeStein, a novel method that detoxifies LMs by applying representation engineering in activation spaces with lower resource and time costs. Specifically, we derive detoxification vectors from self-induced, universal steering pairs through arithmetic operations in activation spaces. During inference, detoxification is achieved by fusing the detoxification vectors with the original representations in a head-wise manner. Empirical results demonstrate that our method significantly outperforms previous state-of-the-art approaches on various metrics, while also maintaining satisfactory generation quality and diversity. We further validate the practicality and scalability of DeStein with a series of white-box LLMs. The method is open-sourced at https://github.com/LizLizLi/DeStein. Warning: Some example model outputs may contain highly offensive or disturbing text.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑