通过激活转向研究Llama 2 Chat中的偏见表征
Investigating Bias Representations in Llama 2 Chat via Activation Steering
AI总结：
本研究以Llama 2 7B Chat为对象，用激活转向技术探测缓解社会偏见，发现其经RLHF后仍有性别偏见，且RLHF会提升不同偏见表征的相似性，同时为LLM红队测试策略提供了参考。
摘要：
我们针对大语言模型（LLM）中的社会偏见挑战展开研究，聚焦Llama 2 7B Chat模型。随着LLM日益融入具有重大社会影响的决策流程，确保这类模型不会加剧现有偏见已成为当务之急。我们的方法采用激活转向技术，来探测并缓解与性别、种族、宗教相关的偏见。该方法通过操控模型激活值，引导响应偏向或偏离有偏见的输出，所使用的转向向量源自StereoSet数据集以及GPT4生成的定制性别偏见提示词。我们的研究结果显示，Llama 2 7B Chat存在固有的性别偏见，即便经过人类反馈强化学习（RLHF）后该偏见依然存在。我们还观察到，偏见程度与模型拒绝响应的倾向之间存在可预测的负相关关系。值得注意的是，本研究发现RLHF往往会提升模型对不同形式社会偏见表征的相似性，这引发了关于模型对不同偏见形式是否具备精细化理解的疑问。这项工作还为利用激活转向开展LLM有效的红队测试策略提供了宝贵见解，尤其强调了整合拒绝向量的重要性。
英文摘要：
We address the challenge of societal bias in Large Language Models (LLMs), focusing on the Llama 2 7B Chat model. As LLMs are increasingly integrated into decision-making processes with substantial societal impact, it becomes imperative to ensure these models do not reinforce existing biases. Our approach employs activation steering to probe for and mitigate biases related to gender, race, and religion. This method manipulates model activations to direct responses towards or away from biased outputs, utilizing steering vectors derived from the StereoSet dataset and custom GPT4 generated gender bias prompts. Our findings reveal inherent gender bias in Llama 2 7B Chat, persisting even after Reinforcement Learning from Human Feedback (RLHF). We also observe a predictable negative correlation between bias and the model's tendency to refuse responses. Significantly, our study uncovers that RLHF tends to increase the similarity in the model's representation of different forms of societal biases, which raises questions about the model's nuanced understanding of different forms of bias. This work also provides valuable insights into effective red-teaming strategies for LLMs using activation steering, particularly emphasizing the importance of integrating a refusal vector.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
