arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.08243cs.CLcs.CY
收藏

超越安全捆绑包:审计 Helpful and Harmless 数据集

Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset

McGill University(麦吉尔大学);Mila - Quebec AI Institute(魁北克人工智能研究所)

Khaoula Chehbouni, Jonathan Colaço Carr, Yash More, Jackie CK Cheung, Golnoosh Farnadi

AI总结:

本文审计 Anthropic 的 Helpful and Harmless 数据集,通过内容评估、安全性实验和引用论文分析,揭示其概念化缺陷与质量问题如何导致不同群体间的差异化安全行为,并呼吁采用更细致且上下文敏感的安全缓解方法。

摘要:

为了减轻大型语言模型(LLMs)的危害,从人类反馈中学习(LHF)已被用于引导 LLMs 输出既更少有害又更有帮助的内容。尽管 LHF 在实践中被广泛采用,但这类反馈的质量及其作为安全缓解技术的有效性仍不明确。本研究通过审计 Anthropic 广泛使用的 Helpful and Harmless(HH)数据集来探讨这些问题。我们的工作包括:(1)通过人工和自动评估对该数据集的内容进行全面调查;(2)通过实验展示该数据集对模型安全性的影响;(3)对引用该数据集的 100 篇最具影响力论文进行分析。通过审计,我们展示了 HH 数据集中发现的概念化缺陷和质量问题如何导致不同人口群体之间出现差异化的安全行为,从而产生额外危害。我们的发现强调了在 LLMs 中需要更细致、更具上下文敏感性的安全缓解方法。

英文摘要:

In an effort to mitigate the harms of large language models (LLMs), learning from human feedback (LHF) has been used to steer LLMs towards outputs that are intended to be both less harmful and more helpful. Despite the widespread adoption of LHF in practice, the quality of this feedback and its effectiveness as a safety mitigation technique remain unclear. This study addresses these issues by auditing the widely-used Helpful and Harmless (HH) dataset by Anthropic. Our work includes: (1) a thorough investigation of the dataset's content through both manual and automated evaluation; (2) experiments demonstrating the dataset's impact on models' safety; and (3) an analysis of the 100 most influential papers citing this dataset. Through our audit, we showcase how conceptualization failures and quality issues identified in the HH dataset can create additional harms by leading to disparate safety behaviors across demographic groups. Our findings highlight the need for more nuanced, context-sensitive approaches to safety mitigation in LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑