BeaverTails：通过人类偏好数据集改进大语言模型安全对齐的研究
BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
AI总结：
本研究推出BeaverTails人类偏好数据集，分离标注问答对的有用性与无害性，含超33万QA对安全标签及36万余专家对比数据，可用于内容审核与RLHF，为大语言模型安全对齐研究与落地提供关键资源。
摘要：
本文介绍了BeaverTails数据集，旨在推动大语言模型（LLM）安全对齐领域的研究。该数据集的独特之处在于将问答对的有用性和无害性标注分开，从而为研究这两项关键属性提供了不同视角。我们共收集了333963个问答（QA）对的安全元标签，以及361903对涵盖有用性和无害性两个指标的专家对比数据。我们进一步展示了BeaverTails在内容审核和人类反馈强化学习（RLHF）中的应用，凸显了其在LLM实际安全防护措施方面的潜力。我们认为该数据集为研究社区提供了重要资源，有助于LLM的安全开发与部署。项目页面地址为：https://sites.google.com/view/pku-beavertails。
英文摘要：
In this paper, we introduce the BeaverTails dataset, aimed at fostering research on safety alignment in large language models (LLMs). This dataset uniquely separates annotations of helpfulness and harmlessness for question-answering pairs, thus offering distinct perspectives on these crucial attributes. In total, we have gathered safety meta-labels for 333,963 question-answer (QA) pairs and 361,903 pairs of expert comparison data for both the helpfulness and harmlessness metrics. We further showcase applications of BeaverTails in content moderation and reinforcement learning with human feedback (RLHF), emphasizing its potential for practical safety measures in LLMs. We believe this dataset provides vital resources for the community, contributing towards the safe development and deployment of LLMs. Our project page is available at the following URL: https://sites.google.com/view/pku-beavertails.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
