SafetyPrompts：对用于评估和改进大语言模型安全性的开放数据集的系统综述
SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety
AI总结：
本文系统综述144个LLM安全开放数据集，揭示合成化趋势、非英语与自然场景数据缺失及评估使用碎片化问题，并推出持续更新的SafetyPrompts.com目录。
摘要：
过去两年中，围绕大语言模型（LLM）安全性的担忧迅速增长。研究人员和从业者通过创建大量用于评估和改进LLM安全性的数据集来回应这些担忧。然而，这些工作大多并行开展，且目标差异很大，范围从缓解围绕偏见和有毒内容生成的近期风险，到评估更长期的灾难性风险潜力。这使研究人员和从业者难以为其用例找到最相关的数据集，也难以识别未来工作可以填补的数据集覆盖空白。为弥补这些问题，我们对用于评估和改进LLM安全性的开放数据集进行了首次系统综述。我们综述了144个数据集，这些数据集是在数月时间里通过一个迭代式、社区驱动的流程识别出来的。我们强调了若干模式和趋势，例如趋向完全合成数据集的趋势，以及数据集覆盖方面的空白，例如明显缺乏非英语和自然主义数据集。我们还考察了LLM安全数据集在实践中如何被使用——在LLM发布论文和流行的LLM基准中——发现当前评估实践高度特殊化，并且只使用了可用数据集中的一小部分。我们的贡献基于SafetyPrompts.com，这是一个LLM安全开放数据集的活目录，我们计划随着LLM安全领域的发展对其进行持续更新。
英文摘要：
The last two years have seen a rapid growth in concerns around the safety of large language models (LLMs). Researchers and practitioners have met these concerns by creating an abundance of datasets for evaluating and improving LLM safety. However, much of this work has happened in parallel, and with very different goals in mind, ranging from the mitigation of near-term risks around bias and toxic content generation to the assessment of longer-term catastrophic risk potential. This makes it difficult for researchers and practitioners to find the most relevant datasets for their use case, and to identify gaps in dataset coverage that future work may fill. To remedy these issues, we conduct a first systematic review of open datasets for evaluating and improving LLM safety. We review 144 datasets, which we identified through an iterative and community-driven process over the course of several months. We highlight patterns and trends, such as a trend towards fully synthetic datasets, as well as gaps in dataset coverage, such as a clear lack of non-English and naturalistic datasets. We also examine how LLM safety datasets are used in practice -- in LLM release publications and popular LLM benchmarks -- finding that current evaluation practices are highly idiosyncratic and make use of only a small fraction of available datasets. Our contributions are based on SafetyPrompts.com, a living catalogue of open datasets for LLM safety, which we plan to update continuously as the field of LLM safety develops.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
