当存疑时，级联：构建高效且强大的护栏模型
When in Doubt, Cascade: Towards Building Efficient and Capable Guardrails
AI总结：
针对大型语言模型生成有害文本的问题，本文提出基于使用-提及区分和分类法指令的合成数据生成流水线，生成30万+对比样本，以低成本构建高效且强大的护栏模型。
摘要：
大型语言模型（LLMs）在各种下游任务中表现出令人信服的性能。然而，这些系统容易生成不良输出，如有害和有偏见的文本。为了纠正此类生成，护栏（或检测器）模型的开发已获得关注。受开发社会偏见检测器时发现的启发，我们采用了使用-提及区分的概念——我们将其确定为初步版本社会偏见检测器性能不佳的主要来源。基于这一信息，我们描述了一个完全可扩展且可复现的合成数据生成流水线，该流水线利用基于分类法的指令来创建有针对性的标记数据。使用此流水线，我们生成了超过30万个独特的对比样本，并进行了广泛的实验，以系统评估在一套开源数据集上的性能。我们表明，我们的方法以一小部分计算成本实现了具有竞争力的性能，并为迭代开发高效且强大的护栏模型提供了见解。警告：本文包含有毒、有偏见且可能有害的文本示例。
英文摘要：
Large language models (LLMs) have convincing performance in a variety of downstream tasks. However, these systems are prone to generating undesirable outputs such as harmful and biased text. In order to remedy such generations, the development of guardrail (or detector) models has gained traction. Motivated by findings from developing a detector for social bias, we adopt the notion of a use-mention distinction - which we identified as the primary source of under-performance in the preliminary versions of our social bias detector. Armed with this information, we describe a fully extensible and reproducible synthetic data generation pipeline which leverages taxonomy-driven instructions to create targeted and labeled data. Using this pipeline, we generate over 300K unique contrastive samples and provide extensive experiments to systematically evaluate performance on a suite of open source datasets. We show that our method achieves competitive performance with a fraction of the cost in compute and offers insight into iteratively developing efficient and capable guardrail models. Warning: This paper contains examples of text which are toxic, biased, and potentially harmful.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
