arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.05339cs.LG
收藏

具有鲁棒防护栏的分类适应调谐模型用于大型语言模型

Taxonomy-Adaptive Moderation Model with Robust Guardrails for Large Language Models

Project Lead(项目负责人)

Mahesh Kumar Nandwana, Youngwan Lim, Joseph Liu, Alex Yang, Varun Notibala, Nishchaie Khanna

AI总结:

Roblox Guard 1.0通过指令微调提升大型语言模型的安全性,采用输入输出调谐和可扩展的安全分类评估框架。

摘要:

大型语言模型(LLMs)通常在训练后阶段对安全性进行对齐;然而,它们仍可能生成不适当的内容,这可能对用户造成潜在风险。这一挑战凸显了需要在模型输入和输出上运行的坚固防护措施的必要性。在本工作中,我们介绍了Roblox Guard 1.0,一种最先进的指令微调LLM,旨在通过全面的输入输出调谐来增强LLM系统的安全性,使用LLM管道来增强调谐能力。基于Llama-3.1-8B-Instruct框架,我们的模型经过指令微调,能够泛化到之前未见过的安全分类,并在跨领域安全基准测试中表现出色。指令微调过程使用合成和开源安全数据集的混合,辅以链式推理(CoT)理由和输入反转来增强上下文理解和决策能力。为了支持系统的评估,我们还发布了RobloxGuard-Eval,一个新的基准测试,包含可扩展的安全分类,以评估LLM防护栏和调谐框架的有效性。

英文摘要:

Large Language Models (LLMs) are typically aligned for safety during the post-training phase; however, they may still generate inappropriate outputs that could potentially pose risks to users. This challenge underscores the need for robust safeguards that operate across both model inputs and outputs. In this work, we introduce Roblox Guard 1.0, a state-of-the-art instruction fine-tuned LLM designed to enhance the safety of LLM systems through comprehensive input-output moderation, using a pipeline of LLMs to enhance moderation capability. Built on the Llama-3.1-8B-Instruct backbone, our model is instruction fine-tuned to generalize across previously unseen safety taxonomies and demonstrates strong performance on out-of-domain safety benchmarks. The instruction fine-tuning process uses a mix of synthetic and open-source safety datasets, augmented with chain-of-thought (CoT) rationales and input inversion to enhance contextual understanding and decision making. To support systematic evaluation, we also release RobloxGuard-Eval, a new benchmark featuring an extensible safety taxonomy to assess the effectiveness of LLM guardrails and moderation frameworks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑