ExpGuard: 专门领域中的大语言模型内容审核
ExpGuard: LLM Content Moderation in Specialized Domains
AI总结：
ExpGuard通过专门领域的大语言模型内容审核模型，有效提升对金融、医疗和法律领域有害内容的防御能力。
摘要：
随着大型语言模型（LLMs）在现实世界中的应用日益增长，建立稳健的安全护栏以调节其输入和输出已成为确保遵守安全政策的关键。当前的安全护栏模型主要针对一般的人与LLM交互，使LLMs在特定领域中容易受到有害和对抗性内容的影响，尤其是那些充满技术术语和专业概念的领域。为了解决这一限制，我们引入了ExpGuard，一个稳健且专门的安全护栏模型，旨在保护LLMs免受有害提示和响应的侵害，涵盖金融、医疗和法律领域。此外，我们还提出了ExpGuardMix，一个精心编纂的数据集，包含58,928个标记的提示，配以相应的拒绝和合规响应，来自这些特定领域。该数据集分为两个子集：ExpGuardTrain，用于模型训练，以及ExpGuardTest，一个由领域专家注释的高质量测试集，用于评估模型对技术性和领域特定内容的鲁棒性。在ExpGuardTest和八个已建立的公共基准测试上进行的全面评估显示，ExpGuard在各方面都表现出竞争力，同时在对抗性攻击方面表现出卓越的鲁棒性，超过WildGuard等最先进的模型，在提示分类上高出8.9%，在响应分类上高出15.3%。为了鼓励进一步的研究和开发，我们开源了我们的代码、数据和模型，使能够适应其他领域，并支持创建更加稳健的安全护栏模型。
英文摘要：
With the growing deployment of large language models (LLMs) in real-world applications, establishing robust safety guardrails to moderate their inputs and outputs has become essential to ensure adherence to safety policies. Current guardrail models predominantly address general human-LLM interactions, rendering LLMs vulnerable to harmful and adversarial content within domain-specific contexts, particularly those rich in technical jargon and specialized concepts. To address this limitation, we introduce ExpGuard, a robust and specialized guardrail model designed to protect against harmful prompts and responses across financial, medical, and legal domains. In addition, we present ExpGuardMix, a meticulously curated dataset comprising 58,928 labeled prompts paired with corresponding refusal and compliant responses, from these specific sectors. This dataset is divided into two subsets: ExpGuardTrain, for model training, and ExpGuardTest, a high-quality test set annotated by domain experts to evaluate model robustness against technical and domain-specific content. Comprehensive evaluations conducted on ExpGuardTest and eight established public benchmarks reveal that ExpGuard delivers competitive performance across the board while demonstrating exceptional resilience to domain-specific adversarial attacks, surpassing state-of-the-art models such as WildGuard by up to 8.9% in prompt classification and 15.3% in response classification. To encourage further research and development, we open-source our code, data, and model, enabling adaptation to additional domains and supporting the creation of increasingly robust guardrail models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
