As Large Language Models (LLMs) and generative AI become increasingly widespread, concerns about content safety have grown in parallel. Currently, there is a clear lack of high-quality, human-annotated datasets that address the full spectrum of LLM-related safety risks and are usable for commercial applications. To bridge this gap, we propose a comprehensive and adaptable taxonomy for categorizing safety risks, structured into 12 top-level hazard categories with an extension to 9 fine-grained subcategories. This taxonomy is designed to meet the diverse requirements of downstream users, offering more granular and flexible tools for managing various risk types. Using a hybrid data generation pipeline that combines human annotations with a multi-LLM "jury" system to assess the safety of responses, we obtain Aegis 2.0, a carefully curated collection of 34,248 samples of human-LLM interactions, annotated according to our proposed taxonomy. To validate its effectiveness, we demonstrate that several lightweight models, trained using parameter-efficient techniques on Aegis 2.0, achieve performance competitive with leading safety models fully fine-tuned on much larger, non-commercial datasets. In addition, we introduce a novel training blend that combines safety with topic following data.This approach enhances the adaptability of guard models, enabling them to generalize to new risk categories defined during inference. We plan to open-source Aegis 2.0 data and models to the research community to aid in the safety guardrailing of LLMs.
ConsisGuard: Aligning Safety Deliberation with Policy Enforcement in LLM Guardrails
Alibaba Group（阿里巴巴集团）；Zhejiang University（浙江大学）；Huzhou Normal University（湖州师范学院）；Zhejiang Key Laboratory of Intelligent Education Technology and Application（浙江省智能教育技术与应用重点实验室）
ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback
National Engineering Research Center for Software Engineering, Peking University（软件工程国家工程研究中心，北京大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
The Need for Guardrails with Large Language Models in Medical Safety-Critical Settings: An Artificial Intelligence Application in the Pharmacovigilance Ecosystem
Harvard-MIT Department of Health Sciences and Technology（哈佛-麻省理工卫生科学与技术部）；GlaxoSmithKline（葛兰素史克公司）；London School of Hygiene and Tropical Medicine（伦敦卫生与热带医学院）；Harvard T.H. Chan School of Public Health（哈佛陈曾熙公共卫生学院）
Retrieval-Augmented Guardrails for AI-Drafted Patient-Portal Messages: Error Taxonomy Construction and Large-Scale Evaluation
Stanford Center for Biomedical Informatics Research（斯坦福大学生物医学信息学研究中心）；Department of Medicine, Stanford University（斯坦福大学医学院）；Department of Biomedical Data Science, Stanford University（斯坦福大学生物医学数据科学系）；Stanford Clinical Excellence Research Center, Stanford University（斯坦福大学临床卓越研究中心）
CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment
Tsinghua University（清华大学）；Beijing Normal University（北京师范大学）；South China University of Technology（华南理工大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Shenzhen ShenNong Information Technology Co., Ltd.（深圳神农信息技术有限公司）
Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings
Graduate School of Informatics, Osaka Metropolitan University（大阪公立大学情报学研究科）；BRAC University（BRAC大学）；Bangladesh University of Engineering and Technology (BUET)（孟加拉工程技术大学）；New Mexico State University（新墨西哥州立大学）
HoloAegis: Frozen Representation, Topological Inference --- Minimally Parametric Safety Manifolds and Their Capability Boundaries for LLM Guardrails
Hong Kong Baptist University（香港浸会大学）；Guangdong Polytechnic Normal University（广东技术师范大学）；Guangdong Institute of Digital Industry（广东数字产业研究院）；The Hong Kong Polytechnic University（香港理工大学）；The Education University of Hong Kong（香港教育大学）；Southern University of Science and Technology（南方科技大学）
