Building a Domain-specific Guardrail Model in Production
Emergence AI;Stanford University(斯坦福大学)
Mohammad Niknazar, Paul V Haley, Latha Ramanan, Sang T. Truong, Yedendra Shrinivasan, Ayan Kumar Bhowmick, Prasenjit Dey, Ashish Jagmohan, Hema Maheshwari, Shom… 展开作者
Mohammad Niknazar, Paul V Haley, Latha Ramanan, Sang T. Truong, Yedendra Shrinivasan, Ayan Kumar Bhowmick, Prasenjit Dey, Ashish Jagmohan, Hema Maheshwari, Shom Ponoth, Robert Smith, Aditya Vempaty, Nick Haber, Sanmi Koyejo, Sharad Sundararajan
Generative AI holds the promise of enabling a range of sought-after capabilities and revolutionizing workflows in various consumer and enterprise verticals. However, putting a model in production involves much more than just generating an output. It involves ensuring the model is reliable, safe, performant and also adheres to the policy of operation in a particular domain. Guardrails as a necessity for models has evolved around the need to enforce appropriate behavior of models, especially when they are in production. In this paper, we use education as a use case, given its stringent requirements of the appropriateness of content in the domain, to demonstrate how a guardrail model can be trained and deployed in production. Specifically, we describe our experience in building a production-grade guardrail model for a K-12 educational platform. We begin by formulating the requirements for deployment to this sensitive domain. We then describe the training and benchmarking of our domain-specific guardrail model, which outperforms competing open- and closed- instruction-tuned models of similar and larger size, on proprietary education-related benchmarks and public benchmarks related to general aspects of safety. Finally, we detail the choices we made on architecture and the optimizations for deploying this service in production; these range across the stack from the hardware infrastructure to the serving layer to language model inference optimizations. We hope this paper will be instructive to other practitioners looking to create production-grade domain-specific services based on generative AI and large language models.
The Need for Guardrails with Large Language Models in Medical Safety-Critical Settings: An Artificial Intelligence Application in the Pharmacovigilance Ecosystem
Harvard-MIT Department of Health Sciences and Technology(哈佛-麻省理工卫生科学与技术部);GlaxoSmithKline(葛兰素史克公司);London School of Hygiene and Tropical Medicine(伦敦卫生与热带医学院);Harvard T.H. Chan School of Public Health(哈佛陈曾熙公共卫生学院)
Intrinsic Guardrails: How Semantic Geometry of Personality Interacts with Emergent Misalignment in LLMs
International Institute of Information Technology, Hyderabad(国际信息研究所,海得拉巴);Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi(Mohamed bin Zayed人工智能大学,阿布扎比)
LongGuard: Mechanistic Analysis and Training-Free Mitigation of Long-Context Failure in Safety Guardrails
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation
Boise State University(博伊州立大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Pittsburgh(匹兹堡大学);New York University(纽约大学);Florida International University(佛罗里达国际大学)
CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment
Tsinghua University(清华大学);Beijing Normal University(北京师范大学);South China University of Technology(华南理工大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Shenzhen ShenNong Information Technology Co., Ltd.(深圳神农信息技术有限公司)