Generative AI (GenAI) models have demonstrated remarkable capabilities in a wide variety of medical tasks. However, as these models are trained using generalist datasets with very limited human oversight, they can learn uses of medical products that have not been adequately evaluated for safety and efficacy, nor approved by regulatory agencies. Given the scale at which GenAI may reach users, unvetted recommendations pose a public health risk. In this work, we propose an approach to identify potentially harmful product recommendations, and demonstrate it using a recent multimodal large language model.
A Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Space
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);King Abdullah University of Science and Technology(卡塔尔国王大学科学与技术研究院);University of Oxford(牛津大学)
RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Virginia Tech(弗吉尼亚理工大学);Salesforce Research(Salesforce研究院);University of California Berkeley(加州大学伯克利分校);University of Chicago(芝加哥大学)
Intrinsic Guardrails: How Semantic Geometry of Personality Interacts with Emergent Misalignment in LLMs
International Institute of Information Technology, Hyderabad(国际信息研究所,海得拉巴);Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi(Mohamed bin Zayed人工智能大学,阿布扎比)
It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics
Université de Montréal, MILA(蒙特利尔大学,MILA);Carnegie Mellon University(卡内基梅隆大学);MIT, Center for Research and Teaching in Economics(麻省理工学院,经济研究与教学中心);Cornell University, University of Regina(康奈尔大学, Regina大学);Cornell University, MIT(康奈尔大学,麻省理工学院)
LongGuard: Mechanistic Analysis and Training-Free Mitigation of Long-Context Failure in Safety Guardrails
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
SafeHarbor: Defining Precise Decision Boundaries via Hierarchical Memory-Augmented Guardrail for LLM Agent Safety
School of Cyber Science and Technology, Beihang University, Beijing, China(北京航空航天大学网络安全学院);Institute of Artificial Intelligence, Beihang University, Beijing, China(北京航空航天大学人工智能研究院);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学);AI Security Lab, Beijing, China(360人工智能安全实验室)
Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation
Boise State University(博伊州立大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Pittsburgh(匹兹堡大学);New York University(纽约大学);Florida International University(佛罗里达国际大学)
CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment
Tsinghua University(清华大学);Beijing Normal University(北京师范大学);South China University of Technology(华南理工大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Shenzhen ShenNong Information Technology Co., Ltd.(深圳神农信息技术有限公司)
HoloAegis: Frozen Representation, Topological Inference --- Minimally Parametric Safety Manifolds and Their Capability Boundaries for LLM Guardrails
Hong Kong Baptist University(香港浸会大学);Guangdong Polytechnic Normal University(广东技术师范大学);Guangdong Institute of Digital Industry(广东数字产业研究院);The Hong Kong Polytechnic University(香港理工大学);The Education University of Hong Kong(香港教育大学);Southern University of Science and Technology(南方科技大学)
EVA: Editing for Versatile Alignment against Jailbreaks
ShanghaiTech University(上海科技大学);Sun Yat-sen University(中山大学);State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室);Tsinghua University(清华大学)
From Refusal to Recovery: A Control-Theoretic Approach to Generative AI Guardrails
Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所);Department of Electrical and Computer Engineering, Princeton University(普林斯顿大学电气与计算机工程系);Equal Advising(平等指导)
DDOR: Delta Debugging for Explainable Overrefusal Testing and Repair
Southeast University(东南大学);Singapore Management University(新加坡管理学院);Zhejiang University(浙江大学);Huzhou Institute of Industrial Control Technology(湖州工业控制技术研究所)
The Safety Reminder: A Soft Prompt to Reactivate Delayed Safety Awareness in Vision-Language Models
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);School of Computer Science and Technology, University of Science and Technology of China(中国科学技术大学计算机科学与技术学院);School of Computing and Information Systems, Singapore Management University(新加坡管理学院计算与信息系统学院)
One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intent
Georgia Institute of Technology(佐治亚理工学院);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);UCSD(加州大学圣地亚哥分校);National Taiwan University(台湾大学);IBM Research(IBM研究院);Virtue AI