Trust The Typical通过将安全性视为分布外检测问题,在无需有害示例训练的情况下,实现了在多个安全基准测试中的高性能表现,显著降低误报率。
摘要:
当前LLM安全方法本质上依赖于通过防护措施识别和阻止已知威胁的脆弱猫和老鼠游戏。我们主张一种新的方法:鲁棒安全性并非来自列举有害内容,而是来自深刻理解安全内容。我们引入了Trust The Typical(T3)框架,该框架通过将安全性视为分布外(OOD)检测问题来实现这一原则。T3在语义空间中学习可接受提示的分布,并将任何显著偏差标记为潜在威胁。与以往方法不同,它不需要在有害示例上进行训练,却在18个基准测试中实现了最先进的性能,涵盖毒性、仇恨言论、劫持、多语言危害和过度拒绝。T3将误报率降低了高达40倍。一个仅在安全英文文本上训练的模型可以有效地转移到不同领域和14种以上语言,无需重新训练。最后,我们通过将GPU优化版本集成到vLLM中,证明了生产就绪性,使在生成过程中能够持续进行防护,即使在大规模工作负载下密集评估时,其开销也低于6%。
英文摘要:
Current approaches to LLM safety fundamentally rely on a brittle cat-and-mouse game of identifying and blocking known threats via guardrails. We argue for a fresh approach: robust safety comes not from enumerating what is harmful, but from deeply understanding what is safe. We introduce Trust The Typical (T3), a framework that operationalizes this principle by treating safety as an out-of-distribution (OOD) detection problem. T3 learns the distribution of acceptable prompts in a semantic space and flags any significant deviation as a potential threat. Unlike prior methods, it requires no training on harmful examples, yet achieves state-of-the-art performance across 18 benchmarks spanning toxicity, hate speech, jailbreaking, multilingual harms, and over-refusal, reducing false positive rates by up to 40x relative to specialized safety models. A single model trained only on safe English text transfers effectively to diverse domains and over 14 languages without retraining. Finally, we demonstrate production readiness by integrating a GPU-optimized version into vLLM, enabling continuous guardrailing during token generation with less than 6% overhead even under dense evaluation intervals on large-scale workloads.
VMDT: Decoding the Trustworthiness of Video Foundation Models
University of California, Berkeley(加州大学伯克利分校);University of California, Santa Cruz(加州大学圣克ruz分校);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Chicago(芝加哥大学);Amazon(亚马逊);Information Commissioner’s Office(信息专员办公室)
DarkPatterns-LLM: A Multi-Layer Benchmark for Detecting Manipulative and Harmful AI Behavior
Rensselaer Polytechnic Institute(拉特格斯理工学院);National University of Sciences and Technology(国立科学与技术大学);School of Electrical Engineering & Computer Science(电子与计算机科学学院)
A closer look at how large language models trust humans: patterns and biases
The Hebrew University Business School(希伯来大学商学院);The Federmann Center for the study of Rationality(费德曼理性研究中心);The Hebrew University of Jerusalem(特拉维夫希伯来大学)
Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);RealAI;Tsinghua University(清华大学)
Mitsubishi Electric Corporation(三菱电机株式会社);Pennsylvania State University(宾夕法尼亚州立大学);The Ohio State University(俄亥俄州立大学);Mitsubishi Electric Research Laboratories(三菱电机研究所)
Counterfactual Neighborhood Risk Fields: Evidence-Grounded Selective Safety for Vision-Language Models
Tongji University(同济大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of Electronic Science and Technology of China(电子科技大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environments
Department of Computer Science & Technology, Institute for AI, BNRist Center, THBI Lab, Tsinghua-Bosch Joint Center for ML(计算机科学与技术系、人工智能研究院、BNRist中心、THBI实验室、清华-博世联合机器学习中心);Shanghai Key Lab. of Multidimensional Info. Processing(上海多维信息处理重点实验室)
MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets
Johns Hopkins University(约翰斯·霍普金斯大学);Indraprastha Institute of Information Technology - Delhi(德里英迪拉普拉斯信息技术学院);Wipro AI Labs(威普罗人工智能实验室);Sofia University(索非亚大学);Qatar Computing Research Institute, HBKU(哈马德·本·哈利法大学卡塔尔计算研究所)
网络模因已经成为传播政治、心理和社会文化思想的有力手段。虽然模因通常是幽默的,但最近几天,用于拖钓、网络欺凌和虐待的有害模因不断升级。检测这样的模因是具有挑战性的,因为它们可能是高度讽刺和神秘的。此外,虽然以前的工作集中在模因的具体方面,如仇恨言论和宣传,但很少有关于一般伤害的工作。在这里,我们的目标是弥合这一差距。我们专注于两个任务:(i)检测有害的模因,(ii)识别它们所针对的社会实体。我们进一步扩展了最近发布的HarMeme数据集,该数据集涵盖了COVID-19,增加了额外的模因和一个新的主题:美国政治。为了解决这些任务,我们提出了MOMENTA(MultiModal framework for detecting harmful MemEs aNd Their tTargets),这是一种新型的多模态深度神经网络,它使用全局和局部视角来检测有害模因。MOMENTA系统地分析了输入模因的本地和全球视角(两种模式),并将其与背景语境联系起来。MOMENTA是可解释和可推广的,我们的实验表明,它优于几个强大的竞争方法。
A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学);NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences(神经语言处理与机器智能中心,中国科学院自动化研究所)
Safety Instincts: LLMs Learn to Trust Their Internal Compass for Self-Defense
Beijing Institute of AI Safety and Governance(北京人工智能安全与治理研究院);Beijing Key Laboratory of Safe AI and Superalignment(北京安全人工智能与超对齐重点实验室);BrainCog Lab, Institute of Automation, Chinese Academy of Sciences(脑认知实验室,中国科学院自动化研究所)
Preventing Safety Drift in Large Language Models via Coupled Weight and Activation Constraints
Hunan Normal University(湖南师范大学);The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences(认知与决策智能复杂系统重点实验室,中国科学院自动化研究所);National University of Defense Technology(国防科技大学)