Sociotechnical Safety Evaluation of Generative AI Systems
Google DeepMind(谷歌DeepMind)
Laura Weidinger, Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Juan Mateos-Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach… 展开作者
Laura Weidinger, Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Juan Mateos-Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach, Iason Gabriel, Verena Rieser, William Isaac
Generative AI systems produce a range of risks. To ensure the safety of generative AI systems, these risks must be evaluated. In this paper, we make two main contributions toward establishing such evaluations. First, we propose a three-layered framework that takes a structured, sociotechnical approach to evaluating these risks. This framework encompasses capability evaluations, which are the main current approach to safety evaluation. It then reaches further by building on system safety principles, particularly the insight that context determines whether a given capability may cause harm. To account for relevant context, our framework adds human interaction and systemic impacts as additional layers of evaluation. Second, we survey the current state of safety evaluation of generative AI systems and create a repository of existing evaluations. Three salient evaluation gaps emerge from this analysis. We propose ways forward to closing these gaps, outlining practical steps as well as roles and responsibilities for different actors. Sociotechnical safety evaluation is a tractable approach to the robust and comprehensive safety evaluation of generative AI systems.
AI Development for the Public Interest: From Abstraction Traps to Sociotechnical Risks
Partnership on AI(人工智能伙伴关系);University of California, Berkeley(加州大学伯克利分校);Center for Human-Compatible AI(人类兼容人工智能中心);Harvard University(哈佛大学);Berkman Klein Center for Internet and Society(伯克曼·克莱因互联网与社会中心)
What is Safety? Corporate Discourse, Power, and the Politics of Generative AI Safety
College of Information Sciences and Technology, The Pennsylvania State University(信息科学与技术学院,宾夕法尼亚州立大学);Max Planck Institute for Security and Privacy(安全与隐私研究所)
Agentic Microphysics: A Manifesto for Generative AI Safety
DEXAI – Icaro Lab Sant’Anna School of Advanced Studies(DEXAI–伊卡洛实验室圣安娜高级研究学院);DEXAI – Icaro Lab Sapienza University of Rome(DEXAI–伊卡洛实验室罗马萨皮恩扎大学);DEXAI – Icaro Lab VU Amsterdam(DEXAI–伊卡洛实验室阿姆斯特丹伏里克大学);Lucidi DEXAI – Icaro Lab Sapienza University of Rome(Lucidi DEXAI–伊卡洛实验室罗马萨皮恩扎大学)
T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models
UCAS(中国科学院大学);Tsinghua University(清华大学);Institute for AI(人工智能研究院);Tsinghua-Bosch Joint ML Center(清华-博世联合机器学习中心);THBI Lab(THBI实验室);BNRist Center(BNRist中心);RealAI
MEGen: Generative Backdoor into Large Language Models via Model Editing
School of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学学院);Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University(上海交通大学智能交互与认知工程重点实验室);Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3(上海Web3可信数据流通与治理重点实验室);Cognitive AI Lab(认知人工智能实验室)
Privacy and Safety Experiences and Concerns of U.S. Women Using Generative AI for Seeking Sexual and Reproductive Health Information
King's College London London United Kingdom;VRAIN, Universitat Politècnica de València \& University of Cambridge Valencia Spain Cambridge United Kingdom;King's College London;VRAIN, Universitat Politècnica de València \& University of Cambridge
Toward an Evaluation Science for Generative AI Systems
Google DeepMind(谷歌DeepMind);University of California, Berkeley(加州大学伯克利分校);Microsoft Research(微软研究院);Hugging Face;Stanford University(斯坦福大学);Massachusetts Institute of Technology(麻省理工学院);Anthropic
本文针对生成式 AI 静态基准失效与审计难以规模化的问题,借鉴交通、航空航天和制药领域的安全评估经验,提出构建生成式 AI 评估科学的原则与路径。
Generative Grasp Detection and Estimation with Concept Learning-based Safety Criteria
Innovative Factory Solutions(创新工厂解决方案);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Chair of Machine Tools and Control Systems (WSKL)(机械工具与控制系统教授职位);Rheinland-Pfälzische Technische Universität (RPTU)(莱茵-瓦尔德技术大学)
Generative Auto-Bidding with Unified Modeling and Exploration
Key Laboratory of Aerospace Information Security;Trusted Computing, Ministry of Education, School of Cyber Science;Engineering, Wuhan University\ \& Tmall Group of Alibaba Wuhan China;Taobao \& Tmall Group of Alibaba Hangzhou China;Engineering, Wuhan University Wuhan China;Engineering, Wuhan University\ \& Tmall Group of Alibaba;Taobao \& Tmall Group of Alibaba;Engineering, Wuhan University
From Refusal to Recovery: A Control-Theoretic Approach to Generative AI Guardrails
Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所);Department of Electrical and Computer Engineering, Princeton University(普林斯顿大学电气与计算机工程系);Equal Advising(平等指导)
Initial Risk Probing and Feasibility Testing of Glow: a Generative AI-Powered Dialectical Behavior Therapy Skills Coach for Substance Use Recovery and HIV Prevention
Florida State University(佛罗里达州立大学);Nexcuria Labs, LLC.(Nexcuria实验室有限责任公司);University of North Texas(北得克萨斯大学);University of Washington(华盛顿大学)
Safety Without Semantic Disruptions: Editing-free Safe Image Generation via Context-preserving Dual Latent Reconstruction
University of Western Australia(西澳大学);University of Melbourne(墨尔本大学);University of Central Florida(中佛罗里达大学);Australian National University(澳大利亚国立大学);Google(谷歌)
Provably Safe Generative Sampling with Constricting Barrier Functions
Mechanical Engineering University of California, Riverside(机械工程 加州大学河滨分校);Electrical Engineering and Computer Science University of California, Irvine(电气工程与计算机科学 加州大学伊尔弗分校)