A key concern with the concept of "alignment" is the implicit question of "alignment to what?". AI systems are increasingly used across the world, yet safety alignment is often focused on homogeneous monolingual settings. Additionally, preference training and safety measures often overfit to harms common in Western-centric datasets. Here, we explore the viability of different alignment approaches when balancing dual objectives: addressing and optimizing for a non-homogeneous set of languages and cultural preferences while minimizing both global and local harms. We collect the first set of human annotated red-teaming prompts in different languages distinguishing between global and local harm, which serve as a laboratory for understanding the reliability of alignment techniques when faced with preference distributions that are non-stationary across geographies and languages. While this setting is seldom covered by the literature to date, which primarily centers on English harm mitigation, it captures real-world interactions with AI systems around the world. We establish a new precedent for state-of-the-art alignment techniques across 6 languages with minimal degradation in general performance. Our work provides important insights into cross-lingual transfer and novel optimization approaches to safeguard AI systems designed to serve global populations.
CuMA: Aligning LLMs with Sparse Cultural Values via Demographic-Aware Mixture of Adapters
Southeast University（东南大学）；ByteDance Inc.（字节跳动公司）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China（新一代人工智能技术及其交叉应用重点实验室（东南大学），中华人民共和国教育部，中国）
Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；University of Washington（华盛顿大学）
We focus on creating agents that act in alignment with socially beneficial norms and values in interactive narratives or text-based games -- environments wherein an agent perceives and interacts with a world through natural language. Such interactive agents are often trained via reinforcement learning to optimize task performance, even when such rewards may lead to agent behaviors that violate societal norms -- causing harm either to the agent itself or other entities in the environment. Social value alignment refers to creating agents whose behaviors conform to expected moral and social norms…
DiverValue-Bench: A Benchmark and Fine-Tuning Framework for Aligning Large Language Models with Diverse Human Values
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Renmin University of China（中国人民大学）；Beijing Institute of AI Safety and Governance（北京人工智能安全与治理研究院）；University of Chinese Academy of Sciences（中国科学院大学）
ConsisGuard: Aligning Safety Deliberation with Policy Enforcement in LLM Guardrails
Alibaba Group（阿里巴巴集团）；Zhejiang University（浙江大学）；Huzhou Normal University（湖州师范学院）；Zhejiang Key Laboratory of Intelligent Education Technology and Application（浙江省智能教育技术与应用重点实验室）
ReEvalMed: Rethinking Medical Report Evaluation by Aligning Metrics with Real-World Clinical Judgment
Technical University of Munich（慕尼黑技术大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；University of Massachusetts Boston（马萨诸塞大学波士顿分校）；University of Strasbourg（斯特拉斯堡大学）
TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots
Department of Computer Science, Stanford University, Stanford, CA, USA（计算机科学系，斯坦福大学，斯坦福，加州，美国）；Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA（精神病学与行为科学系，斯坦福大学，斯坦福，加州，美国）；Department of Biomedical Data Science, Stanford University, Stanford, CA, USA（生物医学数据科学系，斯坦福大学，斯坦福，加州，美国）；University of California, Berkeley, Berkeley, CA, USA（加州大学伯克利分校，伯克利，加州，美国）；The University of Hong Kong, Hong Kong（香港大学，香港）
Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Models
Key Lab of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (CAS)（智能信息处理重点实验室，计算技术研究所，中国科学院）；State Key Lab of AI Safety（人工智能安全国家重点实验室）；University of Chinese Academy of Sciences, CAS（中国科学院大学）；Zhejiang University（浙江大学）
SDGO: Self-Discrimination-Guided Optimization for Consistent Safety in Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术全国重点实验室）；Meituan Inc.（美团公司）；Dalian University of Technology（大连理工大学）
