大模型对齐与安全

Anchoring Refusal Direction: Mitigating Safety Risks in Tuning via Projection Constraint

Yanrui Du, Fenglei Fan, Sendong Zhao, Jiawei Cao, Qika Lin, Kai He, Ting Liu, Bing Qin, Mengling Feng

机构 * SCIR Lab, Harbin Institute of Technology, China（哈尔滨工业大学SCIR实验室） ； City University of Hong Kong（香港城市大学） ； National University of Singapore（新加坡国立大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL

Don't Trade Off Safety: Diffusion Regularization for Constrained Offline RL

Junyu Guo, Zhi Zheng, Donghao Ying, Ming Jin, Shangding Gu, Costas Spanos, Javad Lavaei

机构 * University of California Berkeley（加州大学伯克利分校） ； Virginia Tech（弗吉尼亚理工大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

SAFE--MA--RRT: Multi-Agent Motion Planning with Data-Driven Safety Certificates

Babak Esmaeili, Hamidreza Modares

机构 * Department of Mechanical Engineering, Michigan State University（机械工程系，密歇根州立大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments Submitted to IEEE Transactions on Automation Science and Engineering

Safe and Efficient Social Navigation through Explainable Safety Regions Based on Topological Features

Victor Toscano-Duran, Sara Narteni, Alberto Carlevaro, Jérôme Guzzi Rocio Gonzalez-Diaz, Maurizio Mongelli

机构 * Department of Applied Mathematics I, University of Seville（应用数学系，塞维利亚大学） ； CNR-IEIIT Genoa, Italy（意大利热那亚CNR-IEIIT） ； SUPSI, IDSIA Lugano, Switzerland（瑞士卢加诺SUPSI，IDSIA） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Democracy-in-Silico: Institutional Design as Alignment in AI-Governed Polities

Trisanth Srinivasan, Santosh Patapati

机构 * Cyrion Labs（Cyron实验室） 纠错

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.AI

Embedding Safety into RL: A New Take on Trust Region Methods

Nikola Milosevic, Johannes Müller, Nico Scherf

机构 * Max Planck Institute for Human Cognitive and Brain Sciences（马克斯·普朗克人类认知与脑科学研究所） ； Center for Scalable Data Analytics and Artificial Intelligence（可扩展数据分析与人工智能中心） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments Accepted at ICML 2025

Journal ref Proceedings of the 42nd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025

Safe Semantics, Unsafe Interpretations: Tackling Implicit Reasoning Safety in Large Vision-Language Models

Wei Cai, Jian Zhao, Yuchu Jiang, Tianle Zhang, Xuelong Li

机构 * Peking University（北京大学） ； Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究所（TeleAI），中国电信） ； Northwestern Polytechnical University（西北工业大学） ； Southeast University（东南大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages

Priyanshu Kumar, Devansh Jain, Akhila Yerukola, Liwei Jiang, Himanshu Beniwal, Thomas Hartvigsen, Maarten Sap

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL

Comments Accepted to COLM 2025 Main Conference

ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning

Yarden As, Bhavya Sukhija, Lenart Treven, Carmelo Sferrazza, Stelian Coros, Andreas Krause

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Qorgau: Evaluating LLM Safety in Kazakh-Russian Bilingual Contexts

Maiya Goloburda, Nurkhan Laiyk, Diana Turmakhan, Yuxia Wang, Mukhammed Togmanov, Jonibek Mansurov, Askhat Sametov, Nurdaulet Mukhituly, Minghan Wang, Daniil Orel, Zain Muhammad Mujahid, Fajri Koto, Timothy Baldwin, Preslav Nakov

机构 * Mohamed bin Zayed University of Artificial Intelligence（莫扎伊德大学人工智能学院） ； Monash University（莫纳什大学） ； University of Copenhagen（哥本哈根大学） ； The University of Melbourne（墨尔本大学） ； LibrAI（LibrAI公司） ； Institute of Foundation Models（基础模型研究所） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL

Why Do Some Language Models Fake Alignment While Others Don't?

Abhay Sheshadri, John Hughes, Julian Michael, Alex Mallen, Arun Jose, Janus, Fabien Roger

机构 * Anthropic ； Scale AI ； Redwood Research 纠错

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.LG

Generative Grasp Detection and Estimation with Concept Learning-based Safety Criteria

Al-Harith Farhad, Khalil Abuibaid, Christiane Plociennik, Achim Wagner, Martin Ruskowski

机构 * Innovative Factory Solutions（创新工厂解决方案） ； German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心） ； Chair of Machine Tools and Control Systems (WSKL)（机械工具与控制系统教授职位） ； Rheinland-Pfälzische Technische Universität (RPTU)（莱茵-瓦尔德技术大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments RAAD 2025: 34th International Conference on Robotics in Alpe-Adria-Danube Region

Optimal Transport-Guided Safety in Temporal Difference Reinforcement Learning

Zahra Shahrooei, Ali Baheri

机构 * Department of Mechanical Engineering, Rochester Institute of Technology（机械工程系，罗切斯特理工学院） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

SafeLawBench: Towards Safe Alignment of Large Language Models

Chuxue Cao, Han Zhu, Jiaming Ji, Qichao Sun, Zhenghao Zhu, Yinyu Wu, Juntao Dai, Yaodong Yang, Sirui Han, Yike Guo

机构 * Hong Kong University of Science and Technology（香港科技大学） ； Peking University（北京大学） 纠错

专题命中 安全训练 ：alignment(title)；safety(abstract)；分类 cs.CL

Comments Accepted to ACL2025 Findings

Legal Mathematical Reasoning with LLMs: Procedural Alignment through Two-Stage Reinforcement Learning

Kepu Zhang, Guofu Xie, Weijie Yu, Mingyue Xu, Xu Tang, Yaxin Li, Jun Xu

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学耿丽人工智能学院） ； University of International Business and Economics（国际经贸大学） 纠错

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.CL

SafeSteer: Interpretable Safety Steering with Refusal-Evasion in LLMs

Shaona Ghosh, Amrita Bhattacharjee, Yftah Ziser, Christopher Parisien

机构 * NVIDIA（英伟达） ； School of Computing and AI（计算与人工智能学院） ； Arizona State University（亚利桑那州立大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments arXiv admin note: text overlap with arXiv:2410.01174

Should LLM Safety Be More Than Refusing Harmful Instructions?

Utsav Maskey, Mark Dras, Usman Naseem

机构 * Macquarie University（麦考瑞大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL

Comments Preprint

Feasibility-Aware Pessimistic Estimation: Toward Long-Horizon Safety in Offline RL

Zhikun Tao

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments arXiv admin comment: This version removed due to inaccurate authorship

Guided by Guardrails: Control Barrier Functions as Safety Instructors for Robotic Learning

Maeva Guerrier, Karthik Soma, Hassan Fouad, Giovanni Beltrame

机构 * Department of Computer Engineering and Software Engineering, Polytechnique Montreal（计算机工程与软件工程系，蒙特利尔大学） ； Mila - Quebec AI Institute（魁北克人工智能研究所） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Towards Reliable and Interpretable Traffic Crash Pattern Prediction and Safety Interventions Using Customized Large Language Models

Yang Zhao, Pu Wang, Yibo Zhao, Hongru Du, Hao Frank Yang

机构 * Center for Systems Science and Engineering（系统科学与工程中心） ； Johns Hopkins University（约翰霍普金斯大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL

Comments Last revised 13 Feb 2025. Under review in Nature portfolio

Safety and optimality in learning-based control at low computational cost

Dominik Baumann, Krzysztof Kowalczyk, Cristian R. Rojas, Koen Tiels, Pawel Wachel

机构 * Aalto University（阿alto大学） ； Uppsala University（乌普萨拉大学） ； Wrocław University of Science and Technology（沃斯克拉大学科学与技术大学） ； KTH Royal Institute of Technology（皇家理工学院） ； Eindhoven University of Technology（埃因霍温理工大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments Accepted final version to appear in the IEEE Transactions on Automatic Control

Demonstrating ViSafe: Vision-enabled Safety for High-speed Detect and Avoid

Parv Kapoor, Ian Higgins, Nikhil Keetha, Jay Patrikar, Brady Moon, Zelin Ye, Yao He, Ivan Cisneros, Yaoyu Hu, Changliu Liu, Eunsuk Kang, Sebastian Scherer

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments 13 pages, RSS 2025 Demo track, https://theairlab.org/visafe/

Safe and Efficient CAV Lane Changing using Decentralised Safety Shields

Bharathkumar Hegde, Melanie Bouroche

机构 * School of Computer Science and Statistics, Trinity College Dublin（计算机科学与统计学学院，都柏林三一学院） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments Accepted in IEEE IV 2025

Planning Safety Trajectories with Dual-Phase, Physics-Informed, and Transportation Knowledge-Driven Large Language Models

Rui Gan, Pei Li, Keke Long, Bocheng An, Junwei You, Keshu Wu, Bin Ran

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Epistemic Closure and the Irreversibility of Misalignment: Modeling Systemic Barriers to Alignment Innovation

Andy Williams

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.AI

Graphormer-Guided Task Planning: Beyond Static Rules with LLM Safety Perception

Wanjing Huang, Tongjie Pan, Yalan Ye

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Safety Without Semantic Disruptions: Editing-free Safe Image Generation via Context-preserving Dual Latent Reconstruction

Jordan Vice, Naveed Akhtar, Mubarak Shah, Richard Hartley, Ajmal Mian

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments This research is supported by the NISDRG project #20100007, funded by the Australian Government

SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal

Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, Ruoxi Jia, Bo Li, Kai Li, Danqi Chen, Peter Henderson, Prateek Mittal

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments Paper accepted to ICLR 2025

Safeguarding AI Agents: Developing and Analyzing Safety Architectures

Ishaan Domkundwar, Mukunda N S, Ishaan Bhola, Riddhik Kochhar

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Models

Zihao Sheng, Zilin Huang, Yansong Qu, Yue Leng, Sruthi Bhavanam, Sikai Chen

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI