大模型对齐与安全

Episodic Learning for Safe Bipedal Locomotion with Control Barrier Functions and Projection-to-State Safety

Noel Csomay-Shanklin, Ryan K. Cosner, Min Dai, Andrew J. Taylor, Aaron D. Ames

专题命中 安全训练 ：safety(title,abstract)

Comments 13 pages, 4 figures, to appear at the Conference on Learning for Dynamics and Control 2021

Trust and Safety

S. K. Devitt, R. Horne, Z. Assaad, E. Broad, H. Kurniawati, B. Cardier, A. Scott, S. Lazar, M. Gould, C. Adamson, C. Karl, F. Schrever, S. Keay, K. Tranter, E. Shellshear, D. Hunter, M. Brady, T. Putland

专题命中 安全训练 ：safety(title,abstract)

Comments Approved for publication as a chapter in 'Robotics Roadmap for Australia V2' by Robotics Australia Network [forthcoming]. 31 pages, 10 Figures, 1 table

Rules of the Road: Safety and Liveness Guarantees for Autonomous Vehicles

Karena X. Cai, Tung Phan-Minh, Soon-Jo Chung, Richard M. Murray

专题命中 安全训练 ：safety(title,abstract)

Comments 20 pages, 10 figures

Reachability-based Trajectory Safeguard (RTS): A Safe and Fast Reinforcement Learning Safety Layer for Continuous Control

Yifei Simon Shao, Chao Chen, Shreyas Kousik, Ram Vasudevan

专题命中 安全训练 ：safety(title,abstract)

Adversarial Resilience for Sampled-Data Systems under High-Relative-Degree Safety Constraints

James Usevitch, Dimitra Panagou

专题命中 安全训练 ：safety(title,abstract)

An agent-based model of multi-dimensional opinion dynamics and opinion alignment

Simon Schweighofer, David Garcia, Frank Schweitzer

专题命中 安全训练 ：alignment(title,abstract)

Comments 34 pp

Energy-Efficient Control Adaptation with Safety Guarantees for Learning-Enabled Cyber-Physical Systems

Yixuan Wang, Chao Huang, Qi Zhu

专题命中 安全训练 ：safety(title,abstract)

Comments Accepted by 39th International Conference On Computer Aided Design(ICCAD 2020)

Improving Train Track Safety using Drones, Computer Vision and Machine Learning

Kirthi Kumar, Anuraag Kaashyap

专题命中 安全训练 ：safety(title,abstract)

Comments 27 pages and 19 figures

Tackling Occlusions & Limited Sensor Range with Set-based Safety Verification

Piotr Franciszek Orzechowski, Annika Meyer, Martin Lauer

专题命中 安全训练 ：safety(title,abstract)

Journal ref 2018 21st International Conference on Intelligent Transportation Systems (ITSC)

Learning Sequential Channel Selection for Interference Alignment using Reconfigurable Antennas

Nikhil Gulati, Rohit Bahl, Kapil R. Dandekar

专题命中 安全训练 ：alignment(title,abstract)

Reasoning About Safety-Critical Information Flow Between Pilot and Computer

Seth Ahrenbach

专题命中 安全训练 ：safety(title,abstract)

Comments Published in the Proceedings from the 9th International Symposium of NASA Formal Methods, 2017

Journal ref In: Barrett C., Davies M., Kahsai T. (eds) NASA Formal Methods. NFM 2017. Lecture Notes in Computer Science, vol 10227. Springer, Cham

Safety Barrier Certificates for Heterogeneous Multi-Robot Systems

Li Wang, Aaron Ames, Magnus Egerstedt

专题命中 安全训练 ：safety(title,abstract)

Comments 8 pages version of 2016ACC conference paper, experimental results added

Journal ref American Control Conference (ACC), pages 5213-5218, July 2016

Understanding and Mitigating Over-refusal for Large Language Models via Representation Intervention

通过安全表示理解并缓解大语言模型的过度拒绝

Junbo Zhang, Ran Chen, Qianli Zhou, Xinyang Deng, Wen Jiang

机构 * Northwestern Polytechnical University（西北工业大学） 纠错

专题命中 安全训练 ：jailbreak(abstract,abstract_cn)；safety(abstract)；分类 cs.CL

AI总结 本文提出MOSR方法，通过干预大语言模型的安全表示来缓解过度拒绝问题，同时保持安全性。

Comments Added experiments

FanarGuard: A Culturally-Aware Moderation Filter for Arabic Language Models

FanarGuard: 一种文化感知的阿拉伯语言模型审查过滤器

Masoomali Fatehkia, Enes Altinisik, Husrev Taha Sencar

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；harmlessness(abstract)；分类 cs.CL

AI总结 FanarGuard是一种双语审查过滤器，通过评估阿拉伯语和英语中的安全性和文化对齐性，提升内容审查的准确性与文化敏感性。

LLMs Encode Harmfulness and Refusal Separately

大语言模型分别编码有害性和拒绝性

Jiachen Zhao, Jing Huang, Zhengxuan Wu, David Bau, Weiyan Shi

机构 * Northeastern University（东北大学） ； Stanford University（斯坦福大学） 纠错

专题命中 安全训练 ：safety(abstract)；jailbreak(abstract)；AI safety(abstract)；分类 cs.CL

AI总结 研究大语言模型对有害性的理解，发现其有害性和拒绝性在模型内分别编码，有害性方向与拒绝方向不同，据此揭示越狱方法原理及对抗微调影响，提出潜在危害表示的安全应用。

Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Use

学习何时行动或拒绝：为安全的多步骤工具使用守护代理推理模型

Aradhye Agarwal, Gurdit Siyan, Yash Pandya, Joykirat Singh, Akshay Nambi, Ahmed Awadallah

机构 * Microsoft Research（微软研究院） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；prompt injection(abstract)；分类 cs.CL

AI总结 提出MOSAIC框架，通过显式安全推理和基于偏好的强化学习，使代理模型在工具使用中安全决策，减少有害行为并保持良性性能。

Comments Accepted to the 43rd International Conference on Machine Learning (ICML 2026)

Adaptive Steering and Remasking for Safe Generation in Diffusion Language Models

自适应转向与重掩码用于扩散语言模型中的安全生成

Yejin Lee, Yo-Sub Han

机构 * Department of Computer Science（计算机科学系） ； Yonsei University（延世大学） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL

AI总结 本文提出基于去噪过程分步干预的安全防御框架，通过对比安全方向（SGD）检测有害生成并重掩码以提高安全性，实验显示有效降低逃逸成功率至0.64%。

Comments 17 pages, 3 figures

Robust Policy Optimization to Prevent Catastrophic Forgetting

鲁棒策略优化以防止灾难性遗忘

Mahdi Sabbaghi, George Pappas, Adel Javanmard, Hamed Hassani

机构 * University of Pennsylvania（宾夕法尼亚大学） ； University of Southern California（南加州大学） 纠错

专题命中 安全训练 ：RLHF(abstract,abstract_cn)；safety(abstract)；分类 cs.LG

AI总结 本文提出FRPO框架，通过优化当前策略及下游适应可达的KL限制邻域内策略，提升鲁棒性，减少安全性能退化，同时保持下游任务表现。

From Shallow to Deep: Pinning Semantic Intent via Causal GRPO

从浅层到深层：通过因果GRPO固定语义意图

Shuyi Zhou, Zeen Song, Wenwen Qiang, Jiyan Sun, Yao Zhou, Yinlong Liu, Wei Ma

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.LG

AI总结 通过因果GRPO框架，解决大型语言模型对对抗性前缀攻击的脆弱性问题，实现意图固定以提升安全性和鲁棒性。

Being Kind Isn't Always Being Safe: Diagnosing Affective Hallucination in LLMs

善良并不总是安全：诊断大语言模型中的情感幻觉

Sewon Kim, Jiwon Kim, Seungwoo Shin, Hyejin Chung, Daeun Moon, Yejin Kwon, Hyunsoo Yoon

机构 * Department of Industrial Engineering, Yonsei University（工业工程系，延世大学） 纠错

专题命中 安全训练 ：alignment(abstract)；DPO(abstract)；safety(abstract)；分类 cs.CL

AI总结 本文提出AHaBench和AHaPairs用于诊断LLM中的情感幻觉问题，通过DPO优化减少虚假社会存在感，提升模型的心理安全性和事实可靠性。

Comments EACL 2026 Findings

Offline Safe Policy Optimization From Heterogeneous Feedback

基于异质反馈的离线安全策略优化

Ze Gong, Pradeep Varakantham, Akshat Kumar

机构 * Shenzhen Institute of Advanced Technology (SIAT), CAS（深圳先进技术研究院（SIAT）、中国科学院） ； Singapore Management University（新加坡管理学院） 纠错

专题命中 安全训练 ：alignment(abstract)；RLHF(abstract)；safety(abstract)；分类 cs.AI

AI总结 本文提出PreSa方法，通过直接学习基于偏好和安全性的策略，解决离线安全强化学习中的性能下降问题。

Comments Accepted at AAMAS 2026 (Extended Abstract)

Synthetic Voices, Real Threats: Evaluating Large Text-to-Speech Models in Generating Harmful Audio

Guangke Chen, Yuhui Wang, Shouling Ji, Xiapu Luo, Ting Wang

机构 * Stony Brook University（石英布鲁克大学） ； Zhejiang University（浙江大学） ； The Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.AI

Agentic Moderation: Multi-Agent Design for Safer Vision-Language Models

Juan Ren, Mark Dras, Usman Naseem

机构 * School of Computing, Macquarie University, Australia（计算机学院，麦考瑞大学，澳大利亚） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.AI

RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards

Jingnan Zheng, Xiangtian Ji, Yijun Lu, Chenhang Cui, Weixiang Zhao, Gelei Deng, Zhenkai Liang, An Zhang, Tat-Seng Chua

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.AI

Journal ref 39th Conference on Neural Information Processing Systems (NeurIPS 2025). 39th Conference on Neural Information Processing Systems (NeurIPS 2025). 39th Conference on Neural Information Processing Systems (NeurIPS 2025)

Too Helpful, Too Harmless, Too Honest or Just Right?

Gautam Siddharth Kashyap, Mark Dras, Usman Naseem

机构 * School of Computing, Macquarie University（计算机学院，麦考瑞大学） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；harmlessness(abstract)；分类 cs.CL

Comments EMNLP'25 Main

Mind the Inconspicuous: Revealing the Hidden Weakness in Aligned LLMs' Refusal Boundaries

Jiahao Yu, Haozheng Luo, Jerry Yao-Chieh Hu, Wenbo Guo, Han Liu, Xinyu Xing

专题命中 安全训练 ：alignment(abstract)；RLHF(abstract)；jailbreak(abstract)；分类 cs.AI

Comments published at USENIX Security 25

Towards Safe and Honest AI Agents with Neural Self-Other Overlap

Marc Carauleanu, Michael Vaiana, Judd Rosenblatt, Cameron Berg, Diogo Schwerz de Lucena

专题命中 安全训练 ：safety(abstract)；trustworthy(abstract)；AI safety(abstract)；分类 cs.AI

Comments NeurIPS 2024 Safe Generative AI Workshop

Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents

Priyanshu Kumar, Elaine Lau, Saranya Vijayakumar, Tu Trinh, Scale Red Team, Elaine Chang, Vaughn Robinson, Sean Hendryx, Shuyan Zhou, Matt Fredrikson, Summer Yue, Zifan Wang

专题命中 安全训练 ：safety(abstract)；jailbreak(abstract)；red teaming(abstract)；分类 cs.LG

Can a large language model be a gaslighter?

Wei Li, Luyao Zhu, Yang Song, Ruixi Lin, Rui Mao, Yang You

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.CL、cs.AI、cs.CY

Comments 10/26 (Main Body/Total), 8 figures

GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher

Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, Zhaopeng Tu

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；red teaming(abstract)；分类 cs.CL

Comments Accepted by ICLR 2024. 21 pages, 3 figures, 13 tables