大模型对齐与安全

Best Arm Identification with Safety Constraints

Zhenlin Wang, Andrew Wagenmaker, Kevin Jamieson

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

On Assessing The Safety of Reinforcement Learning algorithms Using Formal Methods

Paulina Stevia Nouwou Mindom, Amin Nikanjam, Foutse Khomh, John Mullins

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Multi-Objective SPIBB: Seldonian Offline Policy Improvement with Safety Constraints in Finite MDPs

Harsh Satija, Philip S. Thomas, Joelle Pineau, Romain Laroche

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Improving Safety in Deep Reinforcement Learning using Unsupervised Action Planning

Hao-Lun Hsu, Qiuhua Huang, Sehoon Ha

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Enhancing Navigational Safety in Crowded Environments using Semantic-Deep-Reinforcement-Learning-based Navigation

Linh Kästner, Junhui Li, Zhengcheng Shen, Jens Lambrecht

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments 7 pages, 5 figures, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning

Danial Kamran, Tizian Engelgeh, Marvin Busch, Johannes Fischer, Christoph Stiller

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Weakly Supervised Reinforcement Learning for Autonomous Highway Driving via Virtual Safety Cages

Sampo Kuutti, Richard Bowden, Saber Fallah

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments Published in Sensors

Journal ref Sensors 2021, 21, 2032

Decentralized Multi-Agent Linear Bandits with Safety Constraints

Sanae Amani, Christos Thrampoulidis

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functions

Sarah Dean, Andrew J. Taylor, Ryan K. Cosner, Benjamin Recht, Aaron D. Ames

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Learning to be Safe: Deep RL with a Safety Critic

Krishnan Srinivasan, Benjamin Eysenbach, Sehoon Ha, Jie Tan, Chelsea Finn

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments In submission, 16 pages (including appendix)

L2B: Learning to Balance the Safety-Efficiency Trade-off in Interactive Crowd-aware Robot Navigation

Mai Nishimura, Ryo Yonetani

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments Accepted at IROS2020. Project site: https://denkiwakame.github.io/l2b/

Assessing Safety-Critical Systems from Operational Testing: A Study on Autonomous Vehicles

Xingyu Zhao, Kizito Salako, Lorenzo Strigini, Valentin Robu, David Flynn

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments Accepted by Information and Software Technology. arXiv admin note: substantial text overlap with arXiv:1908.06540

SMT-based Safety Verification of Parameterised Multi-Agent Systems

Paolo Felli, Alessandro Gianola, Marco Montali

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Intelligent Reputation System for Safety Messages in VANET

Ghassan Samara

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments 9 pages

Journal ref IAES International Journal of Artificial Intelligence (IJ-AI) Vol. 9, No. 3, September 2020, pp. 439~447, ISSN: 2252-8938, DOI: 10.11591/ijai.v9.i3.pp439-447

Learning for Dose Allocation in Adaptive Clinical Trials with Safety Constraints

Cong Shen, Zhiyang Wang, Sofia S. Villar, Mihaela van der Schaar

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments Accepted to the 37th International Conference on Machine Learning (ICML 2020)

Using Computer Vision to enhance Safety of Workforce in Manufacturing in a Post COVID World

Prateek Khandelwal, Anuj Khandelwal, Snigdha Agarwal, Deep Thomas, Naveen Xavier, Arun Raghuraman

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments 6 pages, 7 figure, 1 table

Learning for Safety-Critical Control with Control Barrier Functions

Andrew Taylor, Andrew Singletary, Yisong Yue, Aaron Ames

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments Extended version (12 Pages), Short version submitted to Learning for Dynamics & Control (L4DC) 2020 Conference

Practical Solutions for Machine Learning Safety in Autonomous Vehicles

Sina Mohseni, Mandar Pitale, Vasu Singh, Zhangyang Wang

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments Accepted at Safe AI workshop at AAAI 2020

Improving Safety in Reinforcement Learning Using Model-Based Architectures and Human Intervention

Bharat Prakash, Mohit Khatwani, Nicholas Waytowich, Tinoosh Mohsenin

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Constructing Trustworthy and Safe Communities on a Blockchain-Enabled Social Credits System

Ronghua Xu, Xuheng Lin, Qi Dong, Yu Chen

专题命中 安全训练 ：trustworthy(title,abstract)；分类 cs.CY

Comments Position paper, accepted by The 1st Workshop on Distributed Ledger of Things (DLoT 2018), co-located with EAI Mobiquitous 2018. Nov. 5, 2018, New York, NY, USA

Safe Exploration in Markov Decision Processes with Time-Variant Safety using Spatio-Temporal Gaussian Process

Akifumi Wachi, Hiroshi Kajino, Asim Munawar

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments 12 pages, 7 figures

Safety-Aware Apprenticeship Learning

Weichao Zhou, Wenchao Li

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments Accepted by International Conference on Computer Aided Verification (CAV) 2018

On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products

Kush R. Varshney, Homa Alemzadeh

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CY

Comments Big Data, 2017

Distributed Safety Critical Control among Uncontrollable Agents Using Reconstructed Control Barrier Functions

在不可控代理中使用重建的控制屏障函数实现分布式安全关键控制

Yuzhang Peng, Wei Wang, Jiaqi Yan, Mengze Yu

专题命中 安全训练 ：safety(title,abstract)

AI总结 本文研究了在存在不确定行为的不可控代理的多智能体系统中分布式安全关键控制的问题，提出了一种新的重建控制屏障函数方法，通过分布式自适应观测器获得其他代理的状态估计来重建耦合的控制屏障函数，并设计了安全关键二次规划控制器，证明了所提出的分布式控制方案在不确定动态环境中能够严格保证系统安全。

Journal ref Y. Peng, W. Wang, J. Yan and M. Yu, "Distributed Safety Critical Control among Uncontrollable Agents Using Reconstructed Control Barrier Functions," 2026 European Control Conference (ECC), Reykjavík, Iceland, 2026, pp. 1084-1089

Policy Prototyping for LLMs: Pluralistic Alignment via Interactive and Collaborative Policymaking

K. J. Kevin Feng, Inyoung Cheong, Quan Ze Chen, Amy X. Zhang

专题命中 安全训练 ：alignment(title,abstract)

Comments Bidirectional Human-AI Alignment (Bi-Align) Workshop @ ICLR 2025

Design and Development of a Roaming Wireless Safety Emergency Stop

Henry Beuster, Thomas Doebbert, Christoph Cammin, Dmytro Krush, Gerd Scholl

专题命中 安全训练 ：safety(title,abstract)

Comments 7 Pages, accepted for Safety of Industrial Automated Systems, SIAS 2024

Low-Agreeableness Persona Conditioning for Safe LLM Fine-Tuning

低宜人性人格条件化用于安全的大语言模型微调

Austin MY Cheung, Yi Yang

机构 * Hong Kong University of Science and Technology（香港科技大学） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

AI总结 针对温暖微调降低大语言模型安全性的问题，提出基于低宜人性人格的改写流水线，在保持对话温暖的同时降低越狱成功率与有害输出率。

Comments 9 pages, 8 tables, 5 figures

NP-Hard Lower Bound Complexity for Semantic Self-Verification

语义自验证的NP难下界复杂度

Robin Young

机构 * University of Cambridge（剑桥大学） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；AI safety(abstract)；分类 cs.CL、cs.AI

AI总结 本文证明语义自验证问题在特定框架下为NP完全，通过将3SAT问题归约到SSV，揭示了即使简化形式的语义自验证也面临计算障碍，对AI安全和公平性方法有重要影响。

Comments EACL 2026

Persona Features Control Emergent Misalignment

Miles Wang, Tom Dupré la Tour, Olivia Watkins, Alex Makelov, Ryan A. Chi, Samuel Miserendino, Jeffrey Wang, Achyuta Rajaram, Johannes Heidecke, Tejal Patwardhan, Dan Mossing

机构 * OpenAI 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；AI safety(abstract)；分类 cs.AI、cs.LG

A Call to Action for a Secure-by-Design Generative AI Paradigm

Dalal Alharthi, Ivan Roberto Kawaminami Garcia

机构 * University of Arizona（亚利桑那大学） 纠错

专题命中 安全训练 ：safety(abstract)；prompt injection(abstract)；AI safety(abstract)；分类 cs.AI、cs.LG