大模型对齐与安全 - arXivDaily 专题

2503.00610 2025-03-04 cs.CY cs.AI 81%

Urban Safety Perception Through the Lens of Large Multimodal Models: A Persona-based Approach

Ciro Beneduce, Bruno Lepri, Massimiliano Luca

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.CY

2405.18540 2025-03-03 cs.CL cs.CR cs.LG 81%

Learning diverse attacks on large language models for robust red-teaming and safety tuning

Seanie Lee, Minsu Kim, Lynn Cherif, David Dobre, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi, Gauthier Gidel, Yoshua Bengio, Nikolay Malkin, Moksh Jain

专题命中安全训练：safety(title,abstract)；分类 cs.CL、cs.LG

Comments ICLR 2025

2502.13430 2025-02-20 cs.AI cs.LG 81%

Vision-Based Generic Potential Function for Policy Alignment in Multi-Agent Reinforcement Learning

Hao Ma, Shijie Wang, Zhiqiang Pu, Siyao Zhao, Xiaolin Ai

专题命中安全训练：alignment(title,abstract)；分类 cs.AI、cs.LG

2502.12025 2025-02-18 cs.AI cs.CL 81%

SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities

Fengqing Jiang, Zhangchen Xu, Yuetai Li, Luyao Niu, Zhen Xiang, Bo Li, Bill Yuchen Lin, Radha Poovendran

专题命中安全训练：safety(title,abstract)；分类 cs.CL、cs.AI

2408.09366 2025-02-12 cs.CL cs.CY cs.SI 81%

Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communities

Minh Duc Chu, Zihao He, Rebecca Dorn, Kristina Lerman

专题命中安全训练：alignment(title,abstract)；分类 cs.CL、cs.CY

2406.03912 2025-01-15 cs.AI cs.LG cs.RO cs.SY eess.SY 81%

GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Model

Zhehua Zhou, Xuan Xie, Jiayang Song, Zhan Shu, Lei Ma

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

2412.16038 2025-01-03 cs.CY cs.AI 81%

Intelligent Approaches to Predictive Analytics in Occupational Health and Safety in India

Ritwik Raj Saxena

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.CY

2412.06869 2024-12-11 cs.LG cs.AI cs.CV cs.SE 81%

Safety Monitoring of Machine Learning Perception Functions: a Survey

Raul Sena Ferreira, Joris Guérin, Kevin Delmas, Jérémie Guiochet, Hélène Waeselynck

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments 25 pages, 2 figures

2407.20208 2024-12-02 cs.AI cs.LG cs.NE 81%

Supertrust foundational alignment: mutual trust must replace permanent control for safe superintelligence

James M. Mazzu

专题命中安全训练：alignment(title,abstract)；分类 cs.AI、cs.LG

2407.19860 2024-11-06 cs.LG cs.AI 81%

Anomalous State Sequence Modeling to Enhance Safety in Reinforcement Learning

Leen Kweider, Maissa Abou Kassem, Ubai Sandouk

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Journal ref IEEE Access, vol. 12, pp. 157140-157148, Oct. 25, 2024

2406.10847 2024-10-29 cs.AI cs.CE cs.CL cs.MA 81%

TorchOpera: A Compound AI System for LLM Safety

Shanshan Han, Zijian Hu, Alay Dilipbhai Shah, Han Jin, Yuhang Yao, Dimitris Stripelis, Zhaozhuo Xu, Chaoyang He

专题命中安全训练：safety(title,abstract)；分类 cs.CL、cs.AI

2410.15489 2024-10-22 cs.RO cs.AI cs.LG 81%

Generative AI Agents in Autonomous Machines: A Safety Perspective

Jason Jabbour, Vijay Janapa Reddi

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

2410.10014 2024-10-15 cs.CL cs.AI 81%

Safety-Aware Fine-Tuning of Large Language Models

Hyeong Kyu Choi, Xuefeng Du, Yixuan Li

专题命中安全训练：safety(title,abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024 Workshop on Safe Generative AI

2310.12567 2024-10-08 cs.AI cs.LG 81%

Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark

Jiaming Ji, Borong Zhang, Jiayi Zhou, Xuehai Pan, Weidong Huang, Ruiyang Sun, Yiran Geng, Yifan Zhong, Juntao Dai, Yaodong Yang

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments Published at NeurIPS 2023

2206.09743 2024-09-13 cs.AI cs.LG cs.SY eess.SY 81%

Guided Safe Shooting: model based reinforcement learning with safety constraints

Giuseppe Paolo, Jonas Gonzalez-Billandon, Albert Thomas, Balázs Kégl

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

2401.00391 2024-08-08 cs.RO cs.AI cs.CV cs.LG 81%

SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries

Wei-Jer Chang, Francesco Pittaluga, Masayoshi Tomizuka, Wei Zhan, Manmohan Chandraker

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by ECCV2024; Project website: https://safe-sim.github.io/

2403.10462 2024-03-20 cs.CY cs.AI 81%

Safety Cases: How to Justify the Safety of Advanced AI Systems

Joshua Clymer, Nick Gabrieli, David Krueger, Thomas Larsen

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.CY

2306.17279 2024-03-14 cs.LG cs.AI 81%

Probabilistic Constraint for Safety-Critical Reinforcement Learning

Weiqin Chen, Dharmashankar Subramanian, Santiago Paternain

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments arXiv admin note: text overlap with arXiv:2210.00596

2312.10230 2023-12-19 cs.AI cs.LG 81%

Constrained Meta-Reinforcement Learning for Adaptable Safety Guarantee with Differentiable Convex Programming

Minjae Cho, Chuangchuang Sun

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

2309.15025 2023-09-27 cs.CL cs.AI 81%

Large Language Model Alignment: A Survey

Tianhao Shen, Renren Jin, Yufei Huang, Chuang Liu, Weilong Dong, Zishan Guo, Xinwei Wu, Yan Liu, Deyi Xiong

专题命中安全训练：alignment(title,abstract)；分类 cs.CL、cs.AI

Comments 76 pages

2307.02867 2023-07-07 cs.SE cs.AI cs.LG cs.SY eess.SY 81%

Towards a safe MLOps Process for the Continuous Development and Safety Assurance of ML-based Systems in the Railway Domain

Marc Zeller, Thomas Waschulzik, Reiner Schmid, Claus Bahlmann

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

2306.06135 2023-06-13 cs.LG cs.AI 81%

Safety and Fairness for Content Moderation in Generative Models

Susan Hao, Piyush Kumar, Sarah Laszlo, Shivani Poddar, Bhaktipriya Radharapu, Renee Shelby

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments CVPR Workshop Paper

2305.04727 2023-05-09 cs.LG cs.AI 81%

DEFENDER: DTW-Based Episode Filtering Using Demonstrations for Enhancing RL Safety

André Correia, Luís Alexandre

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

2302.09318 2023-02-21 cs.LG cs.AI 81%

Effective Multimodal Reinforcement Learning with Modality Alignment and Importance Enhancement

Jinming Ma, Feng Wu, Yingfeng Chen, Xianpeng Ji, Yu Ding

专题命中安全训练：alignment(title,abstract)；分类 cs.AI、cs.LG

Comments 10 pages, 12 figures, This article is an extended version of the Extended Abstract accepted by the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-2022)

2301.11857 2023-02-07 cs.AI cs.LG cs.MA 81%

Policy-Value Alignment and Robustness in Search-based Multi-Agent Learning

Niko A. Grupen, Michael Hanlon, Alexis Hao, Daniel D. Lee, Bart Selman

专题命中安全训练：alignment(title,abstract)；分类 cs.AI、cs.LG

Comments 9 pages, 5 figures

2211.10237 2022-11-21 cs.RO cs.AI cs.LG 81%

Rationale-aware Autonomous Driving Policy utilizing Safety Force Field implemented on CARLA Simulator

Ho Suk, Taewoo Kim, Hyungbin Park, Pamul Yadav, Junyong Lee, Shiho Kim

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments 9 pages including appendices, 4 figures, NeurIPS 2022 Workshop: Machine Learning for Autonomous Driving (ML4AD)

2211.04530 2022-11-10 cs.LG cs.CY 81%

Creating a Safety Assurance Case for an ML Satellite-Based Wildfire Detection and Alert System

Richard Hawkins, Chiara Picardi, Lucy Donnell, Murray Ireland

专题命中安全训练：safety(title,abstract)；分类 cs.CY、cs.LG

2201.12427 2022-10-12 cs.LG cs.AI cs.RO 81%

Towards Safe Reinforcement Learning with a Safety Editor Policy

Haonan Yu, Wei Xu, Haichao Zhang

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2022 camera-ready version

2111.07395 2022-06-24 cs.LG cs.AI cs.RO 81%

Explicit Explore, Exploit, or Escape ($E^4$): near-optimal safety-constrained reinforcement learning in polynomial time

David M. Bossens, Nicholas Bishop

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted at Machine Learning

2201.12819 2022-02-01 cs.AI cs.LG cs.RO cs.SY eess.SY 81%

A Safety-Critical Decision Making and Control Framework Combining Machine Learning and Rule-based Algorithms

Andrei Aksjonov, Ville Kyrki

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments 11 pages, 9 figures, 2 tables

1. 安全训练 3288 篇

Urban Safety Perception Through the Lens of Large Multimodal Models: A Persona-based Approach

Learning diverse attacks on large language models for robust red-teaming and safety tuning

Vision-Based Generic Potential Function for Policy Alignment in Multi-Agent Reinforcement Learning

SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities

Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communities

GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Model

Intelligent Approaches to Predictive Analytics in Occupational Health and Safety in India

Safety Monitoring of Machine Learning Perception Functions: a Survey

Supertrust foundational alignment: mutual trust must replace permanent control for safe superintelligence

Anomalous State Sequence Modeling to Enhance Safety in Reinforcement Learning

TorchOpera: A Compound AI System for LLM Safety

Generative AI Agents in Autonomous Machines: A Safety Perspective

Safety-Aware Fine-Tuning of Large Language Models

Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark

Guided Safe Shooting: model based reinforcement learning with safety constraints

SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries

Safety Cases: How to Justify the Safety of Advanced AI Systems

Probabilistic Constraint for Safety-Critical Reinforcement Learning

Constrained Meta-Reinforcement Learning for Adaptable Safety Guarantee with Differentiable Convex Programming

Large Language Model Alignment: A Survey

Towards a safe MLOps Process for the Continuous Development and Safety Assurance of ML-based Systems in the Railway Domain

Safety and Fairness for Content Moderation in Generative Models

DEFENDER: DTW-Based Episode Filtering Using Demonstrations for Enhancing RL Safety

Effective Multimodal Reinforcement Learning with Modality Alignment and Importance Enhancement

Policy-Value Alignment and Robustness in Search-based Multi-Agent Learning

Rationale-aware Autonomous Driving Policy utilizing Safety Force Field implemented on CARLA Simulator

Creating a Safety Assurance Case for an ML Satellite-Based Wildfire Detection and Alert System

Towards Safe Reinforcement Learning with a Safety Editor Policy

Explicit Explore, Exploit, or Escape ($E^4$): near-optimal safety-constrained reinforcement learning in polynomial time

A Safety-Critical Decision Making and Control Framework Combining Machine Learning and Rule-based Algorithms