大模型对齐与安全 - arXivDaily 专题

2409.15461 2024-09-25 cs.AI cs.CL 62%

RAM2C: A Liberal Arts Educational Chatbot based on Retrieval-augmented Multi-role Multi-expert Collaboration

Haoyu Huang, Tong Niu, Rui Yang, Luping Shi

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI

2305.13681 2024-09-25 cs.LG cs.AI cs.RO 62%

GUARD: A Safe Reinforcement Learning Benchmark

Weiye Zhao, Yifan Sun, Feihan Li, Rui Chen, Ruixuan Liu, Tianhao Wei, Changliu Liu

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Comments Published in Transaction of Machine Learning Research

2309.00236 2024-09-19 cs.LG cs.CL cs.CR 62%

Image Hijacks: Adversarial Images can Control Generative Models at Runtime

Luke Bailey, Euan Ong, Stuart Russell, Scott Emmons

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.LG

Comments Project page at https://image-hijacks.github.io

2403.18209 2024-09-13 cs.LG cs.AI cs.RO 62%

Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Driving

Xuemin Hu, Pan Chen, Yijun Wen, Bo Tang, Long Chen

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

2312.03814 2024-09-04 cs.LG cs.AI 62%

Pearl: A Production-ready Reinforcement Learning Agent

Zheqing Zhu, Rodrigo de Salvo Braz, Jalaj Bhandari, Daniel Jiang, Yi Wan, Yonathan Efroni, Liyuan Wang, Ruiyang Xu, Hongbo Guo, Alex Nikulkov, Dmytro Korenkevych, Urun Dogan, Frank Cheng, Zheng Wu, Wanqiao Xu

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Journal ref Journal of Machine Learning Research, 2024

2403.15826 2024-08-29 eess.SY cs.AI cs.LG cs.RO cs.SY 62%

Scaling Learning based Policy Optimization for Temporal Logic Tasks by Controller Network Dropout

Navid Hashemi, Bardh Hoxha, Danil Prokhorov, Georgios Fainekos, Jyotirmoy Deshmukh

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

2402.04580 2024-08-28 cs.RO cs.AI cs.LG 62%

A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents

Haoyi Niu, Jianming Hu, Guyue Zhou, Xianyuan Zhan

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Comments IJCAI 2024

2408.11021 2024-08-21 cs.CL cs.AI cs.MA 62%

Athena: Safe Autonomous Agents with Verbal Contrastive Learning

Tanmana Sadhu, Ali Pesaranghader, Yanan Chen, Dong Hoon Yi

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 2 figures, 4 tables

2408.07962 2024-08-16 cs.LG cs.AI cs.RO cs.SY eess.SY 62%

Meta SAC-Lag: Towards Deployable Safe Reinforcement Learning via MetaGradient-based Hyperparameter Tuning

Homayoun Honari, Amir Mehdi Soufi Enayati, Mehran Ghafarian Tamizi, Homayoun Najjaran

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Comments Main text accepted to the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2024, 10 pages, 4 figures, 3 tables

2407.19760 2024-08-12 cs.CL cs.CY 62%

Legal Minds, Algorithmic Decisions: How LLMs Apply Constitutional Principles in Complex Scenarios

Camilla Bignotti, Carolina Camassa

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.CY

Comments Accepted at AIES24

2402.17649 2024-08-12 cs.CL cs.CY 62%

Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMs

Tanise Ceron, Neele Falk, Ana Barić, Dmitry Nikolaev, Sebastian Padó

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.CY

Comments 12 pages, TACL

2407.21772 2024-08-06 cs.CL cs.LG 62%

ShieldGemma: Generative AI Content Moderation Based on Gemma

Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, Olivia Sturman, Oscar Wahltinez

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.LG

2312.08381 2024-07-31 cs.LG cs.AI 62%

An Explainable Machine Learning Framework for the Accurate Diagnosis of Ovarian Cancer

Asif Newaz, Abdullah Taharat, Md Sakibul Islam, A. G. M. Fuad Hasan Akanda

专题命中安全训练：trustworthy(abstract)；分类 cs.AI、cs.LG

2407.15718 2024-07-23 cs.CY cs.AI cs.HC cs.IR cs.SE 62%

Integrating AI Tutors in a Programming Course

Iris Ma, Alberto Krone Martins, Cristina Videira Lopes

专题命中安全训练：alignment(abstract)；分类 cs.AI、cs.CY

Comments Accepted at SIGCSE Virtual 2024

2407.10995 2024-07-22 cs.CL cs.AI 62%

LionGuard: Building a Contextualized Moderation Classifier to Tackle Localized Unsafe Content

Jessica Foo, Shaun Khoo

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI

Comments Preprint

2205.06212 2024-07-17 eess.SY cs.AI cs.LG cs.SY 62%

Contingency-constrained economic dispatch with safe reinforcement learning

Michael Eichelbeck, Hannah Markgraf, Matthias Althoff

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Journal ref 21st IEEE International Conference on Machine Learning and Applications (ICMLA), 2022, pp. 597-602

2407.05580 2024-07-09 cs.LG cs.AI 62%

$\mathrm{E^{2}CFD}$: Towards Effective and Efficient Cost Function Design for Safe Reinforcement Learning via Large Language Model

Zepeng Wang, Chao Ma, Linjiang Zhou, Libing Wu, Lei Yang, Xiaochuan Shi, Guojun Peng

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

2402.09132 2024-07-09 cs.AI cs.LG 62%

Exploring the Adversarial Capabilities of Large Language Models

Lukas Struppek, Minh Hieu Le, Dominik Hintersdorf, Kristian Kersting

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

2301.11461 2024-07-08 cs.LG cs.AI 62%

Learning to Generate All Feasible Actions

Mirco Theile, Daniele Bernardini, Raphael Trumpp, Cristina Piazza, Marco Caccamo, Alberto L. Sangiovanni-Vincentelli

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

2407.00004 2024-07-02 q-bio.BM cs.AI cs.LG q-bio.QM 62%

Multi-objective generative AI for designing novel brain-targeting small molecules

Ayush Noori, Iñaki Arango, William E. Byrd, Nada Amin

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Comments 20 pages, 4 figures, Generative and Experimental Perspectives for Biomolecular Design Workshop at the 12th International Conference on Learning Representations

2406.15508 2024-06-25 q-fin.CP cs.AI cs.LG cs.RO 62%

What Teaches Robots to Walk, Teaches Them to Trade too -- Regime Adaptive Execution using Informed Data and LLMs

Raeid Saqur

专题命中安全训练：alignment(abstract)；分类 cs.AI、cs.LG

Comments arXiv admin note: substantial text overlap with arXiv:2405.09747

2405.13245 2024-06-25 cs.RO cs.AI cs.CL 62%

A Survey of Robotic Language Grounding: Tradeoffs between Symbols and Embeddings

Vanya Cohen, Jason Xinyu Liu, Raymond Mooney, Stefanie Tellex, David Watkins

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI

Comments IJCAI 2024 Survey Track

2406.11844 2024-06-19 cs.CY cs.AI 62%

Prompting the E-Brushes: Users as Authors in Generative AI

Yiyang Mei

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.CY

Journal ref International Journal of Law, Ethics, and Technology 2024

2405.12604 2024-06-19 cs.CL cs.AI 62%

Tiny Refinements Elicit Resilience: Toward Efficient Prefix-Model Against LLM Red-Teaming

Jiaxu Liu, Xiangyu Yin, Sihao Wu, Jianhong Wang, Meng Fang, Xinping Yi, Xiaowei Huang

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI

Comments Preprint, 10 pages main with 10 pages appendix

2402.11114 2024-06-19 cs.CL cs.CY cs.SI 62%

Whose Emotions and Moral Sentiments Do Language Models Reflect?

Zihao He, Siyi Guo, Ashwin Rao, Kristina Lerman

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.CY

2401.03160 2024-06-18 cs.LG cs.AI cs.RO 62%

HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving

Zilin Huang, Zihao Sheng, Chengyuan Ma, Sikai Chen

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Comments Accepted by Communications in Transportation Research

2406.06657 2024-06-12 cs.CL cs.AI cs.IR 62%

Harnessing AI for efficient analysis of complex policy documents: a case study of Executive Order 14110

Mark A. Kramer, Allen Leavens, Alexander Scarlat

专题命中安全训练：trustworthy(abstract)；分类 cs.CL、cs.AI

Comments 28 pages, 1 figure

2402.06700 2024-06-07 cs.LG cs.AI 62%

Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement

Muning Wen, Junwei Liao, Cheng Deng, Jun Wang, Weinan Zhang, Ying Wen

专题命中安全训练：RLHF(abstract)；分类 cs.AI、cs.LG

2405.20413 2024-06-03 cs.CR cs.CL cs.CV cs.LG 62%

Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters

Haibo Jin, Andy Zhou, Joe D. Menke, Haohan Wang

专题命中安全训练：jailbreak(abstract)；分类 cs.CL、cs.LG

Comments 20 pages

2405.17039 2024-05-28 cs.CL cs.LG 62%

BWArea Model: Learning World Model, Inverse Dynamics, and Policy for Controllable Language Generation

Chengxing Jia, Pengyuan Wang, Ziniu Li, Yi-Chen Li, Zhilong Zhang, Nan Tang, Yang Yu

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.LG

1. 安全训练 3309 篇

RAM2C: A Liberal Arts Educational Chatbot based on Retrieval-augmented Multi-role Multi-expert Collaboration

GUARD: A Safe Reinforcement Learning Benchmark

Image Hijacks: Adversarial Images can Control Generative Models at Runtime

Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Driving

Pearl: A Production-ready Reinforcement Learning Agent

Scaling Learning based Policy Optimization for Temporal Logic Tasks by Controller Network Dropout

A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents

Athena: Safe Autonomous Agents with Verbal Contrastive Learning

Meta SAC-Lag: Towards Deployable Safe Reinforcement Learning via MetaGradient-based Hyperparameter Tuning

Legal Minds, Algorithmic Decisions: How LLMs Apply Constitutional Principles in Complex Scenarios

Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMs

ShieldGemma: Generative AI Content Moderation Based on Gemma

An Explainable Machine Learning Framework for the Accurate Diagnosis of Ovarian Cancer

Integrating AI Tutors in a Programming Course

LionGuard: Building a Contextualized Moderation Classifier to Tackle Localized Unsafe Content

Contingency-constrained economic dispatch with safe reinforcement learning

$\mathrm{E^{2}CFD}$: Towards Effective and Efficient Cost Function Design for Safe Reinforcement Learning via Large Language Model

Exploring the Adversarial Capabilities of Large Language Models

Learning to Generate All Feasible Actions

Multi-objective generative AI for designing novel brain-targeting small molecules

What Teaches Robots to Walk, Teaches Them to Trade too -- Regime Adaptive Execution using Informed Data and LLMs

A Survey of Robotic Language Grounding: Tradeoffs between Symbols and Embeddings

Prompting the E-Brushes: Users as Authors in Generative AI

Tiny Refinements Elicit Resilience: Toward Efficient Prefix-Model Against LLM Red-Teaming

Whose Emotions and Moral Sentiments Do Language Models Reflect?

HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving

Harnessing AI for efficient analysis of complex policy documents: a case study of Executive Order 14110

Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement

Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters

BWArea Model: Learning World Model, Inverse Dynamics, and Policy for Controllable Language Generation