大模型对齐与安全

X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents

Salman Rahman, Liwei Jiang, James Shiffer, Genglin Liu, Sheriff Issaka, Md Rizwan Parvez, Hamid Palangi, Kai-Wei Chang, Yejin Choi, Saadia Gabriel

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation

Jun Zhuang, Haibo Jin, Ye Zhang, Zhengjian Kang, Wenbin Zhang, Gaby G. Dagher, Haohan Wang

机构 * Boise State University（博伊州立大学） ； University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； University of Pittsburgh（匹兹堡大学） ； New York University（纽约大学） ； Florida International University（佛罗里达国际大学） 纠错

专题命中 安全训练 ：safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments Accepted for EMNLP'25 Findings. TL;DR: We propose a new two-stage intent-based prompt-refinement framework, IntentPrompt, that aims to explore the vulnerability of LLMs' content moderation guardrails by refining prompts into benign-looking declarative forms via intent manipulation for red-teaming purposes

Effective Red-Teaming of Policy-Adherent Agents

Itay Nakash, George Kour, Koren Lazar, Matan Vetzler, Guy Uziel, Ateret Anaby-Tavor

专题命中 安全训练 ：jailbreak(abstract)；分类 cs.CL、cs.AI

An Outlook on the Opportunities and Challenges of Multi-Agent AI Systems

Fangqiao Tian, An Luo, Jin Du, Xun Xian, Robert Specht, Ganghua Wang, Xuan Bi, Jiawei Zhou, Ashish Kundu, Jayanth Srinivasa, Charles Fleming, Rui Zhang, Zirui Liu, Mingyi Hong, Jie Ding

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Comments Corrected references

SafeBimanual: Diffusion-based Trajectory Optimization for Safe Bimanual Manipulation

Haoyuan Deng, Wenkai Guo, Qianzhun Wang, Zhenyu Wu, Ziwei Wang

机构 * Nanyang Technological University（南洋理工大学） ； Beijing University of Posts and Telecommunications（北京邮电大学） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.AI

Comments Project website is at: https://denghaoyuan123.github.io/SafeBimanip/

School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMs

Mia Taylor, James Chua, Jan Betley, Johannes Treutlein, Owain Evans

专题命中 安全训练 ：alignment(abstract)；分类 cs.AI

Comments 42 pages, 26 figures

Towards Safeguarding LLM Fine-tuning APIs against Cipher Attacks

Jack Youstra, Mohammed Mahfoud, Yang Yan, Henry Sleight, Ethan Perez, Mrinank Sharma

专题命中 安全训练 ：safety(abstract)；分类 cs.LG