大模型对齐与安全

LinguaSim: Interactive Multi-Vehicle Testing Scenario Generation via Natural Language Instruction Based on Large Language Models

Qingyuan Shi, Qingwen Meng, Hao Cheng, Qing Xu, Jianqiang Wang

机构 * National Key R&D Program of China（中华人民共和国国家重点研发计划） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.AI

Beyond Monoliths: Expert Orchestration for More Capable, Democratic, and Safe Language Models

Philip Quirke, Narmeen Oozeer, Chaithanya Bandi, Amir Abdullah, Jason Hoelscher-Obermaier, Jeff M. Phillips, Joshua Greaves, Clement Neo, Michael Lan, Fazl Barez, Shriyash Upadhyay

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.CY

Comments 8 pages, 2 figures

Improving Consistency in Retrieval-Augmented Systems with Group Similarity Rewards

Faisal Hamman, Chenyang Zhu, Anoop Kumar, Xujun Peng, Sanghamitra Dutta, Daben Liu, Alfy Samuel

机构 * University of Maryland, College Park（马里兰大学学院公园分校） ； Capital One 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI、cs.CY

Comments Accepted at NeurIPS 2025 Workshop on Reliable ML from Unreliable Data

Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs

Jiawei Kong, Hao Fang, Xiaochen Yang, Kuofeng Gao, Bin Chen, Shu-Tao Xia, Ke Xu, Han Qiu

机构 * Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学） ； Department of Software Engineering, Harbin Institute of Technology（哈尔滨工业大学软件工程系） ； School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学深圳校区计算机科学与技术学院） ； Institute for Network Sciences and Cyberspace, Tsinghua University（清华大学网络科学与空间研究院） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.CL

LatentGuard: Controllable Latent Steering for Robust Refusal of Attacks and Reliable Response Generation

Huizhen Shu, Xuying Li, Zhuo Li

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.AI

Comments 9-page NeurIPS 2025 preprint including 3 figures and 1 table, with additional appendix material. Prepared using the NeurIPS 2025 preprint template and compiled with pdfLaTeX. All references are included via the provided .bbl file. Figures are in PDF format. No external supplementary files. All necessary style files and images are included

SAFE: Improving LLM Systems using Sentence-Level In-generation Attribution

João Eduardo Batista, Emil Vatai, Mohamed Wahib

机构 * RIKEN-CCS Kobe, Japan（日本神户RIKEN-CCS） 纠错

专题命中 安全训练 ：safety(abstract)；trustworthy(abstract)；分类 cs.CL

Comments 30 pages (9 pages of content, 5 pages of references, 16 pages of supplementary material), 7 figures, 13 tables

PIGDreamer: Privileged Information Guided World Models for Safe Partially Observable Reinforcement Learning

Dongchi Huang, Jiaqi Wang, Yang Li, Chunhe Xia, Tianle Zhang, Kaige Zhang

机构 * School of Computer Science, University of Beihang, Beijing, China（北京航空航天大学计算机学院） ； School of Computer Science, Chinese University of Hong Kong, Hongkong, China（香港中文大学计算机学院） ； JD Explore Academy, Beijing, China（京东探索研究院） ； North Automatic Control Institute, Taiyuan, China（太原北自动控制研究所） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.LG

Comments ICML 2025

Explainable AI-Enhanced Supervisory Control for Robust Multi-Agent Robotic Systems

Reza Pirayeshshirazinezhad, Nima Fathi

机构 * Visual Computing and Computational Media, Texas A&M University（视觉计算与计算媒体，德克萨斯A&M大学） ； Mechanical Engineering Department, Texas A&M University（机械工程系，德克萨斯A&M大学） ； Department of Marine Engineering Technology, Texas A&M University（海洋工程技术系，德克萨斯A&M大学） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.AI

Activation Space Interventions Can Be Transferred Between Large Language Models

Narmeen Oozeer, Dhruv Nathawani, Nirmalendu Prakash, Michael Lan, Abir Harrasse, Amirali Abdullah

机构 * Nvidia ； Singapore University of Technology and Design（新加坡技术与设计大学） 纠错

专题命中 安全训练 ：safety(abstract)；AI safety(abstract)；分类 cs.AI

Comments 75 pages. Accepted to ICML 2025

Keep Security! Benchmarking Security Policy Preservation in Large Language Model Contexts Against Indirect Attacks in Question Answering

Hwan Chang, Yumin Kim, Yonghyun Jun, Hwanhee Lee

机构 * Chung-Ang University（Chung-Ang 大学） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.CL

Comments EMNLP 2025 (Main Conference)

No for Some, Yes for Others: Persona Prompts and Other Sources of False Refusal in Language Models

Flor Miriam Plaza-del-Arco, Paul Röttger, Nino Scherrer, Emanuele Borgonovo, Elmar Plischke, Dirk Hovy

机构 * LIACS, Leiden University（莱顿大学 LIACS） ； Bocconi University（博科尼大学） ； Independent Researcher（独立研究者） ； Boconni University（博科尼大学） ； Helmholtz-Zentrum Dresden-Rossendorf（德累斯顿-罗斯托克亥姆霍兹中心） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.CL

\texttt{R$^\textbf{2}$AI}: Towards Resistant and Resilient AI in an Evolving World

Youbang Sun, Xiang Wang, Jie Fu, Chaochao Lu, Bowen Zhou

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Tsinghua University（清华大学） ； University of Science and Technology of China（中国科学技术大学） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.LG

Can Media Act as a Soft Regulator of Safe AI Development? A Game Theoretical Analysis

Henrique Correia da Fonseca, António Fernandes, Zhao Song, Theodor Cimpeanu, Nataliya Balabanova, Adeela Bashir, Paolo Bova, Alessio Buscemi, Alessandro Di Stefano, Manh Hong Duong, Elias Fernandez Domingos, Ndidi Bianca Ogbo, Simon T. Powers, Daniele Proverbio, Zia Ush Shamszaman, Fernando P. Santos, The Anh Han, Marcus Krellner

机构 * INESC-ID and Instituto Superior Técnico, Universidade de Lisboa（INESC-ID和里斯本大学理工学院） ； School of Computing, Engineering and Digital Technologies, Teesside University（计算、工程与数字技术学院，泰赛德大学） ； Biological and Environmental Sciences, University of Stirling（生物与环境科学，斯特林大学） ； School of Mathematics, University of Birmingham（数学学院，伯明翰大学） ； Luxembourg Institute of Science and Technology（卢森堡科学与技术研究院） ； Machine Learning Group, Université libre de Bruxelles（机器学习小组，布鲁塞尔自由大学） ； AI Lab, Vrije Universiteit Brussel（人工智能实验室，布鲁塞尔自由大学） ； Division of Computing Science and Mathematics, University of Stirling（计算科学与数学系，斯特林大学） ； Department of Industrial Engineering, University of Trento（工业工程系，特伦特大学） ； University of Amsterdam（阿姆斯特丹大学） 纠错

专题命中 安全训练 ：safety(abstract)；AI safety(abstract)；分类 cs.AI

Comments 10 Pages, 7 Figures, accepted in the ALIFE 2025 Conference

IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinement

Yuanzhe Shen, Zisu Huang, Zhengkang Guo, Yide Liu, Guanxu Chen, Ruicheng Yin, Xiaoqing Zheng, Xuanjing Huang

专题命中 安全训练 ：safety(abstract)；jailbreak(abstract)；分类 cs.AI

Comments 17 pages, 9 figures

S3LoRA: Safe Spectral Sharpness-Guided Pruning in Adaptation of Agent Planner

Shuang Ao, Gopal Rumchurn

机构 * Shuang Ao ； Gopal Rumchurn 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.AI

Comments 9 pages, 2 figures

Do Large Language Model Agents Exhibit a Survival Instinct? An Empirical Study in a Sugarscape-Style Simulation

Atsushi Masumori, Takashi Ikegami

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.AI

Slow Tuning and Low-Entropy Masking for Safe Chain-of-Thought Distillation

Ziyang Ma, Qingyue Yuan, Linhai Zhang, Deyu Zhou

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.CL

Comments Preprint

Cognitive Cybersecurity for Artificial Intelligence: Guardrail Engineering with CCS-7

Yuksel Aydin

机构 * Independent Researcher（独立研究者） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.AI

Gradient Surgery for Safe LLM Fine-Tuning

Biao Yi, Jiahao Li, Baolei Zhang, Lihai Nie, Tong Li, Tiansheng Huang, Zheli Liu

机构 * Nankai University（南开大学） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.CL

VerificAgent: Domain-Specific Memory Verification for Scalable Oversight of Aligned Computer-Use Agents

Thong Q. Nguyen, Shubhang Desai, Raja Hasnain Anwar, Firoz Shaik, Vishwas Suryanarayanan, Vishal Chowdhary

机构 * Microsoft（微软公司） ； University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.LG

Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems

Bang Liu, Xinfeng Li, Jiayi Zhang, Jinlin Wang, Tanjin He, Sirui Hong, Hongzhang Liu, Shaokun Zhang, Kaitao Song, Kunlun Zhu, Yuheng Cheng, Suyuchen Wang, Xiaoqiang Wang, Yuyu Luo, Haibo Jin, Peiyan Zhang, Ollie Liu, Jiaqi Chen, Huan Zhang, Zhaoyang Yu, Haochen Shi, Boyan Li, Dekun Wu, Fengwei Teng, Xiaojun Jia, Jiawei Xu, Jinyu Xiang, Yizhang Lin, Tianming Liu, Tongliang Liu, Yu Su, Huan Sun, Glen Berseth, Jianyun Nie, Ian Foster, Logan Ward, Qingyun Wu, Yu Gu, Mingchen Zhuge, Xinbing Liang, Xiangru Tang, Haohan Wang, Jiaxuan You, Chi Wang, Jian Pei, Qiang Yang, Xiaoliang Qi, Chenglin Wu

专题命中 安全训练 ：alignment(abstract)；trustworthy(abstract)；分类 cs.AI

Analysis of Threat-Based Manipulation in Large Language Models: A Dual Perspective on Vulnerabilities and Performance Enhancement Opportunities

Atil Samancioglu

机构 * Atil Samancioglu（独立研究者） 纠错

专题命中 安全训练 ：safety(abstract)；AI safety(abstract)；分类 cs.AI

Mitigating Trojanized Prompt Chains in Educational LLM Use Cases: Experimental Findings and Detection Tool Design

Richard M. Charles, James H. Curry, Richard B. Charles

机构 * University of Colorado, Boulder（科罗拉多大学波德罗尔分校） ； Charles Analytics, Aurora（查尔斯分析公司） 纠错

专题命中 安全训练 ：safety(abstract)；AI safety(abstract)；分类 cs.AI

Comments 12 pages, 1 figure

Steering Language Model Refusal with Sparse Autoencoders

Kyle O'Brien, David Majercak, Xavier Fernandes, Richard Edgar, Blake Bullwinkel, Jingya Chen, Harsha Nori, Dean Carignan, Eric Horvitz, Forough Poursabzi-Sangdeh

机构 * Microsoft（微软） 纠错

专题命中 安全训练 ：safety(abstract)；jailbreak(abstract)；分类 cs.LG

SEAL: Vision-Language Model-Based Safe End-to-End Cooperative Autonomous Driving with Adaptive Long-Tail Modeling

Junwei You, Pei Li, Zhuoyu Jiang, Zilin Huang, Rui Gan, Haotian Shi, Bin Ran

机构 * Department of Civil and Environmental Engineering, University of Wisconsin–Madison（威斯康星大学麦迪逊分校土木与环境工程系） ； College of Computing and Data Science, Nanyang Technological University（南洋理工大学 computing and Data Science 学院） ； College of Transportation, Tongji University（同济大学交通运输学院） 纠错

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.AI

Command-V: Pasting LLM Behaviors via Activation Profiles

Barry Wang, Avi Schwarzschild, Alexander Robey, Ali Payani, Charles Fleming, Mingjie Sun, Daphne Ippolito

专题命中 安全训练 ：safety(abstract)；jailbreak(abstract)；分类 cs.LG

SentinelAgent: Graph-based Anomaly Detection in Multi-Agent Systems

Xu He, Di Wu, Yan Zhai, Kun Sun

机构 * Visa Inc.（Visa公司） ； George Mason University（乔治·梅奥大学） 纠错

专题命中 安全训练 ：prompt injection(abstract)；trustworthy(abstract)；分类 cs.AI

Get Experience from Practice: LLM Agents with Record & Replay

Erhu Feng, Wenbo Zhou, Zibin Liu, Le Chen, Yunpeng Dong, Cheng Zhang, Yisheng Zhao, Dong Du, Zhichao Hua, Yubin Xia, Haibo Chen

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.LG

Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Models

Tianqi Chen, Shujian Zhang, Mingyuan Zhou

专题命中 安全训练 ：safety(abstract)；trustworthy(abstract)；分类 cs.LG

Comments ICLR 2025

Learning from Active Human Involvement through Proxy Value Propagation

Zhenghao Peng, Wenjie Mo, Chenda Duan, Quanyi Li, Bolei Zhou

专题命中 安全训练 ：alignment(abstract)；safety(abstract)；分类 cs.AI

Comments NeurIPS 2023 Spotlight. Project page: https://metadriverse.github.io/pvp