大模型对齐与安全 - arXivDaily 专题

2511.21749 2025-12-01 cs.CL cs.AI 73%

Proactive Defense: Compound AI for Detecting Persuasion Attacks and Measuring Inoculation Effectiveness

主动防御：用于检测说服攻击并衡量免疫效果的复合AI

Svitlana Volkova, Will Dupree, Hsien-Te Kao, Peter Bautista, Gabe Ganberg, Jeff Beaubien, Laura Cassani

机构 * Aptima, Inc.（Aptima公司）

专题命中越狱攻击：safety(abstract)；AI safety(abstract)；分类 cs.CL、cs.AI

AI总结本文提出BRIES复合AI架构，通过专门代理检测说服攻击并评估免疫效果，揭示不同模型在处理说服语言上的差异。

2511.13548 2025-11-18 cs.CR cs.AI cs.CL 73%

ForgeDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models

Siyang Cheng, Gaotian Liu, Rui Mei, Yilin Wang, Kejia Zhang, Kaishuo Wei, Yuqi Yu, Weiping Wen, Xiaojie Wu, Junhua Liu

机构 * iFLYTEK Security Laboratory（iFLYTEK安全实验室）； Anhui SparkShield Intelligent Technology Co., Ltd.（安徽SparkShield智能科技有限公司）； Peking University（北京大学）； School of Automation, University of Electronic Science and Technology of China（电子科技大学自动化学院）； Northwest University（西北大学）； University of New South Wales（新南威尔士大学）； National Computer Network Emergency Response Technical Team/Coordination Center of China（中国国家计算机网络应急技术处置协调中心）

专题命中越狱攻击：alignment(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

2407.01599 2025-11-11 cs.CL cs.CR cs.CV cs.LG 73%

JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models

Haibo Jin, Leyang Hu, Xinnuo Li, Peiyan Zhang, Chonghan Chen, Jun Zhuang, Haohan Wang

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）； Brown University（布朗大学）； University of Michigan Ann Arbor（密歇根大学安娜堡分校）； Hong Kong University of Science and Technology（香港科学与技术大学）； Carnegie Mellon University（卡内基梅隆大学）； Boise State University（博伊西州立大学）

专题命中越狱攻击：alignment(abstract)；jailbreak(abstract)；分类 cs.CL、cs.LG

Comments 45 pages

2510.27172 2025-11-03 cs.LG cs.AI 73%

Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler

Zixuan Hu, Li Shen, Zhenyi Wang, Yongxian Wei, Dacheng Tao

机构 * Nanyang Technological University（南洋理工大学）； Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）； University of Central Florida（佛罗里达大学）； Tsinghua University（清华大学）

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.AI、cs.LG

2510.27062 2025-11-03 cs.LG cs.AI 73%

Consistency Training Helps Stop Sycophancy and Jailbreaks

Alex Irpan, Alexander Matt Turner, Mark Kurzeja, David K. Elson, Rohin Shah

机构 * Google（谷歌）

专题命中越狱攻击：alignment(abstract)；jailbreak(abstract)；分类 cs.AI、cs.LG

Comments 19 pages

2510.26981 2025-11-03 cs.LG cs.AI 73%

Fine-Grained Iterative Adversarial Attacks with Limited Computation Budget

Zhichao Hou, Weizhi Gao, Xiaorui Liu

机构 * North Carolina State University（北卡罗来纳州立大学）

专题命中越狱攻击：safety(abstract)；AI safety(abstract)；分类 cs.AI、cs.LG

2510.26847 2025-11-03 cs.CR cs.AI cs.CL cs.IT math.IT 73%

Broken-Token: Filtering Obfuscated Prompts by Counting Characters-Per-Token

Shaked Zychlinski, Yuval Kainan

机构 * JFrog

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 9 figures

2505.16947 2025-10-29 cs.LG cs.AI 73%

MixAT: Combining Continuous and Discrete Adversarial Training for LLMs

Csaba Dékány, Stefan Balauca, Robin Staab, Dimitar I. Dimitrov, Martin Vechev

机构 * INSAIT, Sofia University "St. Kliment Ohridski"（INSAIT，索菲亚大学"圣克莱孟·奥赫里迪斯基"）； ETH Zurich（苏黎世联邦理工学院）； ELTE Eötvös Loránd University, Budapest, Hungary（ELTE 爱斯特霍特·洛兰德大学，布达佩斯，匈牙利）

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.AI、cs.LG

Comments Published at 39th Conference on Neural Information Processing Systems (NeurIPS 2025)

2505.13763 2025-10-27 cs.AI cs.CL q-bio.NC 73%

Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations

Li Ji-An, Hua-Dong Xiong, Robert C. Wilson, Marcelo G. Mattar, Marcus K. Benna

机构 * Neurosciences Graduate Program University of California San Diego（加州大学圣地亚哥分校神经科学研究生项目）； School of Psychology Georgia Tech（佐治亚理工学院心理学系）； Department of Psychology New York University（纽约大学心理学系）； Department of Neurobiology University of California San Diego（加州大学圣地亚哥分校神经生物学系）

专题命中越狱攻击：safety(abstract)；AI safety(abstract)；分类 cs.CL、cs.AI

2510.17000 2025-10-21 cs.CR cs.CL cs.LG 73%

Bits Leaked per Query: Information-Theoretic Bounds on Adversarial Attacks against LLMs

Masahiro Kaneko, Timothy Baldwin

机构 * MBZUAI Abu Dhabi, UAE（阿布扎赫德MBZUAI）

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2025 (spotlight)

2510.05025 2025-10-07 cs.CL cs.AI cs.CR 73%

Imperceptible Jailbreaking against Large Language Models

Kuofeng Gao, Yiming Li, Chao Du, Xin Wang, Xingjun Ma, Shu-Tao Xia, Tianyu Pang

机构 * Tsinghua University（清华大学）； Sea AI Lab, Singapore（新加坡Sea AI实验室）； Nanyang Technological University（南洋理工大学）； Fudan University（复旦大学）； Peng Cheng Laboratory（鹏城实验室）

专题命中越狱攻击：jailbreak(abstract)；prompt injection(abstract)；分类 cs.CL、cs.AI

2510.01494 2025-10-06 cs.LG cs.AI 73%

Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed

Isha Gupta, Rylan Schaeffer, Joshua Kazdan, Ken Ziyu Liu, Sanmi Koyejo

机构 * ETH Zürich（苏黎世联邦理工学院）； Stanford CS（斯坦福大学计算机科学系）

专题命中越狱攻击：alignment(abstract)；jailbreak(abstract)；分类 cs.AI、cs.LG

2509.21634 2025-10-06 cs.CR cs.AI cs.LG cs.NI 73%

MobiLLM: An Agentic AI Framework for Closed-Loop Threat Mitigation in 6G Open RANs

Prakhar Sharma, Haohuang Wen, Vinod Yegneswaran, Ashish Gehani, Phillip Porras, Zhiqiang Lin

机构 * SRI ； The Ohio State University（俄亥俄州立大学）

专题命中越狱攻击：safety(abstract)；trustworthy(abstract)；分类 cs.AI、cs.LG

2509.16530 2025-09-23 cs.CL cs.AI 73%

AIPsychoBench: Understanding the Psychometric Differences between LLMs and Humans

Wei Xie, Shuoyoucheng Ma, Zhenhua Wang, Enze Wang, Kai Chen, Xiaobing Sun, Baosheng Wang

机构 * College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机科学与技术学院）； Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR)（科学研究院高性能计算研究所）； Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）

专题命中越狱攻击：alignment(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments Thank you for your attention. This paper was accepted by the CogSci 2025 conference in April and published in August. The location in the proceedings is: https://escholarship.org/uc/item/39k8f46q

2509.12937 2025-09-17 cs.CR cs.AI cs.CL 73%

Jailbreaking Large Language Models Through Content Concretization

Johan Wahréus, Ahmed Hussain, Panos Papadimitratos

机构 * Networked Systems Security (NSS) Group（网络系统安全组）； KTH Royal Institute of Technology（皇家理工学院）

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments Accepted for presentation in the Conference on Game Theory and AI for Security (GameSec) 2025

2505.11835 2025-09-16 cs.CL cs.AI 73%

Multilingual Collaborative Defense for Large Language Models

Hongliang Li, Jinan Xu, Gengping Cui, Changhao Guan, Fengran Mo, Kaiyu Huang

机构 * Key Laboratory of Big Data & Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education（大数据与人工智能交通运输 key laboratory（北京交通大学））； School of Computer Science and Technology, Beijing Jiaotong University（计算机科学与技术学院（北京交通大学））； University of Montreal（蒙特利尔大学）

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments 21 pages, 4figures

2508.09288 2025-08-20 cs.CR cs.AI cs.CL 73%

Can AI Keep a Secret? Contextual Integrity Verification: A Provable Security Architecture for LLMs

Aayush Gupta

机构 * Aayush Gupta（独立研究者）

专题命中越狱攻击：jailbreak(abstract)；prompt injection(abstract)；分类 cs.CL、cs.AI

Comments 2 figures, 3 tables; code and certification harness: https://github.com/ayushgupta4897/Contextual-Integrity-Verification ; Elite-Attack dataset: https://huggingface.co/datasets/zyushg/elite-attack

2508.06755 2025-08-12 cs.CL cs.AI 73%

Many-Turn Jailbreaking

Xianjun Yang, Liqiang Xiao, Shiyang Li, Faisal Ladhak, Hyokun Yun, Linda Ruth Petzold, Yi Xu, William Yang Wang

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

2507.02990 2025-07-08 cs.CL cs.AI 73%

`For Argument's Sake, Show Me How to Harm Myself!': Jailbreaking LLMs in Suicide and Self-Harm Contexts

Annika M Schoene, Cansu Canca

机构 * Institute for Experiential AI（经验式人工智能研究所）； Northeastern University（东北大学）

专题命中越狱攻击：safety(abstract)；AI safety(abstract)；分类 cs.CL、cs.AI

2408.00523 2025-06-26 cs.CR cs.AI cs.LG 73%

Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models

Yingkai Dong, Xiangtao Meng, Ning Yu, Zheng Li, Shanqing Guo

机构 * School of Cyber Science and Technology, Shandong University（山东大学网络科学与技术学院）； Netflix Eyeline Studios ； State Key Laboratory of Cryptography and Digital Economy Security, Shandong University（山东大学密码与数字经济安全国家重点实验室）； Shandong Key Laboratory of Artificial Intelligence Security, Shandong University（山东省人工智能安全重点实验室）

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI、cs.LG

2501.18626 2025-06-03 cs.CR cs.AI cs.CL 73%

The TIP of the Iceberg: Revealing a Hidden Class of Task-in-Prompt Adversarial Attacks on LLMs

Sergey Berezin, Reza Farahbakhsh, Noel Crespi

机构 * SAMOVAR, Télécom SudParis Institut Polytechnique de Paris（SAMOVAR，电信南巴黎高等理工学院）

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments Accepted to the Main Track of ACL 2025

2505.24428 2025-06-02 cs.CL cs.LG 73%

Model Unlearning via Sparse Autoencoder Subspace Guided Projections

Xu Wang, Zihao Li, Benyou Wang, Yan Hu, Difan Zou

机构 * The University of Hong Kong（香港大学）； New Jersey Institute of Technology（新泽西理工学院）； The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.LG

2311.09948 2025-05-30 cs.LG cs.CL cs.CR 73%

Hijacking Large Language Models via Adversarial In-Context Learning

Xiangyu Zhou, Yao Qiang, Saleh Zare Zade, Prashant Khanduri, Dongxiao Zhu

机构 * Wayne State University（韦恩州立大学）； Oakland University（奥克兰大学）

专题命中越狱攻击：jailbreak(abstract)；prompt injection(abstract)；分类 cs.CL、cs.LG

2505.07610 2025-05-20 cs.CL cs.AI 73%

Concept-Level Explainability for Auditing & Steering LLM Responses

Kenza Amara, Rita Sevastjanova, Mennatallah El-Assady

机构 * Department of Computer Science ETH Zurich（计算机科学系苏黎世联邦理工学院）

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 7 figures, Submission to Neurips 2025

2409.01586 2025-03-18 cs.CL cs.AI 73%

Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation

Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, Ling Liu

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.CL、cs.AI

2503.01865 2025-03-05 cs.LG cs.AI cs.CR 73%

Guiding not Forcing: Enhancing the Transferability of Jailbreaking Attacks on LLMs via Removing Superfluous Constraints

Junxiao Yang, Zhexin Zhang, Shiyao Cui, Hongning Wang, Minlie Huang

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI、cs.LG

2502.14529 2025-02-21 cs.CL cs.AI 73%

CORBA: Contagious Recursive Blocking Attacks on Multi-Agent Systems Based on Large Language Models

Zhenhong Zhou, Zherui Li, Jie Zhang, Yuanhe Zhang, Kun Wang, Yang Liu, Qing Guo

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.CL、cs.AI

2408.06223 2025-02-07 cs.CL cs.AI 73%

On Effects of Steering Latent Representation for Large Language Model Unlearning

Dang Huu-Tien, Trung-Tin Pham, Hoang Thanh-Tung, Naoya Inoue

专题命中越狱攻击：alignment(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments Accepted at AAAI-25 Main Technical Track

2412.15275 2024-12-23 cs.CR cs.AI cs.CL 73%

Fooling LLM graders into giving better grades through neural activity guided adversarial prompting

Atsushi Yamamura, Surya Ganguli

专题命中越狱攻击：safety(abstract)；AI safety(abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 11 figures

2405.13077 2024-10-17 cs.CR cs.AI cs.CL 73%

GPT-4 Jailbreaks Itself with Near-Perfect Success Using Self-Explanation

Govind Ramesh, Yao Dou, Wei Xu

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2024 Main Conference

1. 越狱攻击 1730 篇

Proactive Defense: Compound AI for Detecting Persuasion Attacks and Measuring Inoculation Effectiveness

ForgeDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models

JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models

Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler

Consistency Training Helps Stop Sycophancy and Jailbreaks

Fine-Grained Iterative Adversarial Attacks with Limited Computation Budget

Broken-Token: Filtering Obfuscated Prompts by Counting Characters-Per-Token

MixAT: Combining Continuous and Discrete Adversarial Training for LLMs

Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations

Bits Leaked per Query: Information-Theoretic Bounds on Adversarial Attacks against LLMs

Imperceptible Jailbreaking against Large Language Models

Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed

MobiLLM: An Agentic AI Framework for Closed-Loop Threat Mitigation in 6G Open RANs

AIPsychoBench: Understanding the Psychometric Differences between LLMs and Humans

Jailbreaking Large Language Models Through Content Concretization

Multilingual Collaborative Defense for Large Language Models

Can AI Keep a Secret? Contextual Integrity Verification: A Provable Security Architecture for LLMs

Many-Turn Jailbreaking

`For Argument's Sake, Show Me How to Harm Myself!': Jailbreaking LLMs in Suicide and Self-Harm Contexts

Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models

The TIP of the Iceberg: Revealing a Hidden Class of Task-in-Prompt Adversarial Attacks on LLMs

Model Unlearning via Sparse Autoencoder Subspace Guided Projections

Hijacking Large Language Models via Adversarial In-Context Learning

Concept-Level Explainability for Auditing & Steering LLM Responses

Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation

Guiding not Forcing: Enhancing the Transferability of Jailbreaking Attacks on LLMs via Removing Superfluous Constraints

CORBA: Contagious Recursive Blocking Attacks on Multi-Agent Systems Based on Large Language Models

On Effects of Steering Latent Representation for Large Language Model Unlearning

Fooling LLM graders into giving better grades through neural activity guided adversarial prompting

GPT-4 Jailbreaks Itself with Near-Perfect Success Using Self-Explanation