大模型对齐与安全 - arXivDaily 专题

2402.12329 2024-12-10 cs.CL cs.AI cs.CR cs.LG 67%

Query-Based Adversarial Prompt Generation

Jonathan Hayase, Ema Borevkovic, Nicholas Carlini, Florian Tramèr, Milad Nasr

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

2410.09804 2024-11-28 cs.CR cs.AI cs.CL cs.LG cs.NE 67%

BlackDAN: A Black-Box Multi-Objective Approach for Effective and Contextual Jailbreaking of Large Language Models

Xinyuan Wang, Victor Shea-Jay Huang, Renmiao Chen, Hao Wang, Chengwei Pan, Lei Sha, Minlie Huang

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

2401.17263 2024-11-11 cs.LG cs.AI cs.CL cs.CV 67%

Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks

Andy Zhou, Bo Li, Haohan Wang

专题命中越狱攻击：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2024 Spotlight; code available at https://github.com/lapisrocks/rpo

2312.02119 2024-11-01 cs.LG cs.AI cs.CL cs.CR stat.ML 67%

Tree of Attacks: Jailbreaking Black-Box LLMs Automatically

Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, Amin Karbasi

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted for presentation at NeurIPS 2024. Code: https://github.com/RICommunity/TAP

2406.01288 2024-10-31 cs.CL cs.AI cs.CR cs.LG 67%

Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses

Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Jing Jiang, Min Lin

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2024

2402.17840 2024-10-08 cs.CL cs.AI cs.CR cs.LG 67%

Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems

Zhenting Qi, Hanlin Zhang, Eric Xing, Sham Kakade, Himabindu Lakkaraju

专题命中越狱攻击：prompt injection(abstract)；分类 cs.CL、cs.AI、cs.LG

2406.20053 2024-07-01 cs.CR cs.AI cs.CL cs.LG 67%

Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation

Danny Halawi, Alexander Wei, Eric Wallace, Tony T. Wang, Nika Haghtalab, Jacob Steinhardt

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 22 pages

2406.08725 2024-06-14 cs.CR 67%

RL-JACK: Reinforcement Learning-powered Black-box Jailbreaking Attack against LLMs

Xuan Chen, Yuzhou Nie, Lu Yan, Yunshu Mao, Wenbo Guo, Xiangyu Zhang

专题命中越狱攻击：alignment(abstract)；safety(abstract)

2403.12075 2024-05-15 cs.CY cs.AI cs.CR cs.CV cs.LG 67%

Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generation

Jessica Quaye, Alicia Parrish, Oana Inel, Charvi Rastogi, Hannah Rose Kirk, Minsuk Kahng, Erin van Liemt, Max Bartolo, Jess Tsang, Justin White, Nathan Clement, Rafael Mosquera, Juan Ciro, Vijay Janapa Reddi, Lora Aroyo

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.CY、cs.LG

Comments 10 pages, 6 figures

2403.13507 2024-03-22 cs.CV 67%

FMM-Attack: A Flow-based Multi-modal Adversarial Attack on Video-based LLMs

Jinmin Li, Kuofeng Gao, Yang Bai, Jingyun Zhang, Shu-tao Xia, Yisen Wang

专题命中越狱攻击：alignment(abstract)；safety(abstract)

2403.04769 2024-03-12 cs.CR cs.AI cs.CL cs.LG 67%

Using Hallucinations to Bypass GPT4's Filter

Benjamin Lemkin

专题命中越狱攻击：RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

2402.09674 2024-02-16 cs.CL cs.AI cs.CR cs.LG 67%

PAL: Proxy-Guided Black-Box Attack on Large Language Models

Chawin Sitawarin, Norman Mu, David Wagner, Alexandre Araujo

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

2311.03348 2023-11-27 cs.CL cs.AI cs.LG 67%

Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation

Rusheb Shah, Quentin Feuillade--Montixi, Soroush Pour, Arush Tagade, Stephen Casper, Javier Rando

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

2305.03495 2023-10-20 cs.CL cs.AI cs.LG 67%

Automatic Prompt Optimization with "Gradient Descent" and Beam Search

Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, Michael Zeng

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP 2023

2309.17410 2023-10-02 cs.CL cs.AI cs.LG 67%

Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks

Vaidehi Patil, Peter Hase, Mohit Bansal

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Equal contribution from first two authors. 19 pages, 5 figures. Our code is available at: https://github.com/Vaidehi99/InfoDeletionAttacks

2308.11521 2023-08-28 cs.CL cs.AI cs.LG 67%

Self-Deception: Reverse Penetrating the Semantic Firewall of Large Language Models

Zhenhua Wang, Wei Xie, Kai Chen, Baosheng Wang, Zhiwen Gui, Enze Wang

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Serious errors were found in the experiment, which may lead to the overturning of the overall conclusions of the paper

2308.10443 2023-08-22 cs.AI cs.CL cs.CY 67%

Using Large Language Models for Cybersecurity Capture-The-Flag Challenges and Certification Questions

Wesley Tann, Yuancheng Liu, Jun Heng Sim, Choon Meng Seah, Ee-Chien Chang

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.CY

2305.08005 2023-05-16 cs.CR cs.AI cs.CL cs.CY cs.HC 67%

Beyond the Safeguards: Exploring the Security Risks of ChatGPT

Erik Derner, Kristina Batistič

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.CY

2606.24014 2026-06-24 cs.AI cs.CL 新提交 66%

Reinforcement Learning Towards Broadly and Persistently Beneficial Models

强化学习迈向广泛且持久有益的模型

Akshay V. Jagadeesh, Rahul K. Arora, Khaled Saab, Ali Malik, Mikhail Trofimov, Foivos Tsimpourlas, Johannes Heidecke, Karan Singhal

机构 * OpenAI

专题命中越狱攻击：alignment(abstract,comments)；分类 cs.CL、cs.AI

AI总结通过真实领域中的有益行为强化学习，模型在超过80%的分布外基准上表现提升，并展现出跨领域对齐泛化与对抗攻击的持久性。

Comments Blog: https://alignment.openai.com/beneficial-rl/

2407.20242 2026-06-10 cs.CY cs.AI cs.RO 版本更新 66%

BadRobot: Jailbreaking Embodied LLM Agents in the Physical World

BadRobot: 在物理世界中越狱具身LLM智能体

Hangtao Zhang, Chenyu Zhu, Xianlong Wang, Ziqi Zhou, Changgan Yin, Minghui Li, Lulu Xue, Yichen Wang, Shengshan Hu, Aishan Liu, Peijin Guo, Leo Yu Zhang

机构 * Huazhong University of Science and Technology（华中科技大学）； Beihang University（北航）； Griffith University（格里菲斯大学）

专题命中越狱攻击：safety(abstract,comments)；分类 cs.AI、cs.CY

AI总结提出BadRobot攻击范式，利用LLM在机器人系统中的操纵、语言输出与物理动作的错位以及世界知识缺陷三个漏洞，通过语音交互使具身LLM执行有害行为，并在基准测试中验证了有效性。

Comments Accepted to ICLR 2025. Please cite the conference version. Project page: https://Embodied-LLMs-Safety.github.io

Journal ref International Conference on Learning Representations (ICLR) 2025

2211.09527 2022-11-18 cs.CL cs.AI 66%

Ignore Previous Prompt: Attack Techniques For Language Models

Fábio Perez, Ian Ribeiro

专题命中越狱攻击：alignment(abstract)；分类 cs.CL、cs.AI；safety(comments)

Comments ML Safety Workshop NeurIPS 2022

2606.00566 2026-08-25 cs.LG cs.CL cs.CR 版本更新 62%

Same Payload, Different Channel: Measuring Trust Asymmetry in Tool-Using Language Models

相同载荷，不同通道：测量使用工具的語言模型中的信任不对称性

Mohammed Sameer Syed, Rozhin Yasaei

机构 * University of Arizona（亚利桑那大学）

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.LG

AI总结本研究提出安全不对称分数（SAS），通过匹配恶意载荷仅改变传递上下文，系统测量了语言模型在不同通道（用户消息、工具元数据、工具输出）中对对抗性内容的脆弱性差异，发现代理原生模型在工具描述通道更脆弱，而通用模型相反，且机制研究表明安全相关表示在深层网络非线性编码。

Comments Accepted to EMNLP 2026 (Main Conference)

2608.11624 2026-08-13 cs.CL cs.AI 新提交 62%

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

学习说服暴露了大语言模型（LLMs）放弃正确信念的难易程度

Nimet Beyza Bozdag, Emre Can Acikgoz, Gokhan Tur, Dilek Hakkani-Tür

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI

AI总结研究发现优化后的对抗性说服策略可轻易让LLMs放弃正确信念，攻击成功率高且可迁移，凸显多智能体决策系统需将说服鲁棒性作为安全标准

2608.10537 2026-08-12 cs.AI cs.LG 新提交 62%

Measuring Semantic Abstractness of SAE Features via Nonlocality

通过非局部性测量SAE特征的语义抽象性

Chuqiao Lin, Shivaji Sondhi, Xiao-Liang Qi

专题命中越狱攻击：jailbreak(abstract)；分类 cs.AI、cs.LG

AI总结该研究提出特征非局部性（FNL）指标，可区分SAE特征的抽象层级，用于审计越狱缓解特征及引导DeepSeek-R1模型特征提升数学推理准确率。

Comments 18 pages, 9 figures

2608.06779 2026-08-10 q-bio.QM cs.AI cs.CL 新提交 62%

Genotypic Triggers: Exposing Pharmacogenomic Blind Spots via Host-Specific Backdoors in Generative Antimicrobial Peptide Models

基因型触发器：通过生成抗菌肽模型中宿主特定后门暴露药物基因组学盲区

Doniyorkhon Obidov, Xiaolong Guo, Yonghui Li, Kaichen Yang

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI

AI总结该研究提出基因型触发器后门攻击，操控抗菌肽生成模型使其针对特定HLA等位基因携带者生成高免疫原性风险肽，同时保留抗菌效力与低毒性，以暴露药物基因组学盲区。

2608.03070 2026-08-07 cs.CR cs.AI cs.CL 版本更新 62%

AI Security Leaderboard: Methodology, Results and Minimal Standard

AI安全排行榜：方法、结果与最低标准

Jasper Timm, Lukas Struppek, Ziwei Xu, Grace Cheong, Oscar Mata, Dan Zhao, Mick Yang, Isadora De Andrade, Xiaojun Jia, Yiming Li, Samuel Bauer, Heather McIntyre, Adam Gleave, Edward Yee, Kellin Pelrine

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI

AI总结本研究推出AI安全防护最低标准V1.0，测试Claude Fable5等四款模型的越狱鲁棒性，发现模型间防御能力差异大，部分模型未出现通用越狱，可通过现有技术缩小防御差距。

2503.15560 2026-08-06 cs.CR cs.AI cs.LG 交叉投稿 62%

Temporal Context Awareness: A Defense Framework Against Multi-turn Manipulation Attacks on Large Language Models

时序上下文感知：一种针对大型语言模型多轮操纵攻击的防御框架

Prashant Kulkarni, Assaf Namer

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

AI总结针对LLMs易受多轮操纵攻击的问题，本文提出TCA框架，通过分析语义漂移等实现防御，初步评估显示其能识别传统方法遗漏的微妙操纵模式，为对话式AI系统提供安全保障。

Comments 6 pages, 2 figures, IEEE CAI

2604.10933 2026-08-04 cs.CR cs.AI cs.CV cs.LG quant-ph 版本更新 62%

QShield: Securing Neural Networks Against Adversarial Attacks using Quantum Circuits

QShield: 通过量子电路安全神经网络对抗对抗性攻击

Navid Azimi, Aditya Prakash, Yao Wang, Li Xiong

机构 * Emory University（埃默里大学）

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

AI总结 QShield通过结合传统CNN和量子处理模块，提升经典深度学习模型的对抗鲁棒性，实验表明其在多种数据集上有效降低攻击成功率并提高预测准确性。

2607.25479 2026-07-29 cs.CR cs.AI cs.LG 新提交 62%

Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering

通过表示引导在视觉语言模型供应链中植入架构后门

Maria Rosaria Briglia, Igor Maljkovic, Antonio Emanuele Cinà, Luca Oneto, Iacopo Masi, Fabio Roli

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

AI总结研究视觉语言模型供应链安全问题，提出通过表示引导植入架构后门的攻击方法，该方法不影响训练数据等，通过触发控制模型表示转向攻击者目标，评估表明其损害模型多项性能，还提出审计防御方法。

2607.24339 2026-07-28 cs.AI cs.CL 新提交 62%

Gubernaut: A Deterministic Homeostatic Controller for Affect-Regulated LLM Agents, Validated Across Independent Model Families

Gubernaut：用于情感调节的大语言模型代理的确定性稳态控制器，在独立模型家族中得到验证

Dushyant Sharma

机构 * Gubernaut Research（Gubernaut研究）

专题命中越狱攻击：alignment(abstract)；分类 cs.CL、cs.AI

AI总结研究针对大语言模型代理的故障模式，提出Gubernaut认知控制器，通过在四个前沿模型上预注册的评估协议验证，该控制器能使模型更平静，有恢复特征等机制，附带可重判记录及预注册故障模式。

Comments 26 pages, 7 figures, 3 tables. Data, transcripts, and analysis scripts: https://github.com/thegubernaut/Gubernaut_Validation Project page: https://gubernaut.com (archived at https://doi.org/10.5281/zenodo.21303518)

1. 越狱攻击 1730 篇

Query-Based Adversarial Prompt Generation

BlackDAN: A Black-Box Multi-Objective Approach for Effective and Contextual Jailbreaking of Large Language Models

Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks

Tree of Attacks: Jailbreaking Black-Box LLMs Automatically

Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses

Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems

Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation

RL-JACK: Reinforcement Learning-powered Black-box Jailbreaking Attack against LLMs

Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generation

FMM-Attack: A Flow-based Multi-modal Adversarial Attack on Video-based LLMs

Using Hallucinations to Bypass GPT4's Filter

PAL: Proxy-Guided Black-Box Attack on Large Language Models

Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation

Automatic Prompt Optimization with "Gradient Descent" and Beam Search

Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks

Self-Deception: Reverse Penetrating the Semantic Firewall of Large Language Models

Using Large Language Models for Cybersecurity Capture-The-Flag Challenges and Certification Questions

Beyond the Safeguards: Exploring the Security Risks of ChatGPT

Reinforcement Learning Towards Broadly and Persistently Beneficial Models

BadRobot: Jailbreaking Embodied LLM Agents in the Physical World

Ignore Previous Prompt: Attack Techniques For Language Models

Same Payload, Different Channel: Measuring Trust Asymmetry in Tool-Using Language Models

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

Measuring Semantic Abstractness of SAE Features via Nonlocality

Genotypic Triggers: Exposing Pharmacogenomic Blind Spots via Host-Specific Backdoors in Generative Antimicrobial Peptide Models

AI Security Leaderboard: Methodology, Results and Minimal Standard

Temporal Context Awareness: A Defense Framework Against Multi-turn Manipulation Attacks on Large Language Models

QShield: Securing Neural Networks Against Adversarial Attacks using Quantum Circuits

Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering

Gubernaut: A Deterministic Homeostatic Controller for Affect-Regulated LLM Agents, Validated Across Independent Model Families