大模型对齐与安全 - arXivDaily 专题

2405.09113 2025-02-13 cs.LG 79%

Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization

Kai Hu, Weichen Yu, Yining Li, Kai Chen, Tianjun Yao, Xiang Li, Wenhe Liu, Lijun Yu, Zhiqiang Shen, Matt Fredrikson

专题命中越狱攻击：jailbreak(title,abstract)；分类 cs.LG

Journal ref 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

2502.02438 2025-02-05 cs.CR cs.AI 79%

Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment

Yaling Shen, Zhixiong Zhuang, Kun Yuan, Maria-Irina Nicolae, Nassir Navab, Nicolas Padoy, Mario Fritz

专题命中越狱攻击：alignment(title,abstract)；分类 cs.AI

Comments Accepted at AAAI 2025

2412.17522 2025-01-07 cs.CL 79%

DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak

Hao Wang, Hao Li, Junda Zhu, Xinyuan Wang, Chengwei Pan, MinLie Huang, Lei Sha

专题命中越狱攻击：jailbreak(title,abstract)；分类 cs.CL

2412.05346 2024-12-10 cs.CR cs.LG 79%

BadGPT-4o: stripping safety finetuning from GPT models

Ekaterina Krupkina, Dmitrii Volkov

专题命中越狱攻击：safety(title,abstract)；分类 cs.LG

2406.10794 2024-12-04 cs.CL 79%

Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis

Yuping Lin, Pengfei He, Han Xu, Yue Xing, Makoto Yamada, Hui Liu, Jiliang Tang

专题命中越狱攻击：jailbreak(title,abstract)；分类 cs.CL

Comments Accepted by EMNLP 2024 Main

2410.22143 2024-10-30 cs.CL 79%

AmpleGCG-Plus: A Strong Generative Model of Adversarial Suffixes to Jailbreak LLMs with Higher Success Rates in Fewer Attempts

Vishal Kumar, Zeyi Liao, Jaylen Jones, Huan Sun

专题命中越狱攻击：jailbreak(title)；alignment(abstract)；分类 cs.CL

2410.03489 2024-10-24 cs.CR cs.AI 79%

Gradient-based Jailbreak Images for Multimodal Fusion Models

Javier Rando, Hannah Korevaar, Erik Brinkman, Ivan Evtimov, Florian Tramèr

专题命中越狱攻击：jailbreak(title,abstract)；分类 cs.AI

2410.11459 2024-10-16 cs.CL 79%

Jigsaw Puzzles: Splitting Harmful Questions to Jailbreak Large Language Models

Hao Yang, Lizhen Qu, Ehsan Shareghi, Gholamreza Haffari

专题命中越狱攻击：jailbreak(title,abstract)；分类 cs.CL

2403.17336 2024-10-02 cs.CR cs.CL 79%

Don't Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Models

Zhiyuan Yu, Xiaogeng Liu, Shunning Liang, Zach Cameron, Chaowei Xiao, Ning Zhang

专题命中越狱攻击：jailbreak(title,abstract)；分类 cs.CL

Comments Accepted by USENIX Security 2024

2408.03515 2024-09-10 cs.RO cs.AI 79%

A Study on Prompt Injection Attack Against LLM-Integrated Mobile Robotic Systems

Wenxiao Zhang, Xiangrui Kong, Conan Dewitt, Thomas Braunl, Jin B. Hong

专题命中越狱攻击：prompt injection(title,abstract)；分类 cs.AI

2409.01630 2024-09-04 cs.RO cs.AI cs.ET 79%

SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systems

Wenxiao Zhang, Xiangrui Kong, Thomas Braunl, Jin B. Hong

专题命中越狱攻击：safety(title,abstract)；分类 cs.AI

2402.09091 2024-08-22 cs.CR cs.AI cs.HC 79%

Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues

Zhiyuan Chang, Mingyang Li, Yi Liu, Junjie Wang, Qing Wang, Yang Liu

专题命中越狱攻击：jailbreak(title,abstract)；分类 cs.AI

Comments 13 pages, 6 figures

Journal ref The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)

2406.17104 2024-06-26 cs.CL 79%

Automated Adversarial Discovery for Safety Classifiers

Yash Kumar Lal, Preethi Lahoti, Aradhana Sinha, Yao Qin, Ananth Balashankar

专题命中越狱攻击：safety(title,abstract)；分类 cs.CL

Comments Published at Fourth Workshop on TrustworthyNLP (TrustNLP) at NAACL 2024

2406.12702 2024-06-24 cs.CL 79%

[WIP] Jailbreak Paradox: The Achilles' Heel of LLMs

Abhinav Rao, Monojit Choudhury, Somak Aditya

专题命中越狱攻击：jailbreak(title,abstract)；分类 cs.CL

2402.16006 2024-06-05 cs.CL 79%

ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings

Hao Wang, Hao Li, Minlie Huang, Lei Sha

专题命中越狱攻击：jailbreak(title)；safety(abstract)；分类 cs.CL

2405.19103 2024-05-30 cs.CR cs.LG 79%

Voice Jailbreak Attacks Against GPT-4o

Xinyue Shen, Yixin Wu, Michael Backes, Yang Zhang

专题命中越狱攻击：jailbreak(title,abstract)；分类 cs.LG

2402.13457 2024-05-20 cs.CR cs.AI 79%

A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models

Zihao Xu, Yi Liu, Gelei Deng, Yuekang Li, Stjepan Picek

专题命中越狱攻击：jailbreak(title)；safety(abstract)；分类 cs.AI

Comments 18 pages, 9 figures, Accepted in ACL 2024

2404.04849 2024-04-18 cs.CR cs.AI 79%

Hidden You Malicious Goal Into Benign Narratives: Jailbreak Large Language Models through Logic Chain Injection

Zhilong Wang, Yebo Cao, Peng Liu

专题命中越狱攻击：jailbreak(title,abstract)；分类 cs.AI

2311.08268 2024-04-09 cs.CL 79%

A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily

Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, Shujian Huang

专题命中越狱攻击：jailbreak(title,abstract)；分类 cs.CL

Comments Acccepted by NAACL 2024, 18 pages, 7 figures, 13 tables

2402.14494 2024-03-07 cs.CL 79%

Noise-BERT: A Unified Perturbation-Robust Framework with Noise Alignment Pre-training for Noisy Slot Filling Task

Jinxu Zhao, Guanting Dong, Yueyan Qiu, Tingfeng Hui, Xiaoshuai Song, Daichi Guo, Weiran Xu

专题命中越狱攻击：alignment(title,abstract)；分类 cs.CL

Comments Accepted by ICASSP 2024

2311.11855 2024-02-05 cs.CL 79%

Evil Geniuses: Delving into the Safety of LLM-based Agents

Yu Tian, Xiao Yang, Jingyuan Zhang, Yinpeng Dong, Hang Su

专题命中越狱攻击：safety(title,abstract)；分类 cs.CL

Comments 11 pages

2309.11830 2023-12-11 cs.CL 79%

Goal-Oriented Prompt Attack and Safety Evaluation for LLMs

Chengyuan Liu, Fubang Zhao, Lizhi Qing, Yangyang Kang, Changlong Sun, Kun Kuang, Fei Wu

专题命中越狱攻击：safety(title,abstract)；分类 cs.CL

2511.07107 2026-08-04 cs.AI cs.CL 版本更新 79%

MENTOR: A Metacognition-Driven Self-Evolution Framework for Uncovering and Mitigating Implicit Domain Risks in LLMs

MENTOR: 一种元认知驱动的自我进化框架，用于发现和缓解大语言模型中的隐式领域风险

Liang Shan, Kaicheng Shen, Wen Wu, Zhenyu Ying, Chaochao Lu, Yan Teng, Jingqi Huang, Qingshan Liu, Guangze Ye, Guoqing Wang, Jie Zhou, Liang He

机构 * School of Computer Science and Technology, East China Normal University（东华大学计算机科学与技术学院）； Shanghai AI Lab, Shanghai Innovation Institute（上海人工智能实验室，上海创新研究院）

专题命中越狱攻击：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

AI总结针对大语言模型在特定领域（如教育、金融、管理）中存在的隐式安全风险，提出基于元认知自我评估和动态规则知识图谱的MENTOR框架，通过激活级引导信号有效降低攻击成功率。

2607.21619 2026-07-27 cs.CL cs.AI 新提交 79%

Adversarial Style Optimization: Enhancing VLM Jailbreaks by GRPO-based Stylistic Triggers Optimization

对抗风格优化：通过基于GRPO的风格触发优化增强VLM越狱

Bingjun Luo, Jialin Guo, Yue Yao, Xinpeng Ding

机构 * Tsinghua University（清华大学）； Harbin Engineering University（哈尔滨工程大学）； Shandong University（山东大学）； Xidian University（西安电子科技大学）

专题命中越狱攻击：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

AI总结研究MLLMs安全对齐易受越狱攻击问题，提出基于GRPO的对抗风格优化（ASO）方法，通过优化风格触发增强视觉越狱，实验证明该方法显著提高攻击成功率，凸显风格偏差对MLLMs红队测试的作用。

Comments Accepted by CVPR 2026 (Oral)

2605.09504 2026-07-24 cs.CR cs.AI cs.LG 版本更新 79%

AI Security Policy Should Assess Systems, Not Only Models

位置：AI安全政策应针对系统，而非模型

Michael A. Riegler, Inga Strümke

机构 * AI Safety Department（人工智能安全部门）； SimulaMet and OsloMet（SimulaMet和奥斯陆计量）

专题命中越狱攻击：jailbreak(abstract,abstract_cn)；safety(abstract)；分类 cs.AI、cs.LG

AI总结本文提出swarm-attack框架，展示通过协调轻量级LLM代理发现系统漏洞和绕过安全机制的可能性，证明在低成本硬件上可实现零成本安全测试。

2603.19423 2026-06-19 cs.CR cs.AI cs.LG 版本更新 79%

The Autonomy Tax: Defense Training Breaks LLM Agents

自主性税：防御训练破坏LLM智能体

Shawn Li, Yue Zhao

机构 * University of Southern California（南加州大学）

专题命中越狱攻击：alignment(abstract)；safety(abstract)；prompt injection(abstract)；分类 cs.AI、cs.LG

AI总结揭示防御训练在提升LLM智能体安全性时，系统性地破坏其工具执行能力，导致任务失败率飙升，且无法有效防御复杂攻击。

2606.02995 2026-06-15 cs.CR cs.AI cs.IR cs.LG 版本更新 79%

Patcher: Post-Hoc Patching of Backdoored Large Language Models

Patcher: 后门大型语言模型的事后修补

Anjun Gao, Yueyang Quan, Yufei Xia, Zhuqing Liu, Minghong Fang

机构 * University of Louisville（路易斯维尔大学）； University of North Texas（北得克萨斯大学）

专题命中越狱攻击：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.AI、cs.LG

AI总结提出Patcher框架，仅利用单个失败案例和模型参数，通过基于梯度的显著性定位后门触发器，并采用约束微调消除触发-响应关联，同时保持模型效用。

Comments To appear in the USENIX Security Symposium, 2026

2606.02111 2026-06-02 cs.CV cs.AI cs.CL 79%

Jailbreaking Multimodal Large Language Models using Multi-Clip Video

使用多片段视频破解多模态大语言模型

Choongwon Kang, Seungjong Sun, Hyunmin Jun, Jang Hyun Kim

机构 * Department of Applied Artificial Intelligence, Sungkyunkwan University（应用人工智能系，成均馆大学）； Department of Human-Artificial Intelligence Interaction, Sungkyunkwan University（人机交互系，成均馆大学）

专题命中越狱攻击：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

AI总结提出MCV SafetyBench数据集，通过多片段视频评估多模态大语言模型的安全漏洞，发现视频模态比图像更脆弱，动态和多样化上下文增加攻击成功率，并基于图像模态的鲁棒性提出防御策略。

Comments 27 pages, 20 figures, Accepted to the Main Conference of ACL 2026

2602.22291 2026-03-02 cs.LG cs.AI cs.CR 79%

Manifold of Failure: Behavioral Attraction Basins in Language Models

失败的流形：语言模型中的行为吸引盆地

Sarthak Munshi, Manish Bhatt, Vineeth Sai Narajala, Idan Habler, Ammar Al-Kahfah, Ken Huang, Blake Gatto

机构 * Amazon Web Services（亚马逊网络服务）； Cisco（思科）； Shrewd Security（精明安全）

专题命中越狱攻击：alignment(abstract)；safety(abstract)；AI safety(abstract)；分类 cs.AI、cs.LG

AI总结本文提出了一种基于MAP-Elites的方法，用于系统地映射语言模型的失败流形，揭示不同模型的拓扑结构和漏洞分布。

2507.23465 2026-02-24 cs.CL cs.AI 79%

Role-Aware Language Models for Secure and Contextualized Access Control in Organizations

面向角色的语言模型：用于组织中的安全且上下文化的访问控制

Saeed Almheiri, Yerulan Kongrat, Adrian Santosh, Ruslan Tasmukhanov, Josemaria Loza Vera, Muhammad Dehan Al Kautsar, Fajri Koto

机构 * Mohamed bin Zayed University of Artificial Intelligence（莫扎德·本·扎耶德人工智能大学）； Nazarbayev University（纳扎尔拜耶夫大学）； University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）； New York University Abu Dhabi（纽约大学阿布扎克分校）

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；prompt injection(abstract)；分类 cs.CL、cs.AI

AI总结本文提出面向角色的语言模型，通过三种策略实现基于组织角色的安全访问控制，并通过实验验证其在不同组织结构下的性能和鲁棒性。

Comments AACL 2025 - Main

1. 越狱攻击 1730 篇

Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization

Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment

DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak

BadGPT-4o: stripping safety finetuning from GPT models

Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis

AmpleGCG-Plus: A Strong Generative Model of Adversarial Suffixes to Jailbreak LLMs with Higher Success Rates in Fewer Attempts

Gradient-based Jailbreak Images for Multimodal Fusion Models

Jigsaw Puzzles: Splitting Harmful Questions to Jailbreak Large Language Models

Don't Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Models

A Study on Prompt Injection Attack Against LLM-Integrated Mobile Robotic Systems

SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systems

Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues

Automated Adversarial Discovery for Safety Classifiers

[WIP] Jailbreak Paradox: The Achilles' Heel of LLMs

ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings

Voice Jailbreak Attacks Against GPT-4o

A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models

Hidden You Malicious Goal Into Benign Narratives: Jailbreak Large Language Models through Logic Chain Injection

A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily

Noise-BERT: A Unified Perturbation-Robust Framework with Noise Alignment Pre-training for Noisy Slot Filling Task

Evil Geniuses: Delving into the Safety of LLM-based Agents

Goal-Oriented Prompt Attack and Safety Evaluation for LLMs

MENTOR: A Metacognition-Driven Self-Evolution Framework for Uncovering and Mitigating Implicit Domain Risks in LLMs

Adversarial Style Optimization: Enhancing VLM Jailbreaks by GRPO-based Stylistic Triggers Optimization

AI Security Policy Should Assess Systems, Not Only Models

The Autonomy Tax: Defense Training Breaks LLM Agents

Patcher: Post-Hoc Patching of Backdoored Large Language Models

Jailbreaking Multimodal Large Language Models using Multi-Clip Video

Manifold of Failure: Behavioral Attraction Basins in Language Models

Role-Aware Language Models for Secure and Contextualized Access Control in Organizations