大模型对齐与安全 - arXivDaily 专题

2410.22663 2025-05-27 cs.SE cs.CL cs.CR 70%

Automated Trustworthiness Oracle Generation for Machine Learning Text Classifiers

Lam Nguyen Tung, Steven Cho, Xiaoning Du, Neelofar Neelofar, Valerio Terragni, Stefano Ruberto, Aldeida Aleti

机构 * Monash University（莫纳什大学）； University of Auckland（奥克兰大学）； Royal Melbourne Institute of Technology（皇家墨尔本理工学院）； Joint Research Centre at the European Commission（欧洲委员会联合研究中心）

专题命中越狱攻击：alignment(abstract)；trustworthy(abstract)；分类 cs.CL

Comments 24 pages, 5 tables, 9 figures, Camera-ready version accepted to FSE 2025

2505.16559 2025-05-23 cs.CR cs.CL 70%

CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuning

Biao Yi, Tiansheng Huang, Baolei Zhang, Tong Li, Lihai Nie, Zheli Liu, Li Shen

机构 * College of Cyber Science, Nankai University（南开大学网络科学学院）； Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.CL

2503.12931 2025-05-21 cs.CR cs.AI 70%

MirrorShield: Towards Universal Defense Against Jailbreaks via Entropy-Guided Mirror Crafting

Rui Pu, Chaozhuo Li, Rui Ha, Litian Zhang, Lirong Qiu, Xi Zhang

机构 * Beijing University of Posts and Telecommunications（北京邮电大学）； Beihang University（北航）

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI

2504.09191 2025-04-15 cs.CL 70%

Feature-Aware Malicious Output Detection and Mitigation

Weilong Dong, Peiguang Li, Yu Tian, Xinyi Zeng, Fengdi Li, Sirui Wang

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL

2504.05689 2025-04-09 cs.CL cs.CR 70%

Separator Injection Attack: Uncovering Dialogue Biases in Large Language Models Caused by Role Separators

Xitao Li, Haijun Wang, Jiang Wu, Ting Liu

专题命中越狱攻击：safety(abstract)；prompt injection(abstract)；分类 cs.CL

2408.11313 2025-04-03 cs.AI 70%

An Optimizable Suffix Is Worth A Thousand Templates: Efficient Black-box Jailbreaking without Affirmative Phrases via LLM as Optimizer

Weipeng Jiang, Zhenting Wang, Juan Zhai, Shiqing Ma, Zhengyu Zhao, Chao Shen

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.AI

Comments Be accepeted as NAACL2025 Findings

2503.23250 2025-04-01 cs.CR cs.AI 70%

Encrypted Prompt: Securing LLM Applications Against Unauthorized Actions

Shih-Han Chan

专题命中越狱攻击：safety(abstract)；prompt injection(abstract)；分类 cs.AI

2503.05264 2025-03-10 cs.CR cs.AI 70%

Jailbreaking is (Mostly) Simpler Than You Think

Mark Russinovich, Ahmed Salem

专题命中越狱攻击：safety(abstract)；AI safety(abstract)；分类 cs.AI

2501.18632 2025-03-05 cs.CR cs.CL 70%

Towards Safe AI Clinicians: A Comprehensive Study on Large Language Model Jailbreaking in Healthcare

Hang Zhang, Qian Lou, Yanshan Wang

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.CL

2503.00224 2025-03-04 cs.CL 70%

À la recherche du sens perdu: your favourite LLM might have more to say than you can understand

K. O. T. Erziev

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL

Comments 22 pages

2502.13175 2025-02-26 cs.CR cs.AI cs.RO 70%

Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks

Wenpeng Xing, Minghao Li, Mohan Li, Meng Han

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI

2502.17254 2025-02-25 cs.LG 70%

REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective

Simon Geisler, Tom Wollschläger, M. H. I. Abdalla, Vincent Cohen-Addad, Johannes Gasteiger, Stephan Günnemann

专题命中越狱攻击：alignment(abstract)；jailbreak(abstract)；分类 cs.LG

Comments 30 pages, 6 figures, 15 tables

2501.14073 2025-02-19 cs.CL 70%

LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language

Yubin Ge, Neeraja Kirtane, Hao Peng, Dilek Hakkani-Tür

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL

Comments 16 pages

2502.05772 2025-02-11 cs.CV cs.AI 70%

Effective Black-Box Multi-Faceted Attacks Breach Vision Large Language Model Guardrails

Yijun Yang, Lichao Wang, Xiao Yang, Lanqing Hong, Jun Zhu

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.AI

2410.17922 2025-02-11 cs.AI 70%

Dynamic Guided and Domain Applicable Safeguards for Enhanced Security in Large Language Models

Weidi Luo, He Cao, Zijing Liu, Yu Wang, Aidan Wong, Bing Feng, Yuan Yao, Yu Li

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI

2501.17715 2025-01-30 cs.CL 70%

RICoTA: Red-teaming of In-the-wild Conversation with Test Attempts

Eujeong Choi, Younghun Jeong, Soomin Kim, Won Ik Cho

专题命中越狱攻击：jailbreak(abstract)；red teaming(abstract)；分类 cs.CL

Comments PACLIC 38

2411.19335 2024-12-20 cs.CR cs.AI 70%

PEFT-as-an-Attack! Jailbreaking Language Models during Federated Parameter-Efficient Fine-Tuning

Shenghui Li, Edith C. -H. Ngai, Fanghua Ye, Thiemo Voigt

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.AI

2412.06181 2024-12-10 cs.CR cs.AI 70%

Enhancing Adversarial Resistance in LLMs with Recursion

Bryan Li, Sounak Bagchi, Zizhan Wang

专题命中越狱攻击：safety(abstract)；AI safety(abstract)；分类 cs.AI

2311.03191 2024-12-02 cs.LG cs.CR 70%

DeepInception: Hypnotize Large Language Model to Be Jailbreaker

Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, Bo Han

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.LG

2411.17693 2024-11-27 cs.CL 70%

Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats

Jiaxin Wen, Vivek Hebbar, Caleb Larson, Aryan Bhatt, Ansh Radhakrishnan, Mrinank Sharma, Henry Sleight, Shi Feng, He He, Ethan Perez, Buck Shlegeris, Akbir Khan

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.CL

2411.07494 2024-11-13 cs.CL 70%

Rapid Response: Mitigating LLM Jailbreaks with a Few Examples

Alwin Peng, Julian Michael, Henry Sleight, Ethan Perez, Mrinank Sharma

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL

2410.13691 2024-11-12 cs.RO cs.AI 70%

Jailbreaking LLM-Controlled Robots

Alexander Robey, Zachary Ravichandran, Vijay Kumar, Hamed Hassani, George J. Pappas

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI

2410.16950 2024-10-23 cs.CR cs.AI 70%

Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In

Itay Nakash, George Kour, Guy Uziel, Ateret Anaby-Tavor

专题命中越狱攻击：safety(abstract)；prompt injection(abstract)；分类 cs.AI

2405.17894 2024-10-15 cs.CV cs.AI 70%

White-box Multimodal Jailbreaks Against Large Vision-Language Models

Ruofan Wang, Xingjun Ma, Hanxu Zhou, Chuanjun Ji, Guangnan Ye, Yu-Gang Jiang

专题命中越狱攻击：alignment(abstract)；jailbreak(abstract)；分类 cs.AI

2410.09040 2024-10-14 cs.CL 70%

AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation

Zijun Wang, Haoqin Tu, Jieru Mei, Bingchen Zhao, Yisen Wang, Cihang Xie

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.CL

2406.03589 2024-09-26 cs.CL 70%

Ranking Manipulation for Conversational Search Engines

Samuel Pfrommer, Yatong Bai, Tanmay Gautam, Somayeh Sojoudi

专题命中越狱攻击：safety(abstract)；prompt injection(abstract)；分类 cs.CL

Comments 2024 Conference on Empirical Methods in Natural Language Processing (Main)

2402.18104 2024-06-11 cs.CR cs.AI 70%

Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction

Tong Liu, Yingjie Zhang, Zhe Zhao, Yinpeng Dong, Guozhu Meng, Kai Chen

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI

2405.16567 2024-05-29 cs.AI cs.CR 70%

Automatic Jailbreaking of the Text-to-Image Generative AI Systems

Minseon Kim, Hyomin Lee, Boqing Gong, Huishuai Zhang, Sung Ju Hwang

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.AI

Comments Under review

2310.15851 2024-03-25 cs.CL 70%

Self-Guard: Empower the LLM to Safeguard Itself

Zezhong Wang, Fangkai Yang, Lu Wang, Pu Zhao, Hongru Wang, Liang Chen, Qingwei Lin, Kam-Fai Wong

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL

2403.09513 2024-03-15 cs.CR cs.AI 70%

AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting

Yu Wang, Xiaogeng Liu, Yu Li, Muhao Chen, Chaowei Xiao

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI

Comments Multimodal Large Language Models Defense, 25 Pages

1. 越狱攻击 1730 篇

Automated Trustworthiness Oracle Generation for Machine Learning Text Classifiers

CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuning

MirrorShield: Towards Universal Defense Against Jailbreaks via Entropy-Guided Mirror Crafting

Feature-Aware Malicious Output Detection and Mitigation

Separator Injection Attack: Uncovering Dialogue Biases in Large Language Models Caused by Role Separators

An Optimizable Suffix Is Worth A Thousand Templates: Efficient Black-box Jailbreaking without Affirmative Phrases via LLM as Optimizer

Encrypted Prompt: Securing LLM Applications Against Unauthorized Actions

Jailbreaking is (Mostly) Simpler Than You Think

Towards Safe AI Clinicians: A Comprehensive Study on Large Language Model Jailbreaking in Healthcare

À la recherche du sens perdu: your favourite LLM might have more to say than you can understand

Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks

REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective

LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language

Effective Black-Box Multi-Faceted Attacks Breach Vision Large Language Model Guardrails

Dynamic Guided and Domain Applicable Safeguards for Enhanced Security in Large Language Models

RICoTA: Red-teaming of In-the-wild Conversation with Test Attempts

PEFT-as-an-Attack! Jailbreaking Language Models during Federated Parameter-Efficient Fine-Tuning

Enhancing Adversarial Resistance in LLMs with Recursion

DeepInception: Hypnotize Large Language Model to Be Jailbreaker

Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats

Rapid Response: Mitigating LLM Jailbreaks with a Few Examples

Jailbreaking LLM-Controlled Robots

Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In

White-box Multimodal Jailbreaks Against Large Vision-Language Models

AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation

Ranking Manipulation for Conversational Search Engines

Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction

Automatic Jailbreaking of the Text-to-Image Generative AI Systems

Self-Guard: Empower the LLM to Safeguard Itself

AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting