大模型对齐与安全 - arXivDaily 专题

2404.02067 2024-04-03 cs.CV cs.AI cs.LG 62%

Red-Teaming Segment Anything Model

Krzysztof Jankowski, Bartlomiej Sobieski, Mateusz Kwiatkowski, Jakub Szulc, Michal Janik, Hubert Baniecki, Przemyslaw Biecek

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments CVPR 2024 - The 4th Workshop of Adversarial Machine Learning on Computer Vision: Robustness of Foundation Models

2403.12171 2024-03-20 cs.CL cs.AI 62%

EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models

Weikang Zhou, Xiao Wang, Limao Xiong, Han Xia, Yingshuang Gu, Mingxu Chai, Fukang Zhu, Caishuang Huang, Shihan Dou, Zhiheng Xi, Rui Zheng, Songyang Gao, Yicheng Zou, Hang Yan, Yifan Le, Ruohui Wang, Lijun Li, Jing Shao, Tao Gui, Qi Zhang, Xuanjing Huang

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI

2211.14769 2024-03-19 cs.AI cs.CL cs.CR cs.CV 62%

Navigation as Attackers Wish? Towards Building Robust Embodied Agents under Federated Learning

Yunchao Zhang, Zonglin Di, Kaiwen Zhou, Cihang Xie, Xin Eric Wang

专题命中越狱攻击：alignment(abstract)；分类 cs.CL、cs.AI

2305.13860 2024-03-12 cs.SE cs.AI cs.CL 62%

Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study

Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, Kailong Wang, Yang Liu

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI

2311.16119 2024-03-05 cs.CR cs.AI cs.CL 62%

Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competition

Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Liu Kost, Christopher Carnahan, Jordan Boyd-Graber

专题命中越狱攻击：prompt injection(abstract)；分类 cs.CL、cs.AI

Comments 34 pages, 8 figures Codebase: https://github.com/PromptLabs/hackaprompt Dataset: https://huggingface.co/datasets/hackaprompt/hackaprompt-dataset/blob/main/README.md Playground: https://huggingface.co/spaces/hackaprompt/playground

2402.15690 2024-02-27 cs.CL cs.AI 62%

Foot In The Door: Understanding Large Language Model Jailbreaking via Cognitive Psychology

Zhenhua Wang, Wei Xie, Baosheng Wang, Enze Wang, Zhiwen Gui, Shuoyoucheng Ma, Kai Chen

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI

2402.15570 2024-02-27 cs.CR cs.AI cs.CL 62%

Fast Adversarial Attacks on Language Models In One GPU Minute

Vinu Sankar Sadasivan, Shoumik Saha, Gaurang Sriramanan, Priyatham Kattakinda, Atoosa Chegini, Soheil Feizi

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI

2402.14020 2024-02-22 cs.LG cs.CL cs.CR 62%

Coercing LLMs to do and reveal (almost) anything

Jonas Geiping, Alex Stein, Manli Shu, Khalid Saifullah, Yuxin Wen, Tom Goldstein

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.LG

Comments 32 pages. Implementation available at https://github.com/JonasGeiping/carving

2311.11509 2024-02-20 cs.CL cs.LG 62%

Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information

Zhengmian Hu, Gang Wu, Saayan Mitra, Ruiyi Zhang, Tong Sun, Heng Huang, Viswanathan Swaminathan

专题命中越狱攻击：alignment(abstract)；分类 cs.CL、cs.LG

2402.10196 2024-02-16 cs.CL cs.AI 62%

A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents

Lingbo Mo, Zeyi Liao, Boyuan Zheng, Yu Su, Chaowei Xiao, Huan Sun

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI

2306.14043 2024-02-13 cs.LG cs.AI cs.CR 62%

Machine Learning needs Better Randomness Standards: Randomised Smoothing and PRNG-based attacks

Pranav Dahiya, Ilia Shumailov, Ross Anderson

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments USENIX Security 2024 (https://www.usenix.org/conference/usenixsecurity24/presentation/dahiya)

2402.05467 2024-02-09 cs.AI cs.CL cs.CR 62%

Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxia

Guangyu Shen, Siyuan Cheng, Kaiyuan Zhang, Guanhong Tao, Shengwei An, Lu Yan, Zhuo Zhang, Shiqing Ma, Xiangyu Zhang

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI

2402.02309 2024-02-07 cs.LG cs.CL cs.CR cs.CV 62%

Jailbreaking Attack against Multimodal Large Language Model

Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, Rong Jin

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.LG

2206.06854 2024-02-05 cs.AI cs.CR cs.CV cs.LG stat.ML 62%

On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspective

Mathieu Serrurier, Franck Mamalet, Thomas Fel, Louis Béthune, Thibaut Boissin

专题命中越狱攻击：alignment(abstract)；分类 cs.AI、cs.LG

Journal ref Conference on Neural Information Processing Systems (NeurIPS), Neural Information Processing Systems Foundation, Dec 2023, New Orleans (Louisiana), United States

2311.17035 2023-11-29 cs.LG cs.CL cs.CR 62%

Scalable Extraction of Training Data from (Production) Language Models

Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo, Eric Wallace, Florian Tramèr, Katherine Lee

专题命中越狱攻击：alignment(abstract)；分类 cs.CL、cs.LG

2310.12860 2023-10-31 cs.CL cs.CY 62%

Probing LLMs for hate speech detection: strengths and vulnerabilities

Sarthak Roy, Ashish Harshavardhan, Animesh Mukherjee, Punyajoy Saha

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.CY

Comments 13 pages, 9 figures, 7 tables, accepted to findings of EMNLP 2023

2309.11751 2023-10-17 cs.CV cs.AI cs.CR cs.LG 62%

How Robust is Google's Bard to Adversarial Image Attacks?

Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, Jun Zhu

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments Technical report

2309.00614 2023-09-06 cs.LG cs.CL cs.CR 62%

Baseline Defenses for Adversarial Attacks Against Aligned Language Models

Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, Tom Goldstein

专题命中越狱攻击：alignment(abstract)；分类 cs.CL、cs.LG

Comments 12 pages

2308.15092 2023-08-30 math.NA cs.AI cs.LG cs.NA 62%

Can We Rely on AI?

Desmond J. Higham

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

2202.00091 2023-03-27 cs.LG cs.AI cs.CR cs.CV 62%

Query Efficient Decision Based Sparse Attacks Against Black-Box Deep Learning Models

Viet Quoc Vo, Ehsan Abbasnejad, Damith C. Ranasinghe

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments Published as a conference paper at the International Conference on Learning Representations (ICLR 2022). Code is available at https://sparseevoattack.github.io/

2302.07953 2023-02-17 cs.CR cs.AI cs.LG cs.RO 62%

AI Security Threats against Pervasive Robotic Systems: A Course for Next Generation Cybersecurity Workforce

Sudip Mittal, Jingdao Chen

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

2209.03463 2022-09-12 cs.CY cs.AI cs.CR cs.SI 62%

Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbots

Wai Man Si, Michael Backes, Jeremy Blackburn, Emiliano De Cristofaro, Gianluca Stringhini, Savvas Zannettou, Yang Zhang

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.CY

Journal ref Published in ACM CCS 2022. Please cite the CCS version

2203.15506 2022-03-30 cs.CR cs.AI cs.LG 62%

Trojan Horse Training for Breaking Defenses against Backdoor Attacks in Deep Learning

Arezoo Rajabi, Bhaskar Ramasubramanian, Radha Poovendran

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments Submitted to conference

2203.15498 2022-03-30 cs.CR cs.AI cs.CV cs.LG 62%

Powerful Physical Adversarial Examples Against Practical Face Recognition Systems

Inderjeet Singh, Toshinori Araki, Kazuya Kakizaki

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments Accepted at IEEE/CVF WACV 2022 MAP

2108.01124 2021-08-04 cs.CR cs.AI cs.LG stat.AP 62%

Efficacy of Statistical and Artificial Intelligence-based False Information Cyberattack Detection Models for Connected Vehicles

Sakib Mahmud Khan, Gurcan Comert, Mashrur Chowdhury

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 6 figures

2101.06704 2021-01-19 cs.AI cs.CR cs.CV cs.LG 62%

Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions

Nodens Koren, Qiuhong Ke, Yisen Wang, James Bailey, Xingjun Ma

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments Preprint

2607.09787 2026-07-14 cs.CV cs.LG 新提交 61%

Adversarially Guided Diffusion for LiDAR Range Image Synthesis

用于激光雷达距离图像合成的对抗引导扩散

Stavros Bouras, Antonios Makris, Alexandros Gkillas, Aris S. Lalos, Konstantinos Tserpes

机构 * School of Electrical and Computer Engineering, National Technical University of Athens（雅典国立技术大学电气与计算机工程学院）； Industrial Systems Institute, Athena Research Center（雅典娜研究中心工业系统研究所）

专题命中越狱攻击：safety(abstract)；分类 cs.LG；trustworthy(comments)

AI总结研究针对二维距离图像分割的无限制对抗攻击，提出基于扩散并利用分割损失对抗引导的方法，在SemanticKITTI数据集实验，能跨架构转移，相比基线在有效性与现实性间有独特权衡，实现可控退化。

Comments Accepted at the 1st Workshop on Secure and Trustworthy AI (STAI 2026), co-located with the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2026)

2510.11837 2025-10-15 cs.CR cs.AI 61%

Countermind: A Multi-Layered Security Architecture for Large Language Models

Dominik Schwarz

机构 * Independent Researcher（独立研究者）

专题命中越狱攻击：prompt injection(abstract,comments)；分类 cs.AI

Comments 33 pages, 3 figures, 6 tables. Keywords: LLM security; defense-in-depth; prompt injection; activation steering; multimodal sandbox; threat modeling

2608.22716 2026-08-25 cs.CR cs.AI 新提交 57%

TEE-X: TEE-aware Acceleration Framework for Large Vision Models at the Edge

TEE-X：面向边缘端大视觉模型的感知可信执行环境加速框架

Kurt M Wilson, Mohaiminul Al Nahian, Abeer Matar A. Almalky, Sadat Shahriyar, Souvik Kundu, Zhishan Guo, Abdullah Al Arafat, Adnan Siraj Rakin

专题命中越狱攻击：safety(abstract)；分类 cs.AI

AI总结该研究提出TEE-X框架，在Arm TrustZone的OP-TEE上验证，可在NVIDIA Jetson AGX Xavier上实现大视觉模型的安全高效边缘推理，兼顾性能与安全性。

2604.08169 2026-08-25 cs.AI 版本更新 57%

Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence

激活引导用于不牺牲连贯性的对齐开放式生成

Niklas Herbster, Martin Zborowski, Alberto Tosato, Gauthier Gidel, Tommaso Tosato

机构 * Tara Research ； Technical University of Munich（慕尼黑工业大学）； Mila Quebec AI Institute（Mila魁北克人工智能研究所）

专题命中越狱攻击：alignment(abstract)；分类 cs.AI

AI总结本文提出激活引导方法，通过在生成过程中持续修正偏差激活，提升大模型的对齐性能，实验表明StTP和StMP在保持连贯性的同时更有效维护通用能力。

1. 越狱攻击 1731 篇

Red-Teaming Segment Anything Model

EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models

Navigation as Attackers Wish? Towards Building Robust Embodied Agents under Federated Learning

Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study

Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competition

Foot In The Door: Understanding Large Language Model Jailbreaking via Cognitive Psychology

Fast Adversarial Attacks on Language Models In One GPU Minute

Coercing LLMs to do and reveal (almost) anything

Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information

A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents

Machine Learning needs Better Randomness Standards: Randomised Smoothing and PRNG-based attacks

Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxia

Jailbreaking Attack against Multimodal Large Language Model

On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspective

Scalable Extraction of Training Data from (Production) Language Models

Probing LLMs for hate speech detection: strengths and vulnerabilities

How Robust is Google's Bard to Adversarial Image Attacks?

Baseline Defenses for Adversarial Attacks Against Aligned Language Models

Can We Rely on AI?

Query Efficient Decision Based Sparse Attacks Against Black-Box Deep Learning Models

AI Security Threats against Pervasive Robotic Systems: A Course for Next Generation Cybersecurity Workforce

Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbots

Trojan Horse Training for Breaking Defenses against Backdoor Attacks in Deep Learning

Powerful Physical Adversarial Examples Against Practical Face Recognition Systems

Efficacy of Statistical and Artificial Intelligence-based False Information Cyberattack Detection Models for Connected Vehicles

Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions

Adversarially Guided Diffusion for LiDAR Range Image Synthesis

Countermind: A Multi-Layered Security Architecture for Large Language Models

TEE-X: TEE-aware Acceleration Framework for Large Vision Models at the Edge

Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence