随着大语言模型(Large Language Models, LLMs)的发展,代码生成领域已取得显著进展,LLM能够将自然语言转换为程序代码。这类代码LLM已被海量用户和机构广泛采用。然而,代码中隐藏着一种危险属性,即致命漏洞的存在。尽管部分LLM提供商已尝试通过与人类引导对齐来解决这些问题,但这些努力仍不足以让代码LLM具备实用性和鲁棒性。若不深入了解LLM在实际最坏场景下的表现,将其应用于各类真实场景将令人担忧。本文针对一个关键问题展开研究:现有代码LLM是否能完全避免生成存在漏洞的代码?若不能,在实际部署场景中该问题可能达到的最严重程度是多少?本文提出了DeceptPrompt,这是一种新型算法,能够生成对抗性自然语言指令,诱导代码LLM生成功能正确但存在漏洞的代码。DeceptPrompt通过一套基于进化的系统化算法及细粒度损失设计实现。其独特优势在于,能够找到语义完全良性且无指向性的自然前缀/后缀,同时具备极强的诱导代码LLM生成漏洞代码的能力。这一特性使我们能够在用户使用自然语言的真实场景下,对这些LLM开展接近最坏情况的红队测试。我们针对DeceptPrompt开展的大量实验与分析不仅验证了该方法的有效性,也揭示了LLM在代码生成任务中存在的巨大缺陷。当应用优化后的前缀/后缀时,攻击成功率(attack success rate, ASR)相较于不使用前缀/后缀的情况平均提升50%。
英文摘要:
With the advancement of Large Language Models (LLMs), significant progress has been made in code generation, enabling LLMs to transform natural language into programming code. These Code LLMs have been widely accepted by massive users and organizations. However, a dangerous nature is hidden in the code, which is the existence of fatal vulnerabilities. While some LLM providers have attempted to address these issues by aligning with human guidance, these efforts fall short of making Code LLMs practical and robust. Without a deep understanding of the performance of the LLMs under the practical worst cases, it would be concerning to apply them to various real-world applications. In this paper, we answer the critical issue: Are existing Code LLMs immune to generating vulnerable code? If not, what is the possible maximum severity of this issue in practical deployment scenarios? In this paper, we introduce DeceptPrompt, a novel algorithm that can generate adversarial natural language instructions that drive the Code LLMs to generate functionality correct code with vulnerabilities. DeceptPrompt is achieved through a systematic evolution-based algorithm with a fine grain loss design. The unique advantage of DeceptPrompt enables us to find natural prefix/suffix with totally benign and non-directional semantic meaning, meanwhile, having great power in inducing the Code LLMs to generate vulnerable code. This feature can enable us to conduct the almost-worstcase red-teaming on these LLMs in a real scenario, where users are using natural language. Our extensive experiments and analyses on DeceptPrompt not only validate the effectiveness of our approach but also shed light on the huge weakness of LLMs in the code generation task. When applying the optimized prefix/suffix, the attack success rate (ASR) will improve by average 50% compared with no prefix/suffix applying.
Prompt-in-Content Attacks: Exploiting Uploaded Inputs to Hijack LLM Behavior
Graduate School of Advanced Science and Engineering(先进科学与工程研究生学校);Hiroshima University(广岛大学);Graduate School of Science and Engineering(科学与工程研究生学校);Hosei University(立命馆大学);School of Mathematics and Computer Science(数学与计算机科学学院);Tongling University(铜陵大学);School of Computer Science and Engineering(计算机科学与工程学院);University of Aizu(立海大学)
PiCo: Jailbreaking Multimodal Large Language Models via Pictorial Code Contextualization
School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院);School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs
University of Cambridge(剑桥大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Mila, University of Montreal(蒙特利尔大学Mila研究所);Google DeepMind(谷歌DeepMind)
"Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wild
Griffith University(格里菲斯大学);Nanyang Technological University(南洋理工大学);University of New South Wales(新南威尔士大学);Zhejiang Key Laboratory of Digital Fashion and Data Governance, Zhejiang Sci-Tech University(浙江数字时尚与数据治理重点实验室,浙江科技大学)
Jailbreaking Vision-Language Models Through the Visual Modality
Warsaw University of Technology(华沙理工大学);NASK National Research Institute(波兰国家研究研究院);University of Liverpool(利物浦大学);Indian Institute of Technology Roorkee(印度理工学院拉胡尔分校);Toyota Technological Institute at Chicago(芝加哥丰田技术研究所)