Prompting Techniques for Secure Code Generation: A Systematic Investigation
Hamburg University of Technology（汉堡工业大学）
Catherine Tony, Nicolás E. Díaz Ferreyra, Markus Mutas, Salem Dhiff, Riccardo Scandariato
备注：
Work partially supported by the EU-funded project Sec4AI4Sec: Cybersecurity for AI-Augmented Systems (grant no. 101120393) - ACCEPTED at ACM Transactions on Software Engineering and Methodology (Feb. 2025)
Large Language Models (LLMs) are gaining momentum in software development with prompt-driven programming enabling developers to create code from natural language (NL) instructions. However, studies have questioned their ability to produce secure code and, thereby, the quality of prompt-generated software. Alongside, various prompting techniques that carefully tailor prompts have emerged to elicit optimal responses from LLMs. Still, the interplay between such prompting strategies and secure code generation remains under-explored and calls for further investigations. OBJECTIVE: In this study, we investigate the impact of different prompting techniques on the security of code generated from NL instructions by LLMs. METHOD: First we perform a systematic literature review to identify the existing prompting techniques that can be used for code generation tasks. A subset of these techniques are evaluated on GPT-3, GPT-3.5, and GPT-4 models for secure code generation. For this, we used an existing dataset consisting of 150 NL security-relevant code-generation prompts. RESULTS: Our work (i) classifies potential prompting techniques for code generation (ii) adapts and evaluates a subset of the identified techniques for secure code generation tasks and (iii) observes a reduction in security weaknesses across the tested LLMs, especially after using an existing technique called Recursive Criticism and Improvement (RCI), contributing valuable insights to the ongoing discourse on LLM-generated code security.
An Empirical Evaluation of LLM-Generated Code Security Across Prompting Methods
Department of Computer Science, Birzeit University（计算机科学系，巴勒斯坦比泽大学）；King Fahd University of Petroleum and Minerals（国王法赫德石油和矿物大学）；University of Central Florida（中央佛罗里达大学）
The Prompt Report: A Systematic Survey of Prompt Engineering Techniques
University of Maryland（马里兰大学）；Learn Prompting；OpenAI；Stanford（斯坦福大学）；Microsoft（微软公司）；Vanderbilt（范德堡大学）；Princeton（普林斯顿大学）；Texas State University（德克萨斯州立大学）；Icahn School of Medicine（伊坎医学院）；ASST Brianza（布里安扎综合医院）；Mount Sinai Beth Israel（西奈山 Beth Israel 医院）；Instituto de Telecomunicações（电信研究所）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
Security Degradation in Iterative AI Code Generation -- A Systematic Analysis of the Paradox
Department of Analytics and Information Systems（分析与信息系统系）；University of San Francisco（旧金山大学）；Department of Applied AI and Industry Innovation（应用人工智能与产业创新系）；Vector Institute for Artificial Intelligence（人工智能研究院）；Department of Management Science and Information Systems（管理科学与信息系统系）；University of Massachusetts Boston（马萨诸塞大学波士顿分校）
Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theory
MAIS, Institute of Automation, Chinese Academy of Sciences（自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；University of California, Santa Barbara（加州大学圣巴巴拉分校）；Beijing Wenge Technology Co., Ltd（北京文格科技有限公司）
Controllable Generation from Pre-trained Language Models via Inverse Prompting
Tsinghua University（清华大学）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）；Recurrent AI, Ltd.（Recurrent AI 有限公司）；DAMO Academy, Alibaba Inc.（阿里巴巴达摩院）
Multimodal Large Language Models for End-to-End Affective Computing: Benchmarking and Boosting with Generative Knowledge Prompting
School of Computer Science, South China Normal University（华南师范大学计算机科学学院）；School of Information Technology in Education, South China Normal University（华南师范大学教育信息技术学院）
PromptGuard: An Orchestrated Prompting Framework for Principled Synthetic Text Generation for Vulnerable Populations using LLMs with Enhanced Safety, Fairness, and Controllability
Posts and Telecommunications Institute of Technology（邮电技术研究所）；Hanoi Architectural University（河内建筑大学）
