Generating code via a LLM (rather than writing code from scratch), has exploded in popularity. However, the security implications of LLM-generated code are still unknown. We performed a study that compared the security and quality of human-written code with that of LLM-generated code, for a wide range of programming tasks, including data structures, algorithms, cryptographic routines, and LeetCode questions. To assess code security we used unit testing, fuzzing, and static analysis. For code quality, we focused on complexity and size. We found that LLM can generate incorrect code that fails to implement the required functionality, especially for more complicated tasks; such errors can be subtle. For example, for the cryptographic algorithm SHA1, LLM generated an incorrect implementation that nevertheless compiles. In cases where its functionality was correct, we found that LLM-generated code is less secure, primarily due to the lack of defensive programming constructs, which invites a host of security issues such as buffer overflows or integer overflows. Fuzzing has revealed that LLM-generated code is more prone to hangs and crashes than human-written code. Quality-wise, we found that LLM generates bare-bones code that lacks defensive programming constructs, and is typically more complex (per line of code) compared to human-written code. Next, we constructed a feedback loop that asked the LLM to re-generate the code and eliminate the found issues (e.g., malloc overflow, array index out of bounds, null dereferences). We found that the LLM fails to eliminate such issues consistently: while succeeding in some cases, we found instances where the re-generated, supposedly more secure code, contains new issues; we also found that upon prompting, LLM can introduce issues in files that were issues-free before prompting.
RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Virginia Tech(弗吉尼亚理工大学);Salesforce Research(Salesforce研究院);University of California Berkeley(加州大学伯克利分校);University of Chicago(芝加哥大学)
Data to Defense: The Role of Curation in Customizing LLMs Against Jailbreaking Attacks
Stony Brook University(石溪大学);Binghamton University(宾汉姆顿大学);University of Iowa(爱荷华大学);Dartmouth College(达特茅斯学院);Georgia Institute of Technology(佐治亚理工学院)
Role-Aware Language Models for Secure and Contextualized Access Control in Organizations
Mohamed bin Zayed University of Artificial Intelligence(莫扎德·本·扎耶德人工智能大学);Nazarbayev University(纳扎尔拜耶夫大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);New York University Abu Dhabi(纽约大学阿布扎克分校)
PiCo: Jailbreaking Multimodal Large Language Models via Pictorial Code Contextualization
School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院);School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
A survey of agent interoperability protocols: Model Context Protocol (MCP), Agent Communication Protocol (ACP), Agent-to-Agent Protocol (A2A), and Agent Network Protocol (ANP)
Kent State University(肯特州立大学);Cleveland State University(克利夫兰州立大学);Youngstown State University(青年州立大学);Northeastern University(东北大学)
Root Defence Strategies: Ensuring Safety of LLM at the Decoding Level
Aerospace Information Research Institute, Chinese Academy of Sciences(航天信息研究所,中国科学院);Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua University(计算机科学与技术系,人工智能研究院,清华大学);Shanghai Key Laboratory of Multi. Info. Processing, East China Normal University(上海多信息处理重点实验室,华东师范大学);Kuaishou-Inc(快手公司)
Zero-Trust Foundation Models: A New Paradigm for Secure and Collaborative Artificial Intelligence for Internet of Things
School of Electrical Engineering and Computer Science, TU Berlin(柏林技术大学电气与计算机工程学院);Real-Time and Embedded Computing Systems Research Centre (CISTER)(实时与嵌入式计算系统研究中心);Data61, CSIRO(数据61,澳大利亚联邦科学与工业研究组织);School of Computer Science and Engineering, the University of New South Wales(新南威尔士大学计算机科学与工程学院);State Key Laboratory of Public Big Data, College of Big Data and Information Engineering, Guizhou University(公共大数据国家重点实验室,贵州大学大数据与信息工程学院);Department of Computer Engineering, Qassim University(科威特大学计算机工程系);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);Division of Electrical Engineering, Department of Engineering, University of Cambridge(剑桥大学工程系电子工程系);Center for NeXt-Generation Communications (CXC), Koç University(下一代通信中心(CXC),科隆大学)