Static analysis, the process of examining code without executing it, is crucial for identifying software issues. Yet, static analysis is hampered by its complexity and the need for customization for different targets. Traditional static analysis tools require extensive human effort and are often limited to specific target programs and programming languages. Recent advancements in Large Language Models (LLMs), such as GPT-4 and Llama, offer new capabilities for software engineering tasks. However, their application in static analysis, especially in understanding complex code structures, remains under-explored. This paper introduces a novel approach named E&V , which leverages LLMs to perform static analysis. Specifically, E&V employs LLMs to simulate the execution of pseudo-code, effectively conducting static analysis encoded in the pseudo-code with minimal human effort, thereby improving the accuracy of results. E&V includes a verification process for pseudo-code execution without needing an external oracle. This process allows E&V to mitigate hallucinations of LLMs and enhance the accuracy of static analysis results. We have implemented E&V in a prototype tool designed for triaging crashes through backward taint analysis. This prototype, paired with GPT-4-32k, has been applied to triage 170 recently fixed Linux kernel bugs across seven bug categories. Our experiments demonstrate that the prototype correctly identifies the blamed function in 81.2% of the cases. Additionally, we observe that our novel verification process significantly improves the accuracy, increasing it from 28.2% to 81.2%.
Integrating Symbolic Execution with LLMs for Automated Generation of Program Specifications
Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);UCAS(中国科学院大学);Sch. of Computer Science & Sch. of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学计算机科学与技术学院与人工智能学院);School of Computer Science, Peking University(北京大学计算机学院);Beijing Institute of Control Engineering(北京控制工程研究所)
CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verification
Hong Kong Baptist University(香港浸会大学);UC Santa Barbara(加州大学圣巴巴拉分校);UC Berkeley(加州大学伯克利分校);Alibaba Group(阿里巴巴集团);The University of Tokyo(东京大学);University of Alberta(阿尔伯塔大学)
From Natural Language to Verified Code: Toward AI Assisted Problem-to-Code Generation with Dafny-Based Formal Verification
Department of Computer Science, The University of Alabama(阿拉巴马大学计算机科学系);The University of Alabama(阿拉巴马大学);Alabama Water Institute, The University of Alabama(阿拉巴马水研究院,阿拉巴马大学)
AgentForge: Execution-Grounded Multi-Agent LLM Framework for Autonomous Software Engineering
International Research Center for Complexity Sciences(复杂科学国际研究中心);Hangzhou International Innovation Institute(杭州国际创新研究院);Beihang University(北京航空航天大学);College of Science, Mathematics and Technology(科学、数学与技术学院);Wenzhou-Kean University(温州-凯恩大学);Fakulti Teknologi Maklumat dan Komunikasi(信息技术与通信学院);Universiti Teknikal Malaysia Melaka(马来西亚Melaka理工大学);Computer Systems Engineering Department(计算机系统工程系);Sukkur IBA University(苏库尔IBA大学);University of Electronic Science and Technology of China(电子科学与技术大学)