arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.08477cs.SE
收藏

E&V:通过伪代码执行与验证提示大语言模型执行静态分析

E&V: Prompting Large Language Models to Perform Static Analysis by Pseudo-code Execution and Verification

Yu Hao, Weiteng Chen, Ziqiao Zhou, Weidong Cui

AI总结:

本研究提出利用大语言模型模拟伪代码执行的E&V静态分析方法,搭配无需外部基准的验证流程缓解幻觉问题,在Linux内核bug分类任务中准确率达81.2%。

摘要:

静态分析是在不执行代码的情况下检查代码的过程,对于识别软件问题至关重要。然而,静态分析受限于其复杂性以及需要针对不同目标进行定制化的问题。传统静态分析工具需要大量人力投入,且通常局限于特定的目标程序和编程语言。大语言模型(LLM)如GPT-4和Llama的最新进展为软件工程任务提供了新能力,但它们在静态分析领域,尤其是在理解复杂代码结构方面的应用仍未得到充分探索。\n本文提出了一种名为E&V的新方法,该方法利用大语言模型执行静态分析。具体而言,E&V借助大语言模型模拟伪代码的执行,以最少的人力投入有效完成伪代码中编码的静态分析任务,从而提升结果的准确性。E&V包含一个无需外部基准(oracle)的伪代码执行验证流程,该流程能够缓解大语言模型的幻觉问题,提高静态分析结果的准确率。\n我们将E&V实现为一个原型工具,用于通过逆向污点分析对崩溃进行分类。该原型搭配GPT-4-32k,已被应用于分类7个bug类别中170个近期修复的Linux内核bug。实验表明,该原型在81.2%的案例中正确识别出了问题函数。此外,我们发现所提出的新型验证流程显著提升了准确率,将其从28.2%提高到81.2%。

英文摘要:

Static analysis, the process of examining code without executing it, is crucial for identifying software issues. Yet, static analysis is hampered by its complexity and the need for customization for different targets. Traditional static analysis tools require extensive human effort and are often limited to specific target programs and programming languages. Recent advancements in Large Language Models (LLMs), such as GPT-4 and Llama, offer new capabilities for software engineering tasks. However, their application in static analysis, especially in understanding complex code structures, remains under-explored. This paper introduces a novel approach named E&V , which leverages LLMs to perform static analysis. Specifically, E&V employs LLMs to simulate the execution of pseudo-code, effectively conducting static analysis encoded in the pseudo-code with minimal human effort, thereby improving the accuracy of results. E&V includes a verification process for pseudo-code execution without needing an external oracle. This process allows E&V to mitigate hallucinations of LLMs and enhance the accuracy of static analysis results. We have implemented E&V in a prototype tool designed for triaging crashes through backward taint analysis. This prototype, paired with GPT-4-32k, has been applied to triage 170 recently fixed Linux kernel bugs across seven bug categories. Our experiments demonstrate that the prototype correctly identifies the blamed function in 81.2% of the cases. Additionally, we observe that our novel verification process significantly improves the accuracy, increasing it from 28.2% to 81.2%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑