arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.16974cs.SE
收藏

SelfPiCo:基于LLM的自引导部分代码执行

SelfPiCo: Self-Guided Partial Code Execution with LLMs

Zhipeng Xue, Zhipeng Gao, Shaohua Wang, Xing Hu, Xin Xia, Shanping Li

AI总结:

本文提出SelfPiCo框架,结合开源LLM Code Llama,通过少样本上下文学习和思维链推理动态引导部分代码执行,显著提升代码执行率和运行时错误检测能力。

摘要:

代码可执行性在软件调试和测试中起着至关重要的作用(例如,检测运行时异常或断言违例)。然而,由于缺少定义和复杂的第三方依赖,代码执行,尤其是部分或任意代码执行,是一项艰巨的任务。为了使部分代码(如发布在网络上的代码片段或复杂软件项目深处的代码片段)可执行,现有研究提出了一种机器学习模型来预测未定义的元素类型,并将预定义的虚拟值注入执行中。然而,由于其简单设计的虚拟值和无法继续学习,其工具的性能受到限制。在本文中,我们设计并实现了一个名为SelfPiCo(自引导部分代码执行器)的新框架,通过在交互循环中结合开源LLM(即Code Llama)来动态引导部分代码执行。特别是,SelfPiCo利用少样本上下文学习和思维链推理,基于对Code Llama模型的微调来引导人类知识和逻辑推理。SelfPiCo持续从代码执行结果中学习,并逐步优化其预测。我们的评估表明,SelfPiCo可以执行开源代码和Stack Overflow片段中所有行的72.7%和83.3%,分别比最新的最先进工具Lexecutor高出37.9%和33.5%。此外,SelfPiCo通过执行来自八个GitHub软件项目和43个Stack Overflow帖子的部分代码,成功检测出18个和33个运行时类型错误问题,展示了我们的框架在实际中的实用性和潜在应用。

英文摘要:

Code executability plays a vital role in software debugging and testing (e.g., detecting runtime exceptions or assertion violations). However, code execution, especially partial or arbitrary code execution, is a non-trivial task due to missing definitions and complex third-party dependencies. To make partial code (such as code snippets posted on the web or code fragments deep inside complex software projects) executable, the existing study has proposed a machine learning model to predict the undefined element types and inject the pre-defined dummy values into execution. However, the performance of their tool is limited due to its simply designed dummy values and the inability to continue learning. In this paper, we design and implement a novel framework, named SelfPiCo (Self Guided Partial Code Executor), to dynamically guide partial code execution by incorporating the open-source LLM (i.e., Code Llama) within an interactive loop. Particularly, SelfPiCo leverages few-shot in-context learning and chain-of-thought reasoning to elicit human knowledge and logical reasoning based on fine-tuning the Code Llama model. SelfPiCo continuously learns from code execution results and refines its predictions step after step. Our evaluations demonstrate that SelfPiCo can execute 72.7% and 83.3% of all lines in the open-source code and Stack Overflow snippets, outperforming the most recent state-of-the-art Lexecutor by 37.9% and 33.5%, respectively. Moreover, SelfPiCo successfully detected 18 and 33 runtime type error issues by executing the partial code from eight GitHub software projects and 43 Stack Overflow posts, demonstrating the practical usage and potential application of our framework in practice.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑