基于大语言模型的可解释故障定位的定量与定性评估
A Quantitative and Qualitative Evaluation of LLM-Based Explainable Fault Localization
AI总结：
本文提出AutoFL，一种基于大语言模型的故障定位技术，通过函数调用导航代码仓库并生成缺陷解释，在798个真实缺陷上显著提升定位准确率，且其解释获得开发人员认可。
摘要：
故障定位（FL）是指开发人员试图识别代码中哪一部分出现故障并需要修复的过程，这是调试中反复出现的挑战。为了减轻开发人员的负担，人们提出了许多自动化故障定位技术。然而，先前的工作指出，现有技术未能为所建议的位置提供理由，阻碍了开发人员对这些技术的采用。考虑到这一点，我们提出了AutoFL，一种基于大语言模型（LLM）的故障定位技术，它在给出建议的故障位置的同时生成对缺陷的解释。AutoFL提示大语言模型使用函数调用来导航代码仓库，从而能够在大型软件仓库上有效地定位故障，并克服大语言模型上下文长度的限制。在Java和Python中798个真实世界缺陷上的大量实验表明，AutoFL将方法级acc@1相较于基线提高了最高233.3%。此外，我们还就开发人员对AutoFL生成解释的印象进行了访谈，结果表明开发人员普遍喜欢AutoFL的自然语言解释，并且他们更倾向于阅读少量高质量的解释，而不是大量解释。
英文摘要：
Fault Localization (FL), in which a developer seeks to identify which part of the code is malfunctioning and needs to be fixed, is a recurring challenge in debugging. To reduce developer burden, many automated FL techniques have been proposed. However, prior work has noted that existing techniques fail to provide rationales for the suggested locations, hindering developer adoption of these techniques. With this in mind, we propose AutoFL, a Large Language Model (LLM)-based FL technique that generates an explanation of the bug along with a suggested fault location. AutoFL prompts an LLM to use function calls to navigate a repository, so that it can effectively localize faults over a large software repository and overcome the limit of the LLM context length. Extensive experiments on 798 real-world bugs in Java and Python reveal AutoFL improves method-level acc@1 by up to 233.3% over baselines. Furthermore, developers were interviewed on their impression of AutoFL-generated explanations, showing that developers generally liked the natural language explanations of AutoFL, and that they preferred reading a few, high-quality explanations instead of many.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
