评估 LLM 识别可配置系统中编译错误的能力
Evaluating the Capability of LLMs in Identifying Compilation Errors in Configurable Systems
AI总结：
本研究评估了 ChatGPT4、Le Chat Mistral 和 Gemini Advanced 1.5 三种 LLM 在识别可配置系统编译错误方面的能力，发现 ChatGPT4 能成功识别大多数错误，展现出辅助开发者的潜力。
摘要：
编译是开发 Linux 等可配置系统的重要过程。然而，由于传统编译器不具备变异性感知能力，在可配置系统中识别编译错误并非易事。以往检测部分此类编译错误的方法通常依赖高级技术，需要程序员投入大量精力。本研究评估了大型语言模型（LLM），特别是 ChatGPT4、Le Chat Mistral 和 Gemini Advanced 1.5，在识别可配置系统编译错误方面的有效性。我们最初评估了 50 个用 C++、Java 和 C 语言编写的小型产品，随后评估了 30 个用 C 语言编写的小型可配置系统，涵盖 17 种不同类型的编译错误。ChatGPT4 成功识别了单个产品和可配置系统中的大多数编译错误，而 Le Chat Mistral 和 Gemini Advanced 1.5 检测出了其中部分错误。LLM 已展现出在协助开发者识别可配置系统编译错误方面的潜力。
英文摘要：
Compilation is an important process in developing configurable systems, such as Linux. However, identifying compilation errors in configurable systems is not straightforward because traditional compilers are not variability-aware. Previous approaches that detect some of these compilation errors often rely on advanced techniques that require significant effort from programmers. This study evaluates the efficacy of Large Language Models (LLMs), specifically ChatGPT4, Le Chat Mistral and Gemini Advanced 1.5, in identifying compilation errors in configurable systems. Initially, we evaluate 50 small products in C++, Java, and C languages, followed by 30 small configurable systems in C, covering 17 different types of compilation errors. ChatGPT4 successfully identified most compilation errors in individual products and in configurable systems, while Le Chat Mistral and Gemini Advanced 1.5 detected some of them. LLMs have shown potential in assisting developers in identifying compilation errors in configurable systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
