实际代码生成中的大语言模型幻觉：现象、机制与缓解方法
LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation
AI总结：
本文针对仓库级实际复杂代码生成场景，实证研究LLM代码幻觉的现象、机制与缓解方法，构建幻觉分类体系、分析分布与成因，并提出基于RAG的通用有效缓解方法。
摘要：
代码生成旨在根据输入需求自动生成代码，可显著提升开发效率。近期基于大语言模型（LLM）的方法已展现出可观成效，彻底革新了代码生成任务。尽管性能表现亮眼，LLM 生成的内容常存在幻觉问题，在实际开发流程中需要处理复杂上下文依赖的代码生成场景下尤为突出。\n此前虽有研究分析了 LLM 驱动的代码生成中的幻觉问题，但这类研究仅局限于独立函数生成场景。本文针对仓库级生成场景中更贴近实际、更复杂的开发语境，开展实证研究，探究 LLM 幻觉的现象、机制与缓解方法。首先，我们人工检视了 6 款主流 LLM 的代码生成结果，构建了 LLM 生成代码的幻觉分类体系。随后，我们详细阐述了幻觉现象，分析了其在不同模型中的分布情况。接着我们分析了幻觉的成因，识别出 4 类可能引发幻觉的潜在因素。最后，我们提出了一种基于检索增强生成（RAG）的缓解方法，该方法在所有受试 LLM 上均表现出稳定的有效性。包含代码、数据与实验结果的复现包可访问 https://github.com/DeepSoftwareAnalytics/LLMCodingHallucination 获取。
英文摘要：
Code generation aims to automatically generate code from input requirements, significantly enhancing development efficiency. Recent large language models (LLMs) based approaches have shown promising results and revolutionized code generation task. Despite the promising performance, LLMs often generate contents with hallucinations, especially for the code generation scenario requiring the handling of complex contextual dependencies in practical development process. Although previous study has analyzed hallucinations in LLM-powered code generation, the study is limited to standalone function generation. In this paper, we conduct an empirical study to study the phenomena, mechanism, and mitigation of LLM hallucinations within more practical and complex development contexts in repository-level generation scenario. First, we manually examine the code generation results from six mainstream LLMs to establish a hallucination taxonomy of LLM-generated code. Next, we elaborate on the phenomenon of hallucinations, analyze their distribution across different models. We then analyze causes of hallucinations and identify four potential factors contributing to hallucinations. Finally, we propose an RAG-based mitigation method, which demonstrates consistent effectiveness in all studied LLMs. The replication package including code, data, and experimental results is available at https://github.com/DeepSoftwareAnalytics/LLMCodingHallucination
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
