基于语言模型与解耦领域数据库的领域自适应代码补全
Domain Adaptive Code Completion via Language Models and Decoupled Domain Databases
AI总结：
本文提出检索增强语言模型 kNM-LM，通过解耦领域数据库与贝叶斯推理实现无需微调的领域自适应代码补全，并可作为插件增强黑盒模型。
摘要：
大型语言模型（LLMs）在代码补全方面已展现出卓越性能。然而，由于缺乏领域特定知识，它们在补全需要密集领域知识的代码时可能并非最优，例如补全库名称。尽管已有若干工作证实了微调技术在使语言模型适应特定领域代码补全方面的有效性，但它们受限于当项目持续迭代时需要不断对模型进行微调。为解决这一局限，本文提出 kNM-LM，一种检索增强语言模型（R-LM），无需微调即可将领域知识集成到语言模型中。与以往技术不同，我们的方法能够自动适应不同的语言模型和领域。具体而言，它利用领域内代码构建与语言模型解耦的基于检索的数据库，然后通过贝叶斯推理将其与语言模型结合以完成代码补全。在项目内和场景内补全上的大量实验证实，与 CodeGPT 和 UnixCoder 相比，kNM-LM 带来了可观的提升。对我们工具的深入分析，包括响应速度、存储占用、特定类型代码补全以及 API 调用补全，证实 kNM-LM 提供了令人满意的性能，使其非常适合领域自适应代码补全。此外，我们的方法在运行时无需直接访问语言模型的参数。因此，它可以无缝集成黑盒代码补全模型，便于将我们的方法作为插件集成以进一步增强这些模型的性能。
英文摘要：
Large Language Models (LLMs) have demonstrated remarkable performance in code completion. However, due to the lack of domain-specific knowledge, they may not be optimal in completing code that requires intensive domain knowledge for example completing the library names. Although there are several works that have confirmed the effectiveness of fine-tuning techniques to adapt language models for code completion in specific domains. They are limited by the need for constant fine-tuning of the model when the project is in constant iteration. To address this limitation, in this paper, we propose $k$NM-LM, a retrieval-augmented language model (R-LM), that integrates domain knowledge into language models without fine-tuning. Different from previous techniques, our approach is able to automatically adapt to different language models and domains. Specifically, it utilizes the in-domain code to build the retrieval-based database decoupled from LM, and then combines it with LM through Bayesian inference to complete the code. The extensive experiments on the completion of intra-project and intra-scenario have confirmed that $k$NM-LM brings about appreciable enhancements when compared to CodeGPT and UnixCoder. A deep analysis of our tool including the responding speed, storage usage, specific type code completion, and API invocation completion has confirmed that $k$NM-LM provides satisfactory performance, which renders it highly appropriate for domain adaptive code completion. Furthermore, our approach operates without the requirement for direct access to the language model's parameters. As a result, it can seamlessly integrate with black-box code completion models, making it easy to integrate our approach as a plugin to further enhance the performance of these models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
