WaveCoder：通过指令微调实现代码大语言模型的广泛且多功能增强
WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuning
AI总结：
本文提出 WaveCoder 系列代码大语言模型，通过多任务指令微调方法和包含 19,915 条指令的 CodeSeaXDataset 数据集，显著提升了模型在多种代码相关任务上的泛化能力。
摘要：
近期研究表明，经过指令微调后，代码大语言模型（Code LLMs）能够获得处理多种代码相关任务的出色能力。然而，当前针对代码大语言模型的指令微调方法主要集中于传统的代码生成任务，导致其在复杂多任务场景下表现不佳。本文聚焦于多种代码相关任务，并提出 WaveCoder，这是一系列使用广泛且多功能增强指令数据训练而成的代码大语言模型。为使模型能够应对复杂的代码相关任务，我们提出了一种在多任务场景下从开源代码数据集中稳定生成多样化、高质量指令数据的方法，并由此构建了 CodeSeaXDataset，该数据集包含 19,915 条指令实例，覆盖 4 类代码相关任务，旨在提升代码大语言模型的泛化能力。实验结果表明，WaveCoder 模型在不同代码相关任务上的泛化能力显著优于其他开源模型。此外，WaveCoder-Ultra-6.7B 在广泛的代码相关任务上展现出最先进的泛化能力。
英文摘要：
Recent work demonstrates that, after instruction tuning, Code Large Language Models (Code LLMs) can obtain impressive capabilities to address a wide range of code-related tasks. However, current instruction tuning methods for Code LLMs mainly focus on the traditional code generation task, resulting in poor performance in complex multi-task scenarios. In this paper, we concentrate on multiple code-related tasks and present WaveCoder, a series of Code LLMs trained with Widespread And Versatile Enhanced instruction data. To enable the models to tackle complex code-related tasks, we propose a method to stably generate diverse, high-quality instruction data from open source code dataset in multi-task scenarios and obtain CodeSeaXDataset, a dataset comprising 19,915 instruction instances across 4 code-related tasks, which is aimed at improving the generalization ability of Code LLM. Our experiments demonstrate that WaveCoder models significantly outperform other open-source models in terms of the generalization ability across different code-related tasks. Moreover, WaveCoder-Ultra-6.7B presents the state-of-the-art generalization abilities on a wide range of code-related tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
