PDC与DM-SFT：增强LLM SQL Bug修复之路
PDC & DM-SFT: A Road for LLM SQL Bug-Fix Enhancing
AI总结：
针对代码LLM在SQL bug修复上的不足，提出PDC数据集构建与DM-SFT微调方法，有效减少训练步数并缓解训练迷失问题，使较小模型超越更大规模的最佳模型。
摘要：
代码大型语言模型（Code LLMs），如Code llama和DeepSeek-Coder，在代码生成任务中表现优异。然而，现有模型多关注生成正确代码的能力，常在bug修复上遇到困难。我们提出了一套增强LLM SQL bug修复能力的方法，主要包括两部分：从零开始的渐进式数据集构建（PDC）和动态掩码监督微调（DM-SFT）。PDC从广度优先和深度优先两个角度提出两种数据扩展方法。DM-SFT引入高效的bug修复监督学习方法，有效减少总训练步数并缓解SQL代码bug修复训练中的“迷失方向”问题。评估中，使用这两种方法训练的代码LLM模型超越了所有当前最佳性能的更大规模模型。
英文摘要：
Code Large Language Models (Code LLMs), such as Code llama and DeepSeek-Coder, have demonstrated exceptional performance in the code generation tasks. However, most existing models focus on the abilities of generating correct code, but often struggle with bug repair. We introduce a suit of methods to enhance LLM's SQL bug-fixing abilities. The methods are mainly consisted of two parts: A Progressive Dataset Construction (PDC) from scratch and Dynamic Mask Supervised Fine-tuning (DM-SFT). PDC proposes two data expansion methods from the perspectives of breadth first and depth first respectively. DM-SFT introduces an efficient bug-fixing supervised learning approach, which effectively reduce the total training steps and mitigate the "disorientation" in SQL code bug-fixing training. In our evaluation, the code LLM models trained with two methods have exceeds all current best performing model which size is much larger.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
