arXiv 2505.231772025-05-29cs.CL

Infinite-Instruct：结合双向合成与静态验证合成缩放代码指令数据

Infinite-Instruct: Synthesizing Scaling Code instruction Data with Bidirectional Synthesis and Static Verification

ByteDance（字节跳动）；Dalian University of Technology（大连理工大学）

Wenjing Xing, Wenke Lu, Yeheng Duan, Bing Zhao, Zhenghui kang, Yaolong Wang, Kai Gao, Lei Qiao

AI总结：

提出Infinite-Instruct框架，通过反向构造、回溯构造和跨语言静态分析合成高质量代码指令数据，使7B和32B模型在代码生成基准上分别提升21.70%和36.95%，用不到十分之一数据达到Qwen-2.5-Coder-Instruct性能。

摘要：

传统代码指令数据合成方法存在多样性有限和逻辑性差的问题。我们提出了Infinite-Instruct，这是一个用于自动合成高质量问答对的框架，旨在增强大型语言模型（LLMs）的代码生成能力。该框架侧重于提升合成问题的内部逻辑和合成代码的质量。首先，“反向构造”将代码片段转化为多样的编程问题。接着，通过“回溯构造”，将编程问题中的关键词结构化为知识图谱，将其重构为内部逻辑更强的编程问题。最后，跨语言静态代码分析流水线过滤无效样本以确保数据质量。实验表明，在主流代码生成基准上，我们微调的模型在7B参数模型上实现了平均21.70%的性能提升，在32B参数模型上实现了36.95%的提升。仅使用不到十分之一的指令微调数据，我们便取得了与Qwen-2.5-Coder-Instruct相当的性能。Infinite-Instruct为编程领域的LLM训练提供了可扩展的解决方案。我们开源了实验中使用的数据集，包括未过滤版本和通过静态分析过滤的版本。数据可在https://github.com/xingwenjing417/Infinite-Instruct-dataset获取。

英文摘要：

Traditional code instruction data synthesis methods suffer from limited diversity and poor logic. We introduce Infinite-Instruct, an automated framework for synthesizing high-quality question-answer pairs, designed to enhance the code generation capabilities of large language models (LLMs). The framework focuses on improving the internal logic of synthesized problems and the quality of synthesized code. First, "Reverse Construction" transforms code snippets into diverse programming problems. Then, through "Backfeeding Construction," keywords in programming problems are structured into a knowledge graph to reconstruct them into programming problems with stronger internal logic. Finally, a cross-lingual static code analysis pipeline filters invalid samples to ensure data quality. Experiments show that on mainstream code generation benchmarks, our fine-tuned models achieve an average performance improvement of 21.70% on 7B-parameter models and 36.95% on 32B-parameter models. Using less than one-tenth of the instruction fine-tuning data, we achieved performance comparable to the Qwen-2.5-Coder-Instruct. Infinite-Instruct provides a scalable solution for LLM training in programming. We open-source the datasets used in the experiments, including both unfiltered versions and filtered versions via static analysis. The data are available at https://github.com/xingwenjing417/Infinite-Instruct-dataset

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。