arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.12504cs.CL
收藏

Case2Code:用于代码生成的可扩展合成数据

Case2Code: Scalable Synthetic Data for Code Generation

School of Computer Science, Fudan University(复旦大学计算机科学学院);Shanghai AI Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学)

Yunfan Shao, Linyang Li, Yichuan Ma, Peiji Li, Demin Song, Qinyuan Cheng, Shimin Li, Xiaonan Li, Pengyu Wang, Qipeng Guo, Hang Yan, Xipeng Qiu, Xuanjing Huang, Dahua Lin

AI总结:

针对现有基于教师模型的合成数据难以扩展的问题,本文提出Case2Code归纳推理任务,结合LLMs生成程序输入并执行获取输出以大规模合成数据,训练后的模型在相关任务上性能提升,验证了该方法的潜力。

摘要:

大型语言模型(LLMs)在代码生成领域展现出显著突破。近期研究通过在由强大LLMs生成的合成数据上训练来改进代码LLMs,但这种方法因依赖教师模型且生成成本高而难以扩展。本文聚焦大规模合成代码数据,利用程序的表达性和正确性提出Case2Code任务。Case2Code是一种归纳推理任务,旨在通过观察输入输出示例或程序行为推断底层代码实现。我们结合LLMs生成程序输入,执行程序以获取输出,从而大规模合成多样化、高质量的Case2Code数据用于训练和评估代码LLMs。实验表明,未经训练的当前代表性LLMs在Case2Code归纳任务上表现不佳;经Case2Code数据训练的模型不仅提升了分布内Case2Code归纳性能,还在多种代码生成任务中表现更优,证明了大规模合成数据和归纳学习的巨大潜力。

英文摘要:

Large Language Models (LLMs) have shown outstanding breakthroughs in code generation. Recent work improves code LLMs by training on synthetic data generated by some powerful LLMs, which can be challenging to scale due to the dependence on a teacher model and high generation costs. In this paper, we focus on synthesizing code data at scale and propose a \textbf{Case2Code} task by exploiting the expressiveness and correctness of programs. \textbf{Case2Code} is an inductive inference task that aims to infer underlying code implementations by observing input-output examples or program behaviors, By incorporating LLMs to generate program inputs, and executing the program with these inputs to obtain the program outputs, we can synthesize diverse and high-quality \textbf{Case2Code} data at scale for training and evaluating code LLMs. Experimental results show that case-to-code induction is challenging for current representative LLMs if they are untrained. Models trained with \textbf{Case2Code} improve performance not only on distribution case-to-code induction but also on various coding-generation tasks, demonstrating the great potential of large-scale synthetic data and inductive learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑