学习解决与验证:一种用于代码和测试生成的自博弈框架
Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation
AI总结:
本研究提出Sol-Ver框架,通过自博弈方式提升代码和测试生成能力,无需人工标注或大模型,实现代码生成和测试生成的性能提升。
摘要:
近年来,大型语言模型(LLMs)在编码基准测试中的性能有所提高。然而,由于高质量数据的枯竭,进步趋于停滞。先前的工作展示了合成自指导数据的潜力,但直接在模型自身的输出上进行训练可能会导致错误积累,尤其是在编码任务中,由于训练数据过于简单或错误,泛化能力可能崩溃,突显了对合成数据进行严格质量检查的必要性。在本工作中,我们探索了一种有效的方法,即让模型自身验证其自身数据的正确性。因此,我们提出了Sol-Ver,一种自博弈求解器-验证器框架,旨在共同提升单个模型的代码和测试生成能力。通过迭代地改进代码(LLM-as-a-solver)和测试(LLM-as-a-verifier)一起,我们增强了两种能力,而无需依赖人工标注或更大的教师模型。使用Llama 3.1 8B模型进行的实验显示了显著的性能提升,在MBPP和LiveCodeBench上,代码生成的平均相对提升为19.63%,测试生成的平均相对提升为17.49%。
英文摘要:
Recent advances in large language models (LLMs) have improved their performance on coding benchmarks. However, improvement is plateauing due to the exhaustion of readily available high-quality data. Prior work has shown the potential of synthetic self-instruct data, but naively training on a model's own outputs can cause error accumulation, especially in coding tasks, where generalization may collapse due to overly simple or erroneous training data, highlighting the need for rigorous quality checks on synthetic data. In this work, we explore an effective approach whereby the model itself verifies the correctness of its own data. We thus propose Sol-Ver, a self-play solver-verifier framework that jointly improves a single model's code and test generation capacity. By iteratively refining code (LLM-as-a-solver) and tests (LLM-as-a-verifier) together, we boost both capabilities without relying on human annotations or larger teacher models. Experiments with the Llama 3.1 8B model demonstrate substantial performance enhancements, achieving average relative improvements of 19.63% in code generation and 17.49% in test generation on MBPP and LiveCodeBench.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。