arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.14658cs.SEcs.AIcs.CL
收藏

OpenCodeInterpreter:将代码生成与执行和精修相集成

OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement

Multimodal Art Projection Research Community(多模态艺术投影研究共同体);University of Waterloo(滑铁卢大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);HKUST(香港科技大学);IN.AI Research(IN.AI研究机构)

Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, Xiang Yue

AI总结:

本文提出开源代码系统 OpenCodeInterpreter,借助含 68K 多轮交互的 Code-Feedback 数据集整合执行与人类反馈进行迭代精修,在多项代码基准上接近 GPT-4 Code Interpreter。

摘要:

大型语言模型的引入显著推动了代码生成的发展。然而,开源模型往往缺乏 GPT-4 Code Interpreter 等先进系统所具备的执行能力和迭代精修能力。为解决这一问题,我们提出 OpenCodeInterpreter,这是一系列开源代码系统,旨在生成、执行并迭代精修代码。在包含 68K 多轮交互的数据集 Code-Feedback 的支持下,OpenCodeInterpreter 整合执行反馈与人类反馈,用于动态代码精修。我们在 HumanEval、MBPP 以及来自 EvalPlus 的增强版本等关键基准上对 OpenCodeInterpreter 进行了全面评估,结果显示其性能出色。值得注意的是,OpenCodeInterpreter-33B 在 HumanEval 和 MBPP 的平均版本(及 plus 版本)上取得 83.2(76.4)的准确率,与 GPT-4 的 84.2(76.2)十分接近;在使用来自 GPT-4 的合成人类反馈后,其成绩进一步提升至 91.6(84.6)。OpenCodeInterpreter 缩小了开源代码生成模型与 GPT-4 Code Interpreter 等专有系统之间的差距。

英文摘要:

The introduction of large language models has significantly advanced code generation. However, open-source models often lack the execution capabilities and iterative refinement of advanced systems like the GPT-4 Code Interpreter. To address this, we introduce OpenCodeInterpreter, a family of open-source code systems designed for generating, executing, and iteratively refining code. Supported by Code-Feedback, a dataset featuring 68K multi-turn interactions, OpenCodeInterpreter integrates execution and human feedback for dynamic code refinement. Our comprehensive evaluation of OpenCodeInterpreter across key benchmarks such as HumanEval, MBPP, and their enhanced versions from EvalPlus reveals its exceptional performance. Notably, OpenCodeInterpreter-33B achieves an accuracy of 83.2 (76.4) on the average (and plus versions) of HumanEval and MBPP, closely rivaling GPT-4's 84.2 (76.2) and further elevates to 91.6 (84.6) with synthesized human feedback from GPT-4. OpenCodeInterpreter brings the gap between open-source code generation models and proprietary systems like GPT-4 Code Interpreter.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑