arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.07549cs.CL
收藏

作为自然逻辑枢纽的注释:通过注释视角提升代码生成

Comments as Natural Logic Pivots: Improve Code Generation via Comment Perspective

Beijing Jiaotong University(北京交通大学);Pattern Recognition Center, WeChat AI, Tencent Inc(腾讯公司微信AI模式识别中心)

Yijie Chen, Yijin Liu, Fandong Meng, Yufeng Chen, Jinan Xu, Jie Zhou

AI总结:

针对现有代码生成方法依赖大模型、中小规模代码LLM能力受限的问题,提出以代码注释为自然逻辑枢纽的MANGO框架,通过注释对比训练与逻辑注释解码策略,在3B-7B参数模型上显著提升代码通过率与鲁棒性。

摘要:

代码生成旨在理解问题描述并生成相应的代码片段,现有研究通常通过提示策略将这类复杂任务分解为中间步骤,例如思维链(Chain-of-Thought)及其变体。尽管这些研究取得了一定成功,但其效果高度依赖GPT-4等先进大语言模型(LLM)的能力,尤其是在API调用方面,这极大地限制了它们的实际适用性。因此,如何在不显著增加训练成本的情况下提升中小规模代码大语言模型的代码生成能力,是一项颇具吸引力的挑战。本文提出,代码注释是自然语言与代码语言之间的天然逻辑枢纽,并提出利用注释来增强代码大语言模型的代码生成能力。具体而言,我们提出了MANGO(comMents As Natural loGic pivOts,注释作为自然逻辑枢纽)框架,包含注释对比训练策略和相应的逻辑注释解码策略。我们在HumanEval和MBPP数据集上开展实验,以StarCoder和WizardCoder作为骨干模型,涵盖3B到7B之间的模型参数规模。结果表明,MANGO在强基线的基础上显著提升了代码通过率。同时,逻辑注释解码策略的鲁棒性明显高于思维链提示。代码已公开于https://github.com/pppa2019/Mango。

英文摘要:

Code generation aims to understand the problem description and generate corresponding code snippets, where existing works generally decompose such complex tasks into intermediate steps by prompting strategies, such as Chain-of-Thought and its variants. While these studies have achieved some success, their effectiveness is highly dependent on the capabilities of advanced Large Language Models (LLMs) such as GPT-4, particularly in terms of API calls, which significantly limits their practical applicability. Consequently, how to enhance the code generation capabilities of small and medium-scale code LLMs without significantly increasing training costs is an appealing challenge. In this paper, we suggest that code comments are the natural logic pivot between natural language and code language and propose using comments to boost the code generation ability of code LLMs. Concretely, we propose MANGO (comMents As Natural loGic pivOts), including a comment contrastive training strategy and a corresponding logical comment decoding strategy. Experiments are performed on HumanEval and MBPP, utilizing StarCoder and WizardCoder as backbone models, and encompassing model parameter sizes between 3B and 7B. The results indicate that MANGO significantly improves the code pass rate based on the strong baselines. Meanwhile, the robustness of the logical comment decoding strategy is notably higher than the Chain-of-thoughts prompting. The code is publicly available at \url{https://github.com/pppa2019/Mango}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑