UniCoder：通过通用代码扩展代码大语言模型
UniCoder: Scaling Code Large Language Model via Universal Code
AI总结：
针对标准思维链提示不适配代码生成任务的问题，本研究提出以混合编程语言约定的UniCode为中间表示，构建UniCoder-Instruct数据集训练UniCoder模型，显著提升代码生成质量，效果远超现有提示方法。
摘要：
中间推理或行动步骤已成功提升大语言模型（LLM）处理各类下游自然语言处理（NLP）任务的能力。在将LLM应用于代码生成时，现有研究主要聚焦于引导模型输出中间自然语言推理步骤（如思维链（CoT）提示），再基于自然语言或其他结构化中间步骤输出代码。但这类输出并不适配代码翻译或生成任务，因为标准CoT的逻辑结构与表达形式和代码存在差异。\n本研究提出通用代码（UniCode）作为中间表示，它是混合编程语言约定（如赋值运算符、条件运算符、循环）的算法步骤描述。我们构建了指令数据集UniCoder-Instruct，用于在多任务学习目标下训练模型UniCoder，该数据集包含自然语言问题、代码解决方案及对应的通用代码。中间通用代码表示与最终代码解决方案的对齐显著提升了生成代码的质量。\n实验结果表明，采用通用代码的UniCoder大幅优于此前的提示方法，验证了伪代码中结构线索的有效性。
英文摘要：
Intermediate reasoning or acting steps have successfully improved large language models (LLMs) for handling various downstream natural language processing (NLP) tasks. When applying LLMs for code generation, recent works mainly focus on directing the models to articulate intermediate natural-language reasoning steps, as in chain-of-thought (CoT) prompting, and then output code with the natural language or other structured intermediate steps. However, such output is not suitable for code translation or generation tasks since the standard CoT has different logical structures and forms of expression with the code. In this work, we introduce the universal code (UniCode) as the intermediate representation. It is a description of algorithm steps using a mix of conventions of programming languages, such as assignment operator, conditional operator, and loop. Hence, we collect an instruction dataset UniCoder-Instruct to train our model UniCoder on multi-task learning objectives. UniCoder-Instruct comprises natural-language questions, code solutions, and the corresponding universal code. The alignment between the intermediate universal code representation and the final code solution significantly improves the quality of the generated code. The experimental results demonstrate that UniCoder with the universal code significantly outperforms the previous prompting methods by a large margin, showcasing the effectiveness of the structural clues in pseudo-code.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
