arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.12186cs.CL
收藏

Qwen2.5-Coder技术报告

Qwen2.5-Coder Technical Report

Alibaba Group(阿里巴巴集团);Qwen Team(通义千问团队)
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang… 展开作者

Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, Yunlong Feng, Xingzhang Ren, Xuancheng Ren, Jingren Zhou, Junyang Lin

AI总结:

本文介绍Qwen2.5-Coder系列模型,基于Qwen2.5架构并在5.5万亿token上预训练,通过精细数据清洗与合成数据生成,在10多个代码基准测试中达到SOTA性能,推动代码智能研究与应用。

摘要:

在本报告中,我们介绍了Qwen2.5-Coder系列,这是对其前身CodeQwen1.5的重大升级。该系列包括六个模型:Qwen2.5-Coder-(0.5B/1.5B/3B/7B/14B/32B)。作为代码专用模型,Qwen2.5-Coder基于Qwen2.5架构构建,并在超过5.5万亿个token的庞大语料库上继续预训练。通过精细的数据清洗、可扩展的合成数据生成和均衡的数据混合,Qwen2.5-Coder展现出令人印象深刻的代码生成能力,同时保留了通用和数学技能。这些模型已在广泛的代码相关任务上进行了评估,在代码生成、补全、推理和修复等10多个基准测试中取得了最先进(SOTA)的性能,持续超越相同模型规模的更大模型。我们相信,Qwen2.5-Coder系列的发布将推动代码智能的研究,并凭借其宽松的许可证,支持开发者在实际应用中更广泛地采用。

英文摘要:

In this report, we introduce the Qwen2.5-Coder series, a significant upgrade from its predecessor, CodeQwen1.5. This series includes six models: Qwen2.5-Coder-(0.5B/1.5B/3B/7B/14B/32B). As a code-specific model, Qwen2.5-Coder is built upon the Qwen2.5 architecture and continues pretrained on a vast corpus of over 5.5 trillion tokens. Through meticulous data cleaning, scalable synthetic data generation, and balanced data mixing, Qwen2.5-Coder demonstrates impressive code generation capabilities while retaining general and math skills. These models have been evaluated on a wide range of code-related tasks, achieving state-of-the-art (SOTA) performance across more than 10 benchmarks, including code generation, completion, reasoning, and repair, consistently outperforming larger models of the same model size. We believe that the release of the Qwen2.5-Coder series will advance research in code intelligence and, with its permissive licensing, support wider adoption by developers in real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑