arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.14196cs.SEcs.CLcs.LG
收藏

DeepSeek-Coder:当大语言模型遇见编程——代码智能的兴起

DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence

DeepSeek-AI(深度求索人工智能);Key Lab of HCST (PKU), MOE(北京大学高可信软件技术教育部重点实验室;北京大学计算机学院);SCS, Peking University

Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, Wenfeng Liang

AI总结:

针对闭源代码模型限制研发的问题,研究团队推出参数1.3B到33B、基于2万亿token训练的开源DeepSeek-Coder系列模型,采用填空任务增强代码能力,性能超多数开源及部分闭源模型,支持研究与商用。

摘要:

大语言模型的快速发展革新了软件开发领域的代码智能。然而,闭源模型的主导地位限制了广泛的研发工作。为解决这一问题,我们推出DeepSeek-Coder系列,这是一组参数规模从1.3B到33B的开源代码模型,基于2万亿token从零训练而成。这些模型在高质量的项目级代码语料库上进行预训练,并采用带16K窗口的填空任务来增强代码生成与补全能力。我们的大量评估表明,DeepSeek-Coder不仅在多个基准上实现了开源代码模型中的最优性能,还超越了Codex、GPT-3.5等现有闭源模型。此外,DeepSeek-Coder模型采用宽松许可协议,支持研究用途与无限制商业使用。

英文摘要:

The rapid development of large language models has revolutionized code intelligence in software development. However, the predominance of closed-source models has restricted extensive research and development. To address this, we introduce the DeepSeek-Coder series, a range of open-source code models with sizes from 1.3B to 33B, trained from scratch on 2 trillion tokens. These models are pre-trained on a high-quality project-level code corpus and employ a fill-in-the-blank task with a 16K window to enhance code generation and infilling. Our extensive evaluations demonstrate that DeepSeek-Coder not only achieves state-of-the-art performance among open-source code models across multiple benchmarks but also surpasses existing closed-source models like Codex and GPT-3.5. Furthermore, DeepSeek-Coder models are under a permissive license that allows for both research and unrestricted commercial use.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑