In this report, we introduce the Qwen2.5-Coder series, a significant upgrade from its predecessor, CodeQwen1.5. This series includes six models: Qwen2.5-Coder-(0.5B/1.5B/3B/7B/14B/32B). As a code-specific model, Qwen2.5-Coder is built upon the Qwen2.5 architecture and continues pretrained on a vast corpus of over 5.5 trillion tokens. Through meticulous data cleaning, scalable synthetic data generation, and balanced data mixing, Qwen2.5-Coder demonstrates impressive code generation capabilities while retaining general and math skills. These models have been evaluated on a wide range of code-related tasks, achieving state-of-the-art (SOTA) performance across more than 10 benchmarks, including code generation, completion, reasoning, and repair, consistently outperforming larger models of the same model size. We believe that the release of the Qwen2.5-Coder series will advance research in code intelligence and, with its permissive licensing, support wider adoption by developers in real-world applications.
Grammar Induction for Minimalist Grammars using Variational Bayesian Inference : A Technical Report
Stanford University(斯坦福大学);McGill University(麦吉尔大学);École Polytechnique Fédérale de Lausanne(洛桑联邦理工学院);University of California, San Diego(加利福尼亚大学圣地亚哥分校)
Technical report on Conversational Question Answering
ZhuiYi Technology(追一科技)
会话问答是一项具有挑战性的任务,因为它需要了解会话历史。在这个项目中,我们提出了一个新的系统RoBERTa + AT +KD,它涉及理性标记多任务,对抗训练,知识蒸馏和语言后处理策略。在没有数据增强的情况下,我们的单一模型在CoQA测试集上达到90.4(F1),比当前最先进的单一模型高出2.6%F1。
On the Effects of Low-Quality Training Data on Information Extraction from Clinical Reports
Istituto di Scienza e Tecnologie dell’Informazione, Consiglio Nazionale delle Ricerche(意大利国家研究委员会信息科学与技术研究所);Qatar Computing Research Institute, Qatar Foundation(卡塔尔基金会卡塔尔计算研究所);Consiglio Nazionale delle Ricerche(意大利国家研究委员会)