TeleChat 技术报告
TeleChat Technical Report
AI总结：
本文报告了 TeleChat 系列大语言模型（3B/7B/12B）的预训练、人类偏好对齐微调及多任务评估结果，其性能与同规模开源模型相当，并开源了 7B 和 12B 模型检查点、代码及部分预训练数据。
摘要：
在本技术报告中，我们介绍了 TeleChat，这是一组参数量分别为 30 亿、70 亿和 120 亿的大型语言模型（LLMs）集合。它既包括预训练语言模型，也包括经过微调、与人类偏好对齐的聊天模型。TeleChat 最初在一个包含数万亿 token 的广泛语料库上进行预训练，该语料库涵盖英语和中文的多样化文本。随后，模型按照我们所描述的详细方法进行微调，以与人类偏好对齐。我们评估了 TeleChat 在多种任务上的表现，包括语言理解、数学、推理、代码生成和基于知识的问答。我们的研究结果表明，在广泛的公开基准测试中，TeleChat 取得了与相似规模的其他开源模型相当的性能。为了支持未来利用 LLMs 的研究和应用，我们向公众社区发布了 TeleChat 的 7B 和 12B 版本的微调模型检查点，以及代码和部分预训练数据。
英文摘要：
In this technical report, we present TeleChat, a collection of large language models (LLMs) with parameters of 3 billion, 7 billion and 12 billion. It includes pretrained language models as well as fine-tuned chat models that is aligned with human preferences. TeleChat is initially pretrained on an extensive corpus containing a diverse collection of texts from both English and Chinese languages, including trillions of tokens. Subsequently, the model undergoes fine-tuning to align with human preferences, following a detailed methodology that we describe. We evaluate the performance of TeleChat on various tasks, including language understanding, mathematics, reasoning, code generation, and knowledge-based question answering. Our findings indicate that TeleChat achieves comparable performance to other open-source models of similar size across a wide range of public benchmarks. To support future research and applications utilizing LLMs, we release the fine-tuned model checkpoints of TeleChat's 7B and 12B variant, along with code and a portion of our pretraining data, to the public community.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
