TelecomGPT：构建电信领域专用大语言模型的框架
TelecomGPT: A Framework to Build Telecom-Specfic Large Language Models
AI总结：
针对主流大语言模型缺乏电信领域专业知识的问题，提出适配通用LLM为电信专用模型的流程，构建三类专用数据集与三类新基准，微调得到的TelecomGPT在多项电信任务上性能优异。
摘要：
大语言模型（LLM）具备革新第六代（6G）通信网络的潜力，但当前主流LLM普遍缺乏电信领域的专业知识。本文首次提出一套可将任意通用LLM适配为电信专用LLM的流程。我们收集并构建了电信专用预训练数据集、指令数据集、偏好数据集，分别用于持续预训练、指令微调与对齐微调。此外，由于电信领域缺乏广泛认可的评估基准，我们扩展了现有评估基准，并提出三个新基准，即Telecom Math Modeling（电信数学建模）、Telecom Open QnA（电信开放问答）和Telecom Code Tasks（电信代码任务）。这些新基准可全面评估LLM在电信领域的数学建模、开放式问答、代码生成、补全、摘要与分析能力。我们微调得到的LLM TelecomGPT在Telecom Math Modeling基准上显著优于包括GPT-4、Llama-3和Mistral在内的当前最优（SOTA）LLM，并在TeleQnA、3GPP技术文档分类、电信代码摘要与生成及补全等多项评估基准上取得了相当的性能。
英文摘要：
Large Language Models (LLMs) have the potential to revolutionize the Sixth Generation (6G) communication networks. However, current mainstream LLMs generally lack the specialized knowledge in telecom domain. In this paper, for the first time, we propose a pipeline to adapt any general purpose LLMs to a telecom-specific LLMs. We collect and build telecom-specific pre-train dataset, instruction dataset, preference dataset to perform continual pre-training, instruct tuning and alignment tuning respectively. Besides, due to the lack of widely accepted evaluation benchmarks in telecom domain, we extend existing evaluation benchmarks and proposed three new benchmarks, namely, Telecom Math Modeling, Telecom Open QnA and Telecom Code Tasks. These new benchmarks provide a holistic evaluation of the capabilities of LLMs including math modeling, Open-Ended question answering, code generation, infilling, summarization and analysis in telecom domain. Our fine-tuned LLM TelecomGPT outperforms state of the art (SOTA) LLMs including GPT-4, Llama-3 and Mistral in Telecom Math Modeling benchmark significantly and achieve comparable performance in various evaluation benchmarks such as TeleQnA, 3GPP technical documents classification, telecom code summary and generation and infilling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
