arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2306.07933cs.CLcs.AI
收藏

通过大语言模型理解电信领域语言

Understanding Telecom Language Through Large Language Models

Technology Innovation Institute(技术创新研究院)

Lina Bariah, Hang Zou, Qiyang Zhao, Belkacem Mouhouche, Faouzi Bader, Merouane Debbah

AI总结:

本研究将大语言模型适配电信领域,微调BERT、RoBERTa等模型识别3GPP工作组,最高准确率达84.6%,为生成式AI在电信领域应用及自演进网络实现奠定基础。

摘要:

人工智能(AI)的最新进展为电信网络设计、实施与部署中诸多任务的自动化开辟了新前沿。生成式AI的演进(包括大语言模型(LLMs)的出现)进一步推动了这一趋势,这类模型被认为是实现自治、交互式AI智能体的基石。受此启发,本文旨在将LLMs范式适配至电信领域。具体而言,我们对包括BERT、蒸馏BERT、RoBERTa和GPT-2在内的多种LLM进行电信领域语言的微调,并展示了一个识别第三代合作伙伴计划(3GPP)标准工作组的应用案例。我们使用2009-2019年的3GPP技术文档(Tdoc)训练所选模型,并对2020-2023年的Tdoc进行类别预测。结果显示,微调后的BERT和RoBERTa模型在识别3GPP工作组任务上达到84.6%的准确率,GPT-2模型则达到83%。参数量减少约50%的蒸馏BERT模型取得了与其他模型相近的性能。这证实了对预训练LLM进行微调可有效识别电信语言类别。所构建的框架是从电信语言迈向意图驱动、自演进无线网络的一块垫脚石,也为生成式AI在电信领域的落地铺平了道路。

英文摘要:

The recent progress of artificial intelligence (AI) opens up new frontiers in the possibility of automating many tasks involved in Telecom networks design, implementation, and deployment. This has been further pushed forward with the evolution of generative artificial intelligence (AI), including the emergence of large language models (LLMs), which is believed to be the cornerstone toward realizing self-governed, interactive AI agents. Motivated by this, in this paper, we aim to adapt the paradigm of LLMs to the Telecom domain. In particular, we fine-tune several LLMs including BERT, distilled BERT, RoBERTa and GPT-2, to the Telecom domain languages, and demonstrate a use case for identifying the 3rd Generation Partnership Project (3GPP) standard working groups. We consider training the selected models on 3GPP technical documents (Tdoc) pertinent to years 2009-2019 and predict the Tdoc categories in years 2020-2023. The results demonstrate that fine-tuning BERT and RoBERTa model achieves 84.6% accuracy, while GPT-2 model achieves 83% in identifying 3GPP working groups. The distilled BERT model with around 50% less parameters achieves similar performance as others. This corroborates that fine-tuning pretrained LLM can effectively identify the categories of Telecom language. The developed framework shows a stepping stone towards realizing intent-driven and self-evolving wireless networks from Telecom languages, and paves the way for the implementation of generative AI in the Telecom domain.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑