arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.06853cs.CL
收藏

大型语言模型的历史、发展与原理——一项入门级综述

History, Development, and Principles of Large Language Models-An Introductory Survey

Florida International University(佛罗里达国际大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)

Zichong Wang, Zhibo Chu, Thang Viet Doan, Shiwen Ni, Min Yang, Wenbin Zhang

AI总结:

本综述追溯语言模型从统计模型到大型语言模型的历史演进,阐明其底层原理与关键发展因素,旨在为缺乏背景知识的从业者提供易于理解的理论概览,并指出未来研究方向。

摘要:

语言模型是自然语言处理(NLP)的基石,利用数学方法归纳语言规律与知识以进行预测和生成。历经数十年的广泛研究,语言建模已从早期的统计语言模型(SLMs)发展为如今的大型语言模型(LLMs)。值得注意的是,LLMs的快速演进已使其具备处理、理解和生成人类水平文本的能力。然而,尽管LLMs在改善工作与生活方面优势显著,但普通从业者对这些模型的背景和原理了解有限,阻碍了其潜力的充分发挥。此外,多数LLM综述聚焦特定领域并使用专业术语,给缺乏相关背景的从业者带来挑战。鉴于此,本综述旨在提供易于理解的LLMs概览,以帮助更广泛的受众。通过探索语言模型的历史背景并追溯其随时间的演进,本综述力求促进全面理解。文章进一步调查了影响LLMs发展的因素,并强调关键贡献。此外,本综述着重阐明LLMs的底层原理,为受众提供必要的理论知识。文章还指出了现有工作的局限性,并展望了未来的发展方向。

英文摘要:

Language models serve as a cornerstone in natural language processing (NLP), utilizing mathematical methods to generalize language laws and knowledge for prediction and generation. Over extensive research spanning decades, language modeling has progressed from initial statistical language models (SLMs) to the contemporary landscape of large language models (LLMs). Notably, the swift evolution of LLMs has reached the ability to process, understand, and generate human-level text. Nevertheless, despite the significant advantages that LLMs offer in improving both work and personal lives, the limited understanding among general practitioners about the background and principles of these models hampers their full potential. Notably, most LLM reviews focus on specific aspects and utilize specialized language, posing a challenge for practitioners lacking relevant background knowledge. In light of this, this survey aims to present a comprehensible overview of LLMs to assist a broader audience. It strives to facilitate a comprehensive understanding by exploring the historical background of language models and tracing their evolution over time. The survey further investigates the factors influencing the development of LLMs, emphasizing key contributions. Additionally, it concentrates on elucidating the underlying principles of LLMs, equipping audiences with essential theoretical knowledge. The survey also highlights the limitations of existing work and points out promising future directions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑