大型语言模型中知识的生命周期:综述
The Life Cycle of Knowledge in Big Language Models: A Survey
AI总结:
本文综述 PLM 中知识的完整生命周期,将其划分为五个阶段并系统梳理知识构建、维护与使用的相关研究、挑战及未来方向。
摘要:
知识在人工智能中发挥着关键作用。近年来,预训练语言模型(PLM)取得的广泛成功,引发了人们对语言模型如何获取、维护、更新和使用知识的高度关注。尽管已有大量相关研究,但对于知识在学习、微调和应用过程中如何在语言模型内部流转,仍缺乏统一视角;这可能阻碍我们进一步理解当前进展之间的联系,或认识现有局限。在本综述中,我们将 PLM 重新视为基于知识的系统,把 PLM 中知识的生命周期划分为五个关键阶段,并考察知识在被构建、维护和使用时如何流转。为此,我们系统回顾了知识生命周期各阶段的现有研究,总结主要挑战和当前局限,并讨论未来方向。
英文摘要:
Knowledge plays a critical role in artificial intelligence. Recently, the extensive success of pre-trained language models (PLMs) has raised significant attention about how knowledge can be acquired, maintained, updated and used by language models. Despite the enormous amount of related studies, there still lacks a unified view of how knowledge circulates within language models throughout the learning, tuning, and application processes, which may prevent us from further understanding the connections between current progress or realizing existing limitations. In this survey, we revisit PLMs as knowledge-based systems by dividing the life circle of knowledge in PLMs into five critical periods, and investigating how knowledge circulates when it is built, maintained and used. To this end, we systematically review existing studies of each period of the knowledge life cycle, summarize the main challenges and current limitations, and discuss future directions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。