arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.07730cs.CL
收藏

Transformer模型:介绍与目录

Transformer models: an introduction and catalog

LinkedIn(领英)

Xavier Amatriain, Ananth Sankar, Jie Bing, Praveen Kumar Bodigutla, Timothy J. Hazen, Michaeel Kazi

AI总结:

本文对主流Transformer模型进行了系统分类与介绍,涵盖自监督学习和人在回路训练两类模型,为理解该领域发展提供了简明参考。

摘要:

在过去几年中,我们见证了数十个Transformer家族基础模型的迅速涌现,它们都有令人难忘、有时有趣但并不自解释的名称。本文的目标是对最流行的Transformer模型提供一个较为全面但简洁的目录和分类。本文还包括对Transformer模型中最重要的方面和创新的介绍。我们的目录将包括使用自监督学习训练的模型(如BERT或GPT3),以及那些进一步使用人在回路训练的模型(如ChatGPT使用的InstructGPT模型)。

英文摘要:

In the past few years we have seen the meteoric appearance of dozens of foundation models of the Transformer family, all of which have memorable and sometimes funny, but not self-explanatory, names. The goal of this paper is to offer a somewhat comprehensive but simple catalog and classification of the most popular Transformer models. The paper also includes an introduction to the most important aspects and innovations in Transformer models. Our catalog will include models that are trained using self-supervised learning (e.g., BERT or GPT3) as well as those that are further trained using a human-in-the-loop (e.g. the InstructGPT model used by ChatGPT).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑