arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.13971cs.CL
收藏

LLaMA:开放高效的基础语言模型

LLaMA: Open and Efficient Foundation Language Models

Meta AI
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, … 展开作者

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, Guillaume Lample

摘要:

我们介绍LLaMA,一个集合的基础语言模型,从7 B到65 B的参数。我们在数万亿个令牌上训练我们的模型,并表明完全可以使用公开的数据集来训练最先进的模型,而无需求助于专有和不可访问的数据集。特别是,LLaMA-13 B在大多数基准测试中的表现优于GPT-3(175 B),LLaMA-65 B与最好的模型Chinchilla-70 B和PaLM-540 B竞争。我们将所有的模型发布给研究界。

英文摘要:

We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑