TigerBot:一个开源的多语言多任务大语言模型
TigerBot: An Open Multilingual Multitask LLM
AI总结:
本研究发布了基于Llama-2和BLOOM开发的多规模开源多语言多任务TigerBot大模型家族,在中英等语言性能上优于Llama-2等SOTA开源模型,旨在推动大模型的普惠构建与落地应用。
摘要:
我们发布并介绍TigerBot大语言模型(LLM)家族,它包含基础模型和对话模型,参数规模涵盖70亿、130亿、700亿和1800亿。我们以Llama-2和BLOOM为起点开发模型,在数据、训练算法、基础设施和应用工具方面进一步突破边界。我们的模型相比现有SOTA开源模型(如Llama-2)取得了显著的性能提升,具体而言英文性能提升6%,中文性能提升20%。TigerBot模型家族在主流学术和工业基准及排行榜上也取得了领先性能。我们认为TigerBot只是大语言模型开源社区飞速发展的一个缩影,因此我们很高兴通过公开发布模型并披露背后的方法来回馈社区,重点关注以普惠化方式构建SOTA大语言模型,并推动大语言模型在真实场景中的应用。
英文摘要:
We release and introduce the TigerBot family of large language models (LLMs), consisting of base and chat models, sized from 7, 13, 70 and 180 billion parameters. We develop our models embarking from Llama-2 and BLOOM, and push the boundary further in data, training algorithm, infrastructure, and application tools. Our models yield meaningful performance gain over SOTA open-source models, e.g., Llama-2, specifically 6% gain in English and 20% gain in Chinese. TigerBot model family also achieves leading performance in major academic and industrial benchmarks and leaderboards. We believe that TigerBot represents just a snapshot of lightning-fast progression in LLM open-source community. Therefore, we are thrilled to give back by publicly releasing our models and reporting our approach behind, with additional emphases on building SOTA LLMs in a democratized way and making LLMs of use in real-world applications.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。