arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.01188cs.LGcs.DC
收藏

Petals:大模型的协同推理和微调

Petals: Collaborative Inference and Fine-tuning of Large Models

HSE University(高等经济大学);Yandex(扬德克斯);University of Washington(华盛顿大学);Hugging Face(拥抱脸);ENS Paris-Saclay(巴黎萨克雷高等师范学院);Yandex School of Data Analysis(扬德克斯数据分析学院)

Alexander Borzunov, Dmitry Baranchuk, Tim Dettmers, Max Ryabinin, Younes Belkada, Artem Chumachenko, Pavel Samygin, Colin Raffel

摘要:

许多NLP任务受益于使用大语言模型(LLM),这些模型通常具有超过1000亿个参数。随着BLOOM-176 B和OPT-175 B的发布,每个人都可以下载这种规模的预训练模型。尽管如此,使用这些模型需要许多研究人员无法获得的高端硬件。在某些情况下,LLM可以通过RAM卸载或托管API更实惠地使用。然而,这些技术有先天的局限性:卸载对于交互式推理来说太慢,而API对于需要访问权重,注意力或logit的研究来说不够灵活。在这项工作中,我们提出了Petals -一个通过加入多方资源来协作推理和微调大型模型的系统。我们证明,这种策略优于卸载非常大的模型,运行推理的BLOOM-176 B的消费者GPU与$\approx$ 1步每秒,这是足够的许多交互式LLM应用程序。与大多数推理API不同,Petals还原生地公开了服务模型的隐藏状态,允许基于有效的微调方法进行训练和共享自定义模型扩展。

英文摘要:

Many NLP tasks benefit from using large language models (LLMs) that often have more than 100 billion parameters. With the release of BLOOM-176B and OPT-175B, everyone can download pretrained models of this scale. Still, using these models requires high-end hardware unavailable to many researchers. In some cases, LLMs can be used more affordably via RAM offloading or hosted APIs. However, these techniques have innate limitations: offloading is too slow for interactive inference, while APIs are not flexible enough for research that requires access to weights, attention or logits. In this work, we propose Petals - a system for inference and fine-tuning of large models collaboratively by joining the resources of multiple parties. We demonstrate that this strategy outperforms offloading for very large models, running inference of BLOOM-176B on consumer GPUs with $\approx$ 1 step per second, which is enough for many interactive LLM applications. Unlike most inference APIs, Petals also natively exposes hidden states of served models, allowing to train and share custom model extensions based on efficient fine-tuning methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑