Petals: Collaborative Inference and Fine-tuning of Large Models
HSE University(高等经济大学);Yandex(扬德克斯);University of Washington(华盛顿大学);Hugging Face(拥抱脸);ENS Paris-Saclay(巴黎萨克雷高等师范学院);Yandex School of Data Analysis(扬德克斯数据分析学院)
Alexander Borzunov, Dmitry Baranchuk, Tim Dettmers, Max Ryabinin, Younes Belkada, Artem Chumachenko, Pavel Samygin, Colin Raffel
备注:
10 pages, 4 figures. The version 2 updates the benchmarks and the description of the chat application. Source code and docs: https://petals.ml
Many NLP tasks benefit from using large language models (LLMs) that often have more than 100 billion parameters. With the release of BLOOM-176B and OPT-175B, everyone can download pretrained models of this scale. Still, using these models requires high-end hardware unavailable to many researchers. In some cases, LLMs can be used more affordably via RAM offloading or hosted APIs. However, these techniques have innate limitations: offloading is too slow for interactive inference, while APIs are not flexible enough for research that requires access to weights, attention or logits. In this work, we propose Petals - a system for inference and fine-tuning of large models collaboratively by joining the resources of multiple parties. We demonstrate that this strategy outperforms offloading for very large models, running inference of BLOOM-176B on consumer GPUs with $\approx$ 1 step per second, which is enough for many interactive LLM applications. Unlike most inference APIs, Petals also natively exposes hidden states of served models, allowing to train and share custom model extensions based on efficient fine-tuning methods.
Resource-Efficient Personal Large Language Models Fine-Tuning with Collaborative Edge Computing
School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院);Department of Information Engineering, The Chinese University of Hong Kong(香港中文大学信息工程系);The Hong Kong University of Science and Technology (Guangzhou)(香港理工大学(广州));Shenzhen Institute of Artificial Intelligence and Robotics for Society(社会人工智能与机器人研究所(深圳))
Connecting the Dots: Collaborative Fine-tuning for Black-Box Vision-Language Models
University of Science and Technology of China(中国科学技术大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Nanjing University(南京大学)
Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference
The University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of California, Los Angeles(加州大学洛杉矶分校);University of Minnesota Twin Cities(明尼苏达大学双城分校);The Chinese University of Hong Kong(香港中文大学)
A Structure-Agnostic Co-Tuning Framework for LLMs and SLMs in Cloud-Edge Systems
Swinburne University of Technology(斯威本理工大学);Tongji University(同济大学);Wuhan University of Technology(武汉理工大学);INFLY TECH (Shanghai) Co., Ltd.(INFLY TECH(上海)有限公司);Wuhan University(武汉大学);RMIT University(皇家墨尔本理工大学)
SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
School of Computer Science, Fudan University(复旦大学计算机科学学院);University of Hong Kong(香港大学);University of Maryland(马里兰大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Massachusetts Institute of Technology(麻省理工学院)
Collaborative Multi-LoRA Experts with Achievement-based Multi-Tasks Loss for Unified Multimodal Information Extraction
School of Software Engineering, South China University of Technology, Guangzhou, China(华南理工大学软件学院);Key Laboratory of Big Data and Intelligent Robot (SCUT), MOE of China(大数据与智能机器人重点实验室);Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China(香港理工大学计算机系);School of Electrical Engineering, Guangxi University, Nanning, China(广西大学电气工程学院);Department of Biostatistics & Health Informatics, King’s College London, London, United Kingdom(伦敦国王学院生物统计与健康信息学系)
CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China(东北大学计算机科学与工程学院);Tsinghua University(清华大学);Future Living Lab of Alibaba(阿里巴巴未来生活实验室)
FedPEAT: Convergence of Federated Learning, Parameter-Efficient Fine Tuning, and Emulator Assisted Tuning for Artificial Intelligence Foundation Models with Mobile Edge Computing