DeServe:通过去中心化实现经济实惠的离线LLM推理
DeServe: Towards Affordable Offline LLM Inference via Decentralization
AI总结:
本文提出去中心化离线LLM推理系统DeServe,利用空闲GPU降低成本,并在高延迟网络中通过优化吞吐量实现6.7-12.6倍性能提升。
摘要:
生成式人工智能的快速增长及其在日常工作流程中的集成,显著增加了对大型语言模型(LLM)推理服务的需求。尽管专有模型仍然流行,但开源LLM的最新进展已使其成为强有力的竞争者。然而,部署这些模型常常受到GPU资源高成本和有限可用性的制约。为此,本文提出了一种用于LLM推理的去中心化离线服务系统的设计。利用空闲GPU资源,我们提出的系统DeServe以较低成本实现了对LLM的去中心化访问。DeServe专门解决了在高延迟网络环境中优化服务吞吐量的关键挑战。实验表明,在此类条件下,DeServe相较于现有服务系统基线实现了6.7倍至12.6倍的吞吐量提升。
英文摘要:
The rapid growth of generative AI and its integration into everyday workflows have significantly increased the demand for large language model (LLM) inference services. While proprietary models remain popular, recent advancements in open-source LLMs have positioned them as strong contenders. However, deploying these models is often constrained by the high costs and limited availability of GPU resources. In response, this paper presents the design of a decentralized offline serving system for LLM inference. Utilizing idle GPU resources, our proposed system, DeServe, decentralizes access to LLMs at a lower cost. DeServe specifically addresses key challenges in optimizing serving throughput in high-latency network environments. Experiments demonstrate that DeServe achieves a 6.7x-12.6x improvement in throughput over existing serving system baselines in such conditions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。