arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2504.20101cs.DCcs.AI
收藏

PlanetServe: 一种去中心化、可扩展且隐私保护的叠加层,用于民主化大语言模型服务

PlanetServe: A Decentralized, Scalable, and Privacy-Preserving Overlay for Democratizing Large Language Model Serving

University of California, Santa Cruz(加州大学圣克鲁兹分校);University of Nevada, Reno(内华达大学拉森)

Fei Fang, Yifan Hua, Shengze Wang, Ruilin Zhou, Yi Liu, Chen Qian, Xiaoxue Zhang

AI总结:

PlanetServe通过去中心化叠加层提升大语言模型服务的可扩展性和隐私保护,实现服务效率与安全性的平衡。

摘要:

尽管在开源和低成本的大语言模型(LLMs)研究和开发方面取得了显著进展,但服务的可扩展性仍然是一个关键挑战,特别是对于小型组织和个人希望部署和测试其LLM创新而言。受点对点网络利用去中心化叠加节点来提高吞吐量和可用性的启发,我们提出了GenTorrent,一种利用去中心化贡献者计算资源的LLM服务叠加层。我们识别出四个关键研究问题,这些问题是启用此类去中心化基础设施固有的:1)叠加网络组织;2)LLM通信隐私;3)叠加转发以提高资源效率;4)服务质量的验证。本工作首次系统研究了这些基本问题在去中心化LLM服务中的情况。原型在一组去中心化节点上实现的评估结果表明,GenTorrent相比无叠加转发的基线设计实现了超过50%的延迟降低。此外,安全特性对服务延迟和吞吐量引入了最小的开销。我们认为这项工作开创了民主化和扩展未来AI服务能力的新方向。

英文摘要:

While significant progress has been made in research and development on open-source and cost-efficient large-language models (LLMs), serving scalability remains a critical challenge, particularly for small organizations and individuals seeking to deploy and test their LLM innovations. Inspired by peer-to-peer networks that leverage decentralized overlay nodes to increase throughput and availability, we propose GenTorrent, an LLM serving overlay that harnesses computing resources from decentralized contributors. We identify four key research problems inherent to enabling such a decentralized infrastructure: 1) overlay network organization; 2) LLM communication privacy; 3) overlay forwarding for resource efficiency; and 4) verification of serving quality. This work presents the first systematic study of these fundamental problems in the context of decentralized LLM serving. Evaluation results from a prototype implemented on a set of decentralized nodes demonstrate that GenTorrent achieves a latency reduction of over 50% compared to the baseline design without overlay forwarding. Furthermore, the security features introduce minimal overhead to serving latency and throughput. We believe this work pioneers a new direction for democratizing and scaling future AI serving capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑