arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.28772cs.DC
收藏

联邦推断用于异构大语言模型通信与协作

Federated Inference for Heterogeneous LLM Communication and Collaboration

Singapore University of Technology and Design(新加坡科技设计大学);ZJU-UIUC Institute, Zhejiang University(浙江大学中法联合学院)

Zihan Chen, Zeshen Li, Howard H. Yang, Tony Q. S. Quek, Jihong Park

AI总结:

本文提出FedRefine框架,解决多LLM协作中的性能、隐私和异构性问题,通过隐私保护的KV缓存通信提升设备端推断能力。

摘要:

鉴于设备端大语言模型(LLM)性能和效率有限,多个LLM之间的协作可带来性能提升,数据、token和模型权重可在LLM间共享。此过程受任务导向的QoS需求、隐私要求和系统异构性所限制。为充分挖掘设备端推断能力,本文提出一种新型联邦推断框架FedRefine,该框架为异构LLM在隐私保护下协同执行推断提供新范式。提供了一些数值结果以突出FedRefine的优越性,并强调了未来研究的几个有趣话题。通过探索LLM原生通信,旨在为该广泛领域提供新的范式。

英文摘要:

Given the limited performance and efficiency of on-device Large Language Models (LLMs), the collaborations between multiple LLMs enable desirable performance enhancements, in which data, tokens, and model weights could be shared across LLMs. This process is constrained by task-oriented QoS demands, privacy requirements, and inherent system heterogeneity. In view of the above challenge and to fully exploit the on-device inference capabilities, we present a novel federated inference framework in this position paper, termed federated refinement \texttt{FedRefine}. This framework presents a new paradigm for heterogeneous LLMs collaboratively performing inference with communicating KV caches in a privacy-preserving manner. Some numerical results are provided to highlight the superiority of \texttt{FedRefine}. Several interesting topics are also highlighted for future research. By exploring the LLM-native communications, we wish to provide a new paradigm for this broad area.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑