Floe：面向实时LLM-SLM推理的联邦专业化
Floe: Federated Specialization for Real-Time LLM-SLM Inference
AI总结：
Floe通过结合云LLM和边缘SLM实现低延迟隐私保护推理，提升实时环境下的个性化与性能
摘要：
在实时系统中部署大型语言模型（LLMs）仍然具有挑战性，因为它们有巨大的计算需求和隐私问题。我们提出了Floe，一种混合联邦学习框架，专为延迟敏感、资源受限的环境设计。Floe将基于云的黑盒LLM与轻量级小型语言模型（SLMs）结合在边缘设备上，以实现低延迟、隐私保护的推理。个人数据和微调仍保留在设备上，而云LLM提供通用知识而不暴露专有权重。一种异质性感知的LoRA适应策略使在不同硬件上高效部署边缘设备成为可能，而一个logit级别融合机制则实现了边缘和云模型之间的实时协调。广泛的实验表明，Floe增强了用户隐私和个性化。此外，它在实时约束下显著提高了边缘设备上的模型性能，并减少了推理延迟，优于基线方法。
英文摘要：
Deploying large language models (LLMs) in real-time systems remains challenging due to their substantial computational demands and privacy concerns. We propose Floe, a hybrid federated learning framework designed for latency-sensitive, resource-constrained environments. Floe combines a cloud-based black-box LLM with lightweight small language models (SLMs) on edge devices to enable low-latency, privacy-preserving inference. Personal data and fine-tuning remain on-device, while the cloud LLM contributes general knowledge without exposing proprietary weights. A heterogeneity-aware LoRA adaptation strategy enables efficient edge deployment across diverse hardware, and a logit-level fusion mechanism enables real-time coordination between edge and cloud models. Extensive experiments demonstrate that Floe enhances user privacy and personalization. Moreover, it significantly improves model performance and reduces inference latency on edge devices under real-time constraints compared with baseline approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
