EACO-RAG:利用边缘辅助协同检索增强生成(RAG)与自适应知识更新实现分布式分层大语言模型部署
EACO-RAG: Towards Distributed Tiered LLM Deployment using Edge-Assisted and Collaborative RAG with Adaptive Knowledge Update
AI总结:
针对边缘环境部署小型语言模型(SLMs)的约束,提出EACO-RAG框架,通过边缘辅助协同检索增强生成(RAG)与自适应知识更新,匹配云基系统准确性且大幅降低成本,推动分布式分层大语言模型部署。
摘要:
大型语言模型(LLMs)在语言任务中展现出令人印象深刻的能力,但它们需要强大的计算能力且依赖静态知识。为克服这些限制,检索增强生成(RAG)无需大量微调即可将最新外部信息整合到LLMs中。同时,部署在边缘设备上的小型语言模型(SLMs)具备高效性和低延迟优势,但常难以处理复杂推理任务。遗憾的是,当前RAG方法主要基于集中式数据库,尚未针对边缘环境部署SLMs的独特约束进行适配。为填补这一空白,我们提出边缘辅助协同RAG(EACO-RAG),这是一种轻量级框架,利用分布式边缘节点实现自适应知识更新与检索。EACO-RAG还采用分层协同门控机制,动态选择本地、边缘辅助和云基策略,并基于安全在线贝叶斯优化设计了精心的算法以最大化潜在性能提升。实验结果表明,EACO-RAG匹配云基知识图谱RAG系统的准确性,同时在宽松延迟约束下总成本降低高达84.6%,在更严格延迟要求下降低65.3%。这项工作是我们实现分布式可扩展分层LLM部署的初步努力,EACO-RAG作为解锁混合边缘-云智能全部潜力的有前景的第一步。
英文摘要:
Large language models (LLMs) have demonstrated impressive capabilities in language tasks, but they require high computing power and rely on static knowledge. To overcome these limitations, Retrieval-Augmented Generation (RAG) incorporates up-to-date external information into LLMs without extensive fine-tuning. Meanwhile, small language models (SLMs) deployed on edge devices offer efficiency and low latency but often struggle with complex reasoning tasks. Unfortunately, current RAG approaches are predominantly based on centralized databases and have not been adapted to address the distinct constraints associated with deploying SLMs in edge environments. To bridge this gap, we propose Edge-Assisted and Collaborative RAG (EACO-RAG), a lightweight framework that leverages distributed edge nodes for adaptive knowledge updates and retrieval. EACO-RAG also employs a hierarchical collaborative gating mechanism to dynamically select among local, edge-assisted, and cloud-based strategies, with a carefully designed algorithm based on Safe Online Bayesian Optimization to maximize the potential performance enhancements. Experimental results demonstrate that EACO-RAG matches the accuracy of cloud-based knowledge graph RAG systems while reducing total costs by up to 84.6% under relaxed delay constraints and by 65.3% under stricter delay requirements. This work represents our initial effort toward achieving a distributed and scalable tiered LLM deployments, with EACO-RAG serving as a promising first step in unlocking the full potential of hybrid edge-cloud intelligence.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。