arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.10367cs.CLcs.AI
收藏

用于增强边缘部署小模型的大小模型协作

Large-Small Model Collaboration for Enhancing Edge-Deployed Small Models

Zhejiang University(浙江大学);Tongyi Lab Alibaba Group(阿里巴巴集团通义实验室);Zhejiang Gongshang University(浙江工商大学);Alibaba Cloud(阿里云)

Peigen Liu, Yijiang Fan, Zixuan Xu, Yuren Mao, Longbin Lai, Ying Zhang

AI总结:

该研究提出自适应边缘-云框架G-Boost,通过动态选择推理或对数融合的协作方式,提升边缘部署小语言模型的任务性能,在两个数学推理数据集上优于多种基线方法。

摘要:

边缘设备托管着资源有限的特定领域小语言模型(SLM),而私有云则提供更大的大语言模型(LLM)。我们提出了G-Boost,这是一种自适应边缘-云框架,无需参数更新即可提升已部署SLM的任务性能。它将推理表述为树搜索,在每一步选择仅使用SLM推理,还是使用SLM-LLM对数融合——该方法在不暴露私有数据的情况下,将SLM适配版本的领域知识传递给云LLM。过程奖励模型引导蒙特卡洛树搜索,动态选择有益的协作步骤。边缘设备运行SLM和搜索控制器,云则托管LLM和奖励模型,仅交换当前上下文。在GSM8K和MATH-500数据集上,使用Qwen2.5和LLaMA2模型进行评估,结果显示G-Boost的性能优于单独的SLM、静态融合以及微调基线方法,相较于蒙特卡洛树搜索(MCTS)和代理调优(Proxy-Tuning),分别提升了多达8.6和10.7个百分点。这些结果证实,基于步骤级、奖励引导的动态协作可增强已部署边缘SLM的推理能力和领域利用效率。

英文摘要:

Edge devices host domain-specific small language models (SLMs) with limited resources, while private clouds offer larger LLMs. We propose G-Boost, an adaptive edge-cloud framework that improves a deployed SLM's task performance without parameter updates. It formulates reasoning as a tree search, choosing at each step between SLM-only inference and SLM-LLM logit fusion---which transfers domain knowledge from the SLM's adapted version to the cloud LLM without exposing private data. A process reward model guides Monte Carlo tree search to select beneficial collaboration steps dynamically. The edge runs the SLM and search controller; the cloud hosts the LLM and reward model, exchanging only current context. Evaluated on GSM8K and MATH-500 with Qwen2.5 and LLaMA2, G-Boost outperforms the SLM alone, static fusion, and fine-tuned baselines, gaining up to 8.6 and 10.7 percentage points over MCTS and Proxy-Tuning, respectively. Results confirm that step-level, reward-guided dynamic collaboration enhances reasoning and domain utilization for deployed edge SLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑