arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.16219cs.CL
收藏

SpecSteer:协同局部上下文与全局推理以实现高效个性化生成

SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation

University of Science and Technology of China(中国科学技术大学);Huawei Technologies Co., Ltd.(华为技术有限公司)

Hang Lv, Sheng Liang, Hao Wang, Yongyue Zhang, Hongchao Gu, Wei Guo, Defu Lian, Yong Liu, Enhong Chen

AI总结:

本文提出SpecSteer框架,通过结合本地上下文与云端推理,解决个性化生成中的隐私与推理能力矛盾,实现高效生成并提升性能。

摘要:

实现个性化智能面临核心矛盾:将用户历史发送至集中式大语言模型会引发隐私问题,而本地小型语言模型缺乏生成高质量内容所需的推理能力。我们的初步研究显示,纯粹的局部增强不足以可靠弥合这一差距。因此,我们提出SpecSteer,一种非对称协作推理框架,协同私有本地上下文与大规模云端推理。SpecSteer将协作视为贝叶斯知识融合,并重新利用推测解码作为分布式对齐协议,产生一个Draft--Verify--Recover流程:本地模型起草个性化序列;云端通过基于比率的机制验证,将推理验证与私有上下文解耦,过滤逻辑错误而不访问原始用户上下文;在被拒绝时,一个引导恢复在纠正过程中注入本地意图。实验表明,SpecSteer成功缩小了推理差距,实现了优越的个性化生成性能,同时相比标准基线提升了2.36倍的速度。

英文摘要:

Realizing personalized intelligence faces a core dilemma: sending user history to centralized large language models raises privacy concerns, while on-device small language models lack the reasoning capacity required for high-quality generation. Our pilot study shows that purely local enhancements remain insufficient to reliably bridge this gap. We therefore propose SpecSteer, an asymmetric collaborative inference framework that synergizes private on-device context with cloud-scale reasoning. SpecSteer casts collaboration as Bayesian knowledge fusion and repurposes speculative decoding as a distributed alignment protocol, yielding a Draft--Verify--Recover pipeline: the on-device model drafts personalized sequences; the cloud validates via a ratio-based mechanism that decouples reasoning verification from private context, filtering logical flaws without accessing raw user context; upon rejection, a steering recovery injects local intent during correction. Experiments demonstrate that SpecSteer successfully closes the reasoning gap and achieves superior personalized generation performance, while delivering a 2.36x speedup over standard baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑