无需训练的路由：通过可靠性门控实现可控比例的大语言模型卸载
Routing Without Training: Controllable-Ratio LLM Offloading via Reliability Gating
- Purdue University（普渡大学）
- University of Exeter（埃克塞特大学）
- Army Research Laboratory（陆军研究实验室）
- Princeton University（普林斯顿大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
研究在资源受限下本地-云端协作部署大语言模型的问题，提出无需训练的CARGO路由框架，通过提示多样化采样等方法估计一致性并控制不确定性，在多任务和模型上表现出色，证明可从本地模型内在行为实现有效协作。
AI中文摘要：
本地-云端协作是在资源受限情况下部署大语言模型的实用方法，但现有方法常依赖训练好的路由器或协作感知微调，将路由行为与特定运行模式绑定。本文表明此类训练可能不必要，本地模型自身在采样响应间的推理时间一致性已为决定何时信任本地执行、何时卸载到更强的云端模型提供了有力信号。我们提出CARGO，一个无需训练的路由框架，通过提示多样化采样估计这种一致性，应用贝叶斯早期停止进行高效样本不确定性控制，并通过轻量级部署时校准支持任意目标协作比例。在各种推理和问答任务、多个本地大语言模型家族和规模以及预训练和微调的本地模型上，CARGO始终优于其他无需训练的基线，在多种设置下超越有监督学习的路由器。这些结果表明，有效且适应性强的本地-云端协作可直接从本地模型的内在响应行为中产生，无需额外训练的路由器。
英文摘要：
Local-cloud collaboration is a practical way to deploy large language models under resource constraints, but existing methods often rely on trained routers or collaboration-aware finetuning that tie routing behavior to a particular operating regime. In this work, we show that such training may be unnecessary: the local model's own inference-time agreement across sampled responses already provides a strong signal for deciding when to trust local execution and when to offload to a stronger cloud model. We propose CARGO, a training-free routing framework that estimates this agreement through prompt-varied sampling, applies Bayesian early stopping for sample-efficient uncertainty control, and supports arbitrary target collaboration ratios through lightweight deployment-time calibration. Across diverse reasoning and question-answering tasks, multiple local LLM families and scales, and both pretrained and finetuned local models, CARGO consistently outperforms other training-free baselines and in several settings surpasses supervised learned routers. These results suggest that effective and adaptable local-cloud collaboration can emerge directly from the local model's intrinsic response behavior, without requiring an additional trained router.
