AsymSpec：非对称网络下的高效云边投机解码
AsymSpec: Efficient Cloud-Edge Speculative Decoding over Asymmetric Networks
浏览论文内容
中文总结 AI 辅助
AsymSpec针对云边投机解码的通信与资源浪费问题，提出非对称验证协议与确认前缀管道，在多场景下将输出令牌吞吐量提升至基线的2.82-28.03倍。
中文摘要 AI 辅助
云边投机解码将轻量级草稿模型部署在边缘网关，更高质量的目标模型部署在云端，但会在每个投机块中插入通信。在上行链路受限的情况下，候选消息可能会在验证器空闲时排队；停-等调度会导致边缘计算利用率不足，而乐观的同请求超前执行则会在拒绝或意外奖励令牌使依赖草稿失效时浪费工作。我们提出AsymSpec，通过两种对应机制解决上行链路门控验证和依赖工作失效的问题：其非对称验证协议将公共路径接受上传内容压缩，并将更丰富的仅拒绝校正信息移至下行链路；残差分布的总变差（TV）证书确定小的目标Top-K响应是否足够，若不足，协议会通过基于提议的精确恢复逐步升级，再回退至完整分布。其确认前缀管道仅向边缘调度器暴露独立有效请求，并让云端对到达的块重新批处理，在有其他确认前缀请求就绪时隐藏验证等待，无需使用同请求超前执行。在三组草稿-目标对、两个工作负载和三个非对称网络配置下，端到端评估显示，AsymSpec的输出令牌吞吐量是最强基线的2.82至28.03倍。
英文摘要
Cloud-edge speculative decoding places a lightweight draft model at an edge gateway and a higher-quality target model in the cloud, but inserts communication into every speculative block. Under a constrained uplink, candidate messages may queue while the verifier is idle. Stop-and-wait scheduling leaves edge compute underutilized; optimistic same-request runahead can waste work when a rejection or an unexpected bonus token invalidates dependent drafts. We present AsymSpec, which addresses uplink-gated verification and invalid dependent work with two corresponding mechanisms. Its asymmetric verification protocol keeps the common-path acceptance upload compact and moves richer, rejection-only correction information to the downlink. A total-variation (TV) certificate for the residual distribution determines whether a small target top-K response suffices; if not, the protocol progressively escalates through proposal-based exact recovery before falling back to the full distribution. Its confirmed-prefix pipeline exposes only independent, valid requests to the edge scheduler and lets the cloud re-batch arrived blocks, hiding verification waits when another confirmed-prefix request is ready without using same-request runahead. Across three draft-target pairs, two workloads, and three asymmetric network profiles, our end-to-end evaluation shows that AsymSpec delivers 2.82-28.03$\times$ the output-token throughput of the strongest baseline.
