BanditSpec: Adaptive Speculative Decoding via Bandit Algorithms
BanditSpec: 通过多臂老虎机算法实现自适应推测解码
机构 * National University of Singapore(国立新加坡大学) ; Sea AI Lab(Sea人工智能实验室) ; Singapore Management University(新加坡管理学院) ; Yale University(耶鲁大学)
AI总结 BanditSpec通过多臂老虎机算法实现自适应推测解码,优化超参数选择以提升LLM推理效率。
Comments 35 pages, 4 figures, accepted to ICML, typos and affiliations are corrected