推测，再协作：解码阶段融合大语言模型的知识
Speculate, then Collaborate: Fusing Knowledge of Language Models during Decoding
AI总结：
针对大语言模型跨领域性能受限的问题，提出无需额外训练的协作推测解码算法CoSD，通过草稿模型+决策规则调用辅助模型融合知识，提升准确率与推理效率，可跨域迁移且可解释性强。
摘要：
大语言模型（LLM）通常在特定领域表现优异，但受训练数据限制，在其他领域存在不足。因此，让LLM通过整合互补知识协作解决问题，有望提升其跨领域性能。为实现这一潜力，我们提出了一种新型协作推测解码（Collaborative Speculative Decoding, CoSD）算法，可在测试阶段高效实现LLM知识融合，无需额外模型训练。CoSD使用草稿模型生成初始序列，通过易学习的规则或决策树判断何时调用辅助模型优化草稿。CoSD不仅强化了知识融合，还提升了推理效率，可跨领域、跨模型迁移，且具备更强的可解释性。实验结果表明，与现有方法相比，CoSD在各基准测试上的准确率最高提升10%，为基于LLM的应用提供了可扩展的有效解决方案。
英文摘要：
Large Language Models (LLMs) often excel in specific domains but fall short in others due to the limitations of their training. Thus, enabling LLMs to solve problems collaboratively by integrating their complementary knowledge promises to improve their performance across domains. To realize this potential, we introduce a novel Collaborative Speculative Decoding (CoSD) algorithm that enables efficient LLM knowledge fusion at test time without requiring additional model training. CoSD employs a draft model to generate initial sequences and an easy-to-learn rule or decision tree to decide when to invoke an assistant model to improve these drafts. CoSD not only enhances knowledge fusion but also improves inference efficiency, is transferable across domains and models, and offers greater explainability. Experimental results demonstrate that CoSD improves accuracy by up to 10\% across benchmarks compared to existing methods, providing a scalable and effective solution for LLM-based applications
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
