Plato：面向大语言模型推理的高效解码规划方法
Plato: Plan to Efficiently Decode for Large Language Model Inference
AI总结：
针对大语言模型推理效率低且现有并行解码方法损害答案质量的问题，提出Plato方法，通过构建子问题依赖图实现语义感知并行解码，结合流水线设计与缓存优化，在提升吞吐量68%的同时，相比SoT实现90%的质量净胜率。
摘要：
大语言模型（LLMs）在自然语言任务中取得了显著成功，但其推理过程会产生大量计算和内存开销。为提升效率，Skeleton-of-Thought（SoT）等并行解码方法将提示分解为子问题以进行并发处理。然而，这些方法将语义关联的子问题视为独立问题，严重降低了答案质量。我们提出Plato，一种协同设计算法与系统的语义感知并行解码新方法。Plato利用LLMs基于逻辑和因果关系将子问题组织成依赖图，在保留答案连贯性和质量的同时，实现非依赖节点的并发解码。为进一步提升效率，Plato将规划阶段与节点解码阶段进行流水线处理，实现全局上下文缓存，并精心构造节点推理提示以最大化键值缓存复用并最小化开销。我们的评估显示，Plato相比自回归解码将吞吐量提升68%，同时在答案质量上实现40%的净胜率；与SoT相比，Plato的质量净胜率高达90%。消融研究表明，我们的流水线设计将加速比提升29%，而键值缓存复用优化将开销降低75%。
英文摘要：
Large language models (LLMs) have achieved remarkable success in natural language tasks, but their inference incurs substantial computational and memory overhead. To improve efficiency, parallel decoding methods like Skeleton-of-Thought (SoT) decompose prompts into sub-problems for concurrent processing. However, these methods significantly compromise answer quality by treating semantically linked sub-problems as independent. We propose Plato, a novel approach that co-designs algorithms and systems for semantic-aware parallel decoding. Plato leverages LLMs to organize sub-problems into a dependency graph based on logical and causal relationships, enabling concurrent decoding of non-dependent nodes while preserving answer coherence and quality. To further enhance efficiency, Plato pipelines planning and node decoding stages, implements a global context cache, and carefully structures node inference prompts to maximize key-value cache reuse and minimize overhead. Our evaluations show that Plato improves throughput by 68% over autoregressive decoding while achieving a 40% net win rate in answer quality. Compared to SoT, Plato demonstrates a remarkable 90% quality net-win rate. Ablation studies reveal that our pipeline design improves speedup by 29%, while our KV cache reuse optimization reduces overhead by 75%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
