Skeleton-of-Thought：提示大语言模型实现高效并行生成
Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation
AI总结：
针对LLM顺序解码导致的高延迟，本文提出Skeleton-of-Thought，先生成答案骨架再并行补全各要点，在12个LLM上实现加速并可能改善答案质量。
摘要：
本工作旨在降低大语言模型（LLM）的端到端生成延迟。高生成延迟的主要原因之一是几乎所有最先进的LLM都采用顺序解码方法。在本工作中，受人类思考与写作过程的启发，我们提出Skeleton-of-Thought（SoT），该方法首先引导LLM生成答案的骨架，随后通过并行API调用或批量解码，并行补全每个骨架要点的内容。SoT不仅在12个LLM上带来了可观的加速，还可能在若干问题类别上提升答案质量。SoT是面向推理效率的以数据为中心优化的初步尝试，并展示了通过用语言显式规划答案结构来引出高质量答案的潜力。
英文摘要：
This work aims at decreasing the end-to-end generation latency of large language models (LLMs). One of the major causes of the high generation latency is the sequential decoding approach adopted by almost all state-of-the-art LLMs. In this work, motivated by the thinking and writing process of humans, we propose Skeleton-of-Thought (SoT), which first guides LLMs to generate the skeleton of the answer, and then conducts parallel API calls or batched decoding to complete the contents of each skeleton point in parallel. Not only does SoT provide considerable speed-ups across 12 LLMs, but it can also potentially improve the answer quality on several question categories. SoT is an initial attempt at data-centric optimization for inference efficiency, and showcases the potential of eliciting high-quality answers by explicitly planning the answer structure in language.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
