通过隐藏思维链解码加速并提升大语言模型推理能力
Expediting and Elevating Large Language Model Reasoning via Hidden Chain-of-Thought Decoding
AI总结：
针对大语言模型思维链提示推理延迟高的问题，提出基于语义对齐的隐藏思维链解码方法，通过两阶段训练压缩思维链表示，在三类任务上实现至少1.5倍解码加速且性能相当或更优。
摘要：
大语言模型（LLM）通过思维链（CoT）提示，在需要推理和多步问题解决的任务中展现出了卓越的能力。然而，生成完整的CoT过程会导致输出序列显著变长，进而增加推理阶段的计算成本与延迟。为应对这一挑战，我们提出了一种通过语义对齐压缩CoT过程的新方法，既能实现更高效的解码，又能保留CoT推理的优势。\n我们的方法引入了一个辅助CoT模型，该模型学习生成完整思维过程并将其压缩为与原始CoT输出语义对齐的紧凑特殊令牌表示。随后，这一压缩表示会被整合到隐藏思维链（HCoT）模型的输入中。训练过程分为两个阶段：首先，对CoT模型进行优化，使其通过对比损失生成与真实CoT输出对齐的压缩令牌表示。随后，在冻结CoT模型参数的情况下，对HCoT模型进行微调，使其基于前缀指令和来自CoT模型的压缩CoT表示生成准确的后续预测。\n我们在数学推理、智能体调用、问答这三个具有挑战性的领域开展了大量实验，结果表明，与完整CoT基线相比，我们的语义压缩方法取得了相当甚至更优的性能，同时解码速度至少提升了1.5倍。此外，引入对比学习目标进一步提升了压缩表示的质量，带来了更好的CoT提示效果与更高的任务准确率。我们的工作为在各类应用中更高效地利用LLM的多步推理能力铺平了道路。
英文摘要：
Large language models (LLMs) have demonstrated remarkable capabilities in tasks requiring reasoning and multi-step problem-solving through the use of chain-of-thought (CoT) prompting. However, generating the full CoT process results in significantly longer output sequences, leading to increased computational costs and latency during inference. To address this challenge, we propose a novel approach to compress the CoT process through semantic alignment, enabling more efficient decoding while preserving the benefits of CoT reasoning. Our method introduces an auxiliary CoT model that learns to generate and compress the full thought process into a compact special token representation semantically aligned with the original CoT output. This compressed representation is then integrated into the input of the Hidden Chain-of-Thought (HCoT) model. The training process follows a two-stage procedure: First, the CoT model is optimized to generate the compressed token representations aligned with the ground-truth CoT outputs using a contrastive loss. Subsequently, with the CoT model parameters frozen, the HCoT model is fine-tuned to generate accurate subsequent predictions conditioned on the prefix instruction and the compressed CoT representations from the CoT model. Extensive experiments across three challenging domains - mathematical reasoning, agent invocation, and question answering - demonstrate that our semantic compression approach achieves competitive or improved performance compared to the full CoT baseline, while providing significant speedups of at least 1.5x in decoding time. Moreover, incorporating contrastive learning objectives further enhances the quality of the compressed representations, leading to better CoT prompting and improved task accuracy. Our work paves the way for more efficient exploitation of multi-step reasoning capabilities in LLMs across a wide range of applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
