arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.01281cs.CLcs.AI
收藏

用于LLM高效推理的带有前缀增强的路径一致性

Path-Consistency with Prefix Enhancement for Efficient Inference in LLMs

Shanghai Jiao Tong University(上海交通大学);Microsoft Corporation(微软公司)

Jiace Zhu, Yuanzhe Huang, Yingtao Shen, Jie Zhao, An Zou

AI总结:

本文提出path-consistency方法,利用早期答案置信度选择最优前缀指导后续分支生成,减少自一致性的采样冗余,在保持任务准确性的同时将推理延迟提升达40.5%。

摘要:

为了增强大型语言模型(LLMs)的推理能力,自一致性已成为一种流行方法,它将多次采样与多数投票相结合。然而,由于需要大量采样,当前方法计算成本高且耗时。为了解决这个问题,本文引入了path-consistency,它利用先前生成答案的置信度来识别最有希望的前缀,并指导后续分支的生成。通过基于该前缀动态指导后续分支的生成,path-consistency减轻了自一致性中随机或较少有用采样带来的错误和冗余。这种方法减少了随机采样产生的错误和冗余,通过最小化token消耗显著加速了推理。我们广泛的实证结果表明,path-consistency将推理延迟提高了多达40.5%,同时在包括数学推理、常识推理和符号推理在内的各种任务中保持了任务准确性。

英文摘要:

To enhance the reasoning capabilities of large language models (LLMs), self-consistency has become a popular approach, combining multiple samplings with majority voting. However, current methods are computationally expensive and time-consuming due to the need for numerous samplings. To address this, this paper introduces path-consistency, which leverages the confidence of earlier-generated answers to identify the most promising prefix and guide the generation of subsequent branches. By dynamically guiding the generation of subsequent branches based on this prefix, path-consistency mitigates both the errors and redundancies from random or less useful sampling in self-consistency. This approach reduces errors and redundancies from random sampling, significantly accelerating inference by minimizing token consumption. Our extensive empirical results demonstrate that path-consistency improves inference latency by up to 40.5\%, while maintaining task accuracy across various tasks, including mathematical reasoning, commonsense reasoning, and symbolic reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑