arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.18547cs.CLcs.AIcs.LG
收藏

Token-Budget-Aware LLM推理

Token-Budget-Aware LLM Reasoning

Nanjing University(南京大学);Rutgers University(罗格斯大学);UMass Amherst(马萨诸塞大学阿姆赫斯特分校)

Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, Zhenyu Chen

AI总结:

针对LLM思维链推理中token开销过大的问题,提出token-budget-aware框架,根据问题复杂度动态调整推理token数量,在轻微性能下降下有效降低成本并平衡效率与准确性。

摘要:

推理对于大型语言模型(LLMs)在广泛任务中表现出色至关重要。虽然像思维链(CoT)推理这样的方法通过将问题分解为中间步骤来提升LLM性能,但它们在token使用上也会产生显著开销,从而导致成本增加。我们发现当前LLMs的推理过程不必要地冗长,通过在提示中包含合理的token预算可以对其进行压缩,但token预算的选择在实际压缩效果中起着至关重要的作用。随后,我们提出了一种token-budget-aware LLM推理框架,该框架根据每个问题的推理复杂度动态调整推理token的数量。实验表明,我们的方法在CoT推理中有效降低了token成本,且仅有轻微的性能下降,为平衡LLM推理的效率与准确性提供了实用解决方案。代码:https://github.com/GeniusHTX/TALE

英文摘要:

Reasoning is critical for large language models (LLMs) to excel in a wide range of tasks. While methods like Chain-of-Thought (CoT) reasoning and enhance LLM performance by decomposing problems into intermediate steps, they also incur significant overhead in token usage, leading to increased costs. We find that the reasoning process of current LLMs is unnecessarily lengthy and it can be compressed by including a reasonable token budget in the prompt, but the choice of token budget plays a crucial role in the actual compression effectiveness. We then propose a token-budget-aware LLM reasoning framework that dynamically adjusts the number of reasoning tokens based on the reasoning complexity of each problem. Experiments show that our method effectively reduces token costs in CoT reasoning with only a slight performance reduction, offering a practical solution to balance efficiency and accuracy in LLM reasoning. Code: https://github.com/GeniusHTX/TALE

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑