OccamLLM:单步快速且精确的语言模型算术运算
OccamLLM: Fast and Exact Language Model Arithmetic in a Single Step
AI总结:
本文提出OccamLLM框架,利用LLM隐藏状态控制OccamNet符号架构,在单个自回归步骤内实现精确算术,并在算术运算和数学基准上超越GPT 4o。
摘要:
尽管在文本生成和推理方面取得了显著进展,大型语言模型(LLMs)在准确执行复杂算术运算方面仍面临挑战。语言模型系统通常让LLMs生成用于算术运算的代码,以实现准确计算。然而,这种方法会牺牲速度和安全性,并且微调存在使语言模型丧失先前能力的风险。我们提出一个框架,能够在单个自回归步骤中实现精确算术,从而提供更快、更安全且更具可解释性的具备算术能力的LLM系统。我们利用LLM的隐藏状态来控制执行算术的符号架构。我们使用Llama 3并以OccamNet作为符号模型(OccamLlama)的实现,在单一算术运算($+,-,\times,÷,\sin{},\cos{},\log{},\exp{},\sqrt{}$)上达到100\\%的准确率,优于带和不带代码解释器的GPT 4o。此外,在一系列数学问题求解基准上,OccamLlama平均表现优于带和不带代码解释器的GPT 4o,表明OccamLLMs在算术任务中表现出色,甚至超越规模大得多的模型。我们将很快公开代码。
英文摘要:
Despite significant advancements in text generation and reasoning, Large Language Models (LLMs) still face challenges in accurately performing complex arithmetic operations. Language model systems often enable LLMs to generate code for arithmetic operations to achieve accurate calculations. However, this approach compromises speed and security, and fine-tuning risks the language model losing prior capabilities. We propose a framework that enables exact arithmetic in a single autoregressive step, providing faster, more secure, and more interpretable LLM systems with arithmetic capabilities. We use the hidden states of a LLM to control a symbolic architecture that performs arithmetic. Our implementation using Llama 3 with OccamNet as a symbolic model (OccamLlama) achieves 100\% accuracy on single arithmetic operations ($+,-,\times,÷,\sin{},\cos{},\log{},\exp{},\sqrt{}$), outperforming GPT 4o with and without a code interpreter. Furthermore, OccamLlama outperforms GPT 4o with and without a code interpreter on average across a range of mathematical problem solving benchmarks, demonstrating that OccamLLMs can excel in arithmetic tasks, even surpassing much larger models. We will make our code public shortly.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。