arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.07896cs.AI
收藏

执行算术:将大型语言模型微调为图灵机

Executing Arithmetic: Fine-Tuning Large Language Models as Turing Machines

National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)

Junyu Lai, Jiahe Xu, Yao Yang, Yunpeng Huang, Chun Cao, Jingwei Xu

AI总结:

针对大型语言模型在算术任务中依赖记忆而非计算逻辑的问题,提出可组合算术执行框架(CAEF),通过模拟图灵机让模型逐步执行计算,在LLaMA 3.1-8B上实现近100%准确率并支持百位操作数计算。

摘要:

大型语言模型(LLMs)在广泛的自然语言处理和推理任务中展现了卓越的能力。然而,它们在基础算术领域的表现仍不令人满意。在处理算术任务时,LLMs常常记忆特定示例,而非学习底层的计算逻辑,这限制了它们泛化到新问题的能力。在本文中,我们提出了一种可组合算术执行框架(CAEF),该框架使LLMs能够通过模拟图灵机来学习执行逐步计算,从而真正理解计算逻辑。此外,该框架具有高度可扩展性,允许组合已学习的算子,以显著降低学习复杂算子的难度。在我们的评估中,CAEF在LLaMA 3.1-8B模型上的七种常见数学运算中实现了接近100%的准确率,有效支持涉及多达100位操作数的计算,而在某些设置中GPT-4o在此水平上明显表现不足。

英文摘要:

Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of natural language processing and reasoning tasks. However, their performance in the foundational domain of arithmetic remains unsatisfactory. When dealing with arithmetic tasks, LLMs often memorize specific examples rather than learning the underlying computational logic, limiting their ability to generalize to new problems. In this paper, we propose a Composable Arithmetic Execution Framework (CAEF) that enables LLMs to learn to execute step-by-step computations by emulating Turing Machines, thereby gaining a genuine understanding of computational logic. Moreover, the proposed framework is highly scalable, allowing composing learned operators to significantly reduce the difficulty of learning complex operators. In our evaluation, CAEF achieves nearly 100% accuracy across seven common mathematical operations on the LLaMA 3.1-8B model, effectively supporting computations involving operands with up to 100 digits, a level where GPT-4o falls short noticeably in some settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑