arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.19981cs.CL
收藏

前瞻局限性:为什么多操作数加法对大型语言模型而言是难题

The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs

German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Centre for European Research in Trusted AI (CERTAIN)(欧洲可信人工智能研究中心);Saarland University(萨尔大学)

Tanja Baeumel, Josef van Genabith, Simon Ostermann

AI总结:

本研究揭示大型语言模型因依赖单数字前瞻启发式而在多操作数加法中失败,通过探测实验和逐位评估证实其固有限制,阻碍了复杂数值推理的泛化。

摘要:

自回归大型语言模型(LLMs)在各种任务上表现出色,但在简单算术上却力不从心,例如两个或更多操作数的加法。我们证明,这一困难源于LLMs使用了一种简单的单数字前瞻启发式方法,该方法在双操作数加法中效果尚可(但并非完美),但在多操作数情况下则失效,因为此时进位逻辑更为复杂。我们的探测实验和逐位准确率评估显示,LLMs恰恰在单数字前瞻不足以应对级联进位的地方失败。我们分析了分词策略对算术性能的影响,并表明所有被调查的模型,无论分词方式如何,都因其对单数字前瞻启发式的依赖而在多操作数加法上存在固有限制。我们的发现揭示了阻碍LLMs泛化到更复杂数值推理的根本局限。

英文摘要:

Autoregressive large language models (LLMs) exhibit impressive performance across various tasks but struggle with simple arithmetic, such as addition of two or more operands. We show that this struggle arises from LLMs' use of a simple one-digit lookahead heuristic, which works fairly well (but not perfect) for two-operand addition but fails in multi-operand cases, where the carry-over logic is more complex. Our probing experiments and digit-wise accuracy evaluation show that LLMs fail precisely where a one-digit lookahead is insufficient to account for cascading carries. We analyze the impact of tokenization strategies on arithmetic performance and show that all investigated models, regardless of tokenization, are inherently limited in the addition of multiple operands due to their reliance on a one-digit lookahead heuristic. Our findings reveal fundamental limitations that prevent LLMs from generalizing to more complex numerical reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑