探究多语言数字谜题中语言与数学推理在语言模型中的交互作用
Investigating the interaction of linguistic and mathematical reasoning in language models using multilingual number puzzles
AI总结：
本研究通过多语言数字谜题实验，发现LLMs难以解决跨语言数字推理问题，除非运算符号显式标记，且缺乏人类对数字隐含组合结构的推断能力，这对当前推理模型仍是挑战。
摘要：
在不同语言中，数字系统在构造和组合数字的方式上差异很大。虽然人类能够持续学会应对这种多样性，但大型语言模型（LLMs）在处理涉及跨语言数字系统的语言-数学谜题时却存在困难，而人类能够学会成功解决这些谜题。我们通过一系列实验来探究这一任务对LLMs而言为何困难，这些实验将语言中数字的语言学方面和数学方面分离开来。我们的实验证实，除非问题中的数学运算使用已知符号（如“+”、“×”等，例如“twenty + three”）明确标记，否则模型无法一致地解决此类问题。在进一步的消融研究中，我们探究了数字构造和组合的各个参数如何影响性能。虽然人类利用其对数字的语言学理解来推断数字隐含的组合结构，但LLMs似乎缺乏这种隐含数字结构的概念。我们得出结论，从人类规模数据中的隐含模式灵活推断组合规则的能力，对于当前的推理模型而言仍是一个开放的挑战。
英文摘要：
Across languages, numeral systems vary widely in how they construct and combine numbers. While humans consistently learn to navigate this diversity, large language models (LLMs) struggle with linguistic-mathematical puzzles involving cross-linguistic numeral systems, which humans can learn to solve successfully. We investigate why this task is difficult for LLMs through a series of experiments that untangle the linguistic and mathematical aspects of numbers in language. Our experiments establish that models cannot consistently solve such problems unless the mathematical operations in the problems are explicitly marked using known symbols ($+$, $\times$, etc., as in "twenty + three"). In further ablation studies, we probe how individual parameters of numeral construction and combination affect performance. While humans use their linguistic understanding of numbers to make inferences about the implicit compositional structure of numerals, LLMs seem to lack this notion of implicit numeral structure. We conclude that the ability to flexibly infer compositional rules from implicit patterns in human-scale data remains an open challenge for current reasoning models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
