arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.00459cs.CL
收藏

NumeroLogic:用于增强 LLMs 数值推理的数字编码

NumeroLogic: Number Encoding for Enhanced LLMs' Numerical Reasoning

IBM Research(IBM研究院);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);MIT(麻省理工学院)

Eli Schwartz, Leshem Choshen, Joseph Shtok, Sivan Doveh, Leonid Karlinsky, Assaf Arbelle

AI总结:

针对语言模型因数字文本表示不直观导致算术能力受限的问题,提出 NumeroLogic 数字编码方法,通过在数字前添加位数前缀作为思维链来增强推理,有效提升了算术任务表现及 MMLU 基准的语言理解性能。

摘要:

语言模型在处理数值数据和执行算术运算方面存在困难。我们假设这种局限性部分可归因于不直观的文本数字表示方式。当因果语言模型读取或生成某个数字时,在处理完整个数字之前,它不知道该数字的位值(例如千位与百位)。为了解决这一问题,我们提出了一种对数字表示方式的简单调整,即在每个数字前包含其位数。例如,建议使用“{2:42}”作为新格式,而非“42”。我们将此方法称为 NumeroLogic,它通过作为思维链在数字生成方面提供了额外优势。通过要求模型首先考虑数字位数,它增强了生成实际数字前的推理过程。我们使用算术任务展示了 NumeroLogic 格式的有效性。我们进一步展示了 NumeroLogic 在通用自然语言建模中的适用性,提升了在 MMLU 基准上的语言理解性能。

英文摘要:

Language models struggle with handling numerical data and performing arithmetic operations. We hypothesize that this limitation can be partially attributed to non-intuitive textual numbers representation. When a digit is read or generated by a causal language model it does not know its place value (e.g. thousands vs. hundreds) until the entire number is processed. To address this issue, we propose a simple adjustment to how numbers are represented by including the count of digits before each number. For instance, instead of "42", we suggest using "{2:42}" as the new format. This approach, which we term NumeroLogic, offers an added advantage in number generation by serving as a Chain of Thought (CoT). By requiring the model to consider the number of digits first, it enhances the reasoning process before generating the actual number. We use arithmetic tasks to demonstrate the effectiveness of the NumeroLogic formatting. We further demonstrate NumeroLogic applicability to general natural language modeling, improving language understanding performance in the MMLU benchmark.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑