发表机构
KAIST; Seoul National University; Korea University（韩国科学技术院; 首尔大学; 高丽大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出自适应数值注入框架ANI，通过上下文感知门控选择性注入数值嵌入，结合语义与算术优势，在MATH基准上提升9.5分，保持通用语言性能。
AI 中文摘要
大型语言模型（LLMs）的精确数值推理对于扩展其在复杂现实世界任务中的适用性至关重要。然而，基于文本的分词常常将数字碎片化，显著阻碍了精确的算术推理。与此同时，数值嵌入尽管具有算术精度，却依赖于与上下文无关的替换，忽视了数字作为标识符的语义作用。为了结合两者的互补优势，我们提出了 ANI（自适应数值注入），一种混合框架，根据语义上下文管理数值特征的选择性注入。通过采用上下文感知的门控机制，我们将数值嵌入（特别是 FoNE）选择性地注入到潜在空间中，明确保留名义标识符，同时增强定量操作数。通过对各种 LLMs 的广泛评估，我们证明 ANI 在 MATH 基准上相对于官方参考模型提升了 9.5 分，同时在通用语言基准上保持了稳健的性能。
英文摘要
Precise numerical reasoning with Large Language Models (LLMs) is essential for expanding their applicability to complex real-world tasks. However, text-based tokenization often fragments numbers, significantly hindering precise arithmetic reasoning. Meanwhile, numerical embeddings, despite arithmetic precision, rely on context-agnostic substitution that disregards the semantic role of numbers as identifiers. To combine the complementary strengths, we propose \textbf{ANI (Adaptive Numerical Injection)}, a hybrid framework that governs the selective injection of numerical features based on the semantic context. By employing a context-aware gating mechanism, we selectively inject numerical embeddings (specifically FoNE) into the latent space, explicitly preserving nominal identifiers while enhancing quantitative operands. Through extensive evaluations across various LLMs, we demonstrate that ANI enhances MATH performance by 9.5 points over the official reference model, while maintaining robust performance on general linguistic benchmarks.
CommentsAccepted to EMNLP 2026. 16 pages, 7 figures. Code available at https://github.com/Jinsung-Jeon/ANI_EMNLP
