能力更强，忠实度更低：大语言模型中数学（不）可解性检测的多语言分析
More Capable, Less Faithful: A Multilingual Analysis of Mathematical (Un)Solvability Detection in LLMs
浏览论文内容
中文总结 AI 辅助
该研究构建了法语、希腊语版ReliableMath多语言基准，训练多语言探针分析LLMs的可解性检测能力，发现可解性信念具语言通用性，高资源语言的可解性检测忠实度更低。
中文摘要 AI 辅助
可解性检测是大语言模型（LLMs）数学推理中最具挑战性的方面之一。尽管先前的研究已对该能力进行了广泛探讨，但这些分析仅限于英语。因此，目前尚不清楚多语言场景下的失败是源于内部可解性信念的差异，还是源于表达该信念时对语言的依赖失败。为填补这一空白，我们引入了首个配对可解与不可解数学问题的多语言基准，将ReliableMath扩展至法语和希腊语。利用该基准，我们训练了预测可解性信念的多语言探针，并从行为、表征及忠实度方面分析了最先进LLMs的可解性检测能力。我们发现，可解性信念被编码为一种在很大程度上通用、与语言无关的特征，且英语等高资源语言尽管数学推理性能更强，却表现出更低的可解性检测忠实度。
英文摘要
Solvability detection is one of the most challenging aspects of mathematical reasoning for Large Language Models (LLMs). While prior work has studied this capability extensively, these analyses have been limited to English. Consequently, it remains unclear whether multilingual failures arise from differences in internal Solvability Belief or from language-dependent failures to express it. To address this gap, we introduce the first multilingual benchmark of paired solvable and unsolvable mathematical problems, extending ReliableMath to French and Greek. Using this, we train multilingual probes predicting Solvability Belief and analyze the solvability detection capabilities of state-of-the-art LLMs behaviorally, representationally, and in terms of faithfulness. We find that Solvability Belief is encoded as a largely universal, language-agnostic feature, and that higher-resource languages such as English, despite achieving stronger mathematical reasoning performance, exhibit lower solvability-detection faithfulness.
发表机构
- Institute for Language and Speech Processing, Athena Research Center（语言与语音处理研究所，雅典娜研究中心）
机构由 AI 辅助整理，请以论文原文为准。
