arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.11074cs.CLcs.AI
收藏

RoMath:罗马尼亚语数学推理基准

RoMath: A Mathematical Reasoning Benchmark in Romanian

POLITEHNICA Bucharest National University of Science and Technology(巴格达理工大学科学与技术国立大学);Interdisciplinary School of Doctoral Studies, University of Bucharest(布加勒斯特大学跨学科博士研究学院)

Adrian Cosma, Ana-Maria Bucur, Emilian Radoi

AI总结:

针对现有数学推理基准多以英语为中心的问题,本文提出低资源语言罗马尼亚语的数学推理基准RoMath,含三个覆盖多领域难度的子集,测试了开源模型并强调了小语种专用资源的重要性。

摘要:

长期以来,数学主要通过自然语言来传递,以供人类理解。随着机械化数学和证明助手的兴起,人们对非形式化数学文本的理解需求日益增长,但现有大多数基准仅聚焦于英语,忽略了其他语言。本文介绍了RoMath——一套罗马尼亚语数学推理基准套件,包含三个子集:Baccalaureate(高中毕业考试)、Competitions(竞赛题)和Synthetic(合成题),覆盖多个数学领域和难度层级,旨在改进非英语语言模型并推动多语言AI发展。RoMath聚焦于罗马尼亚语这一具有独特语言特征的低资源语言,解决了以英语为中心的模型的局限性,强调了不能仅依赖简单自动翻译、需打造专用资源的必要性。我们对多个开源权重语言模型进行了基准测试,凸显了为代表性不足的语言构建资源的重要性。代码和数据集将对外开放。

英文摘要:

Mathematics has long been conveyed through natural language, primarily for human understanding. With the rise of mechanized mathematics and proof assistants, there is a growing need to understand informal mathematical text, yet most existing benchmarks focus solely on English, overlooking other languages. This paper introduces RoMath, a Romanian mathematical reasoning benchmark suite comprising three subsets: Baccalaureate, Competitions and Synthetic, which cover a range of mathematical domains and difficulty levels, aiming to improve non-English language models and promote multilingual AI development. By focusing on Romanian, a low-resource language with unique linguistic features, RoMath addresses the limitations of Anglo-centric models and emphasizes the need for dedicated resources beyond simple automatic translation. We benchmark several open-weight language models, highlighting the importance of creating resources for underrepresented languages. Code and datasets are be made available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑