arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2505.15068cs.AIcs.CLcs.LG

ModelingAgent:衔接大语言模型与数学建模以应对现实世界挑战

ModelingAgent: Bridging LLMs and Mathematical Modeling for Real-World Challenges

University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);IBM Research AI(IBM人工智能研究);Columbia University(哥伦比亚大学)

Cheng Qian, Hongyi Du, Hongru Wang, Xiusi Chen, Yuji Zhang, Avirup Sil, Chengxiang Zhai, Kathleen McKeown, Heng Ji

AI总结:

针对现有基准无法反映现实世界数学问题复杂性的问题,该研究推出含多领域开放式建模题的ModelingBench基准,提出多智能体框架ModelingAgent和专家参与的评估系统ModelingJudge,实验显示其方案效果接近人类专家水平。

摘要:

大语言模型(LLMs)的最新进展已在数学问题求解方面取得了实质性突破。然而,现有基准往往无法反映现实世界问题的复杂性,这类问题需要开放式、跨学科的推理以及计算工具的整合。为填补这一空白,我们推出了ModelingBench,这是一个全新的基准,包含来自不同领域数学建模竞赛的、受现实世界启发的开放式问题,涵盖城市交通优化到生态系统资源规划等场景。这些任务要求将自然语言转化为形式化的数学表述,运用合适的工具,并生成结构化、有依据的报告。ModelingBench还支持多种有效解法,体现了实际建模中的模糊性与创造性。我们还提出了ModelingAgent,这是一个多智能体框架,可协调工具使用、支持结构化工作流,并实现迭代式自我优化,以生成有充分依据且具创造性的解决方案。为评估输出结果,我们进一步提出了ModelingJudge,这是一个专家参与的系统,利用LLMs作为领域专业评审,从多个专家视角评估解决方案。实验结果表明,ModelingAgent大幅优于强基线模型,其生成的解决方案通常与人类专家的方案难以区分。综上,我们的工作为评估和推进开放式跨学科建模挑战中的现实问题求解能力提供了一个综合框架。

英文摘要:

Recent progress in large language models (LLMs) has enabled substantial advances in solving mathematical problems. However, existing benchmarks often fail to reflect the complexity of real-world problems, which demand open-ended, interdisciplinary reasoning and integration of computational tools. To address this gap, we introduce ModelingBench, a novel benchmark featuring real-world-inspired, open-ended problems from math modeling competitions across diverse domains, ranging from urban traffic optimization to ecosystem resource planning. These tasks require translating natural language into formal mathematical formulations, applying appropriate tools, and producing structured, defensible reports. ModelingBench also supports multiple valid solutions, capturing the ambiguity and creativity of practical modeling. We also present ModelingAgent, a multi-agent framework that coordinates tool use, supports structured workflows, and enables iterative self-refinement to generate well-grounded, creative solutions. To evaluate outputs, we further propose ModelingJudge, an expert-in-the-loop system leveraging LLMs as domain-specialized judges assessing solutions from multiple expert perspectives. Empirical results show that ModelingAgent substantially outperforms strong baselines and often produces solutions indistinguishable from those of human experts. Together, our work provides a comprehensive framework for evaluating and advancing real-world problem-solving in open-ended, interdisciplinary modeling challenges.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑