arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.01075cs.AI
收藏

ConvexBench: LLMs能否识别凸函数?

ConvexBench: Can LLMs Recognize Convex Functions?

UC Santa Barbara(加州大学圣芭芭拉分校);University of Pennsylvania(宾夕法尼亚大学);UC San Diego(加州大学圣地亚哥分校);The Ohio State University(俄亥俄州立大学)

Yepeng Liu, Yu Huang, Yu-Xiang Wang, Yingbin Liang, Yuheng Bu

AI总结:

ConvexBench通过分治框架解决LLM在深度函数组合中识别凸性的挑战,显著提升大深度下的性能。

摘要:

凸分析是现代数学的一个分支,具有许多应用。随着大语言模型(LLMs)开始自动化研究级别的数学和科学,LLMs能够理解和推理凸性变得至关重要。我们介绍了ConvexBench,这是一个可扩展且可机械验证的基准,用于测试LLMs是否能够识别在深度函数组合下的符号目标的凸性。在前沿LLMs上的实验揭示了一个明显的组合推理差距:随着深度增加,性能迅速下降,从深度2时的F1分数为1.0下降到深度100时约为0.2。对模型推理轨迹的检查表明有两种失败模式:解析失败和懒惰推理。为了解决这些限制,我们提出了一种代理分治框架,该框架(i)将解析任务委托给外部工具以构建抽象语法树(AST),(ii)强制对每个中间子表达式进行递归推理并提供聚焦上下文。该框架能够可靠地缓解深度组合失败,从而在大深度(例如深度100时F1分数=1.0)上实现显著的性能提升。

英文摘要:

Convex analysis is a modern branch of mathematics with many applications. As Large Language Models (LLMs) start to automate research-level math and sciences, it is important for LLMs to demonstrate the ability to understand and reason with convexity. We introduce \cb, a scalable and mechanically verifiable benchmark for testing \textit{whether LLMs can identify the convexity of a symbolic objective under deep functional composition.} Experiments on frontier LLMs reveal a sharp compositional reasoning gap: performance degrades rapidly with increasing depth, dropping from an F1-score of $1.0$ at depth $2$ to approximately $0.2$ at depth $100$. Inspection of models' reasoning traces indicates two failure modes: \textit{parsing failure} and \textit{lazy reasoning}. To address these limitations, we propose an agentic divide-and-conquer framework that (i) offloads parsing to an external tool to construct an abstract syntax tree (AST) and (ii) enforces recursive reasoning over each intermediate sub-expression with focused context. This framework reliably mitigates deep-composition failures, achieving substantial performance improvement at large depths (e.g., F1-Score $= 1.0$ at depth $100$).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑