Achieving strong optimization generalization across diverse optimization problems while requiring limited training resources remains a challenging problem for optimization-oriented large language models (LLMs). Existing learning-based methods, however, typically require large-scale supervised data, expensive intermediate step verification, and extensive computational resources, while exhibiting limited optimization generalization across diverse optimization structures. In this paper, we propose \textsc{MiniOpt}, a lightweight optimization-specialized LLM trained through a reasoning-to-model-and-solve paradigm and a two-stage training framework with reinforcement learning with verifiable rewards (RLVR). MiniOpt encourages structure-oriented optimization learning by reinforcing the completeness of optimization formulations and optimization outcomes. Extensive experiments on eight optimization benchmarks demonstrate that MiniOpt consistently outperforms existing learning-based optimization models while requiring substantially fewer parameters and training data. To further understand optimization generalization, we introduce a complementary parameter-decoupled evaluation protocol that disentangles optimization structure understanding from numerical parameter processing. The results reveal that MiniOpt preserves optimization capability under parameter-decoupled settings significantly better than previous optimization-specialized models, suggesting that its performance gains primarily originate from stronger optimization structure understanding. These findings demonstrate an effective pathway toward efficient and generalizable optimization-specialized LLMs. The code is available at https://github.com/Hsiang-1/MiniOpt.
Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism
Institute of Natural Sciences, MOE-LSC, Shanghai Jiao Tong University(上海交通大学自然科学院);School of Mathematical Sciences, Shanghai Jiao Tong University(上海交通大学数学学院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Huawei Noah’s Ark Lab(华为诺亚实验室);Key Laboratory of Marine Intelligent Equipment and System, Ministry of Education(教育部海洋智能装备与系统重点实验室)
Bridging Formal Language with Chain-of-Thought Reasoning to Geometry Problem Solving
Shenzhen Research Institute of Big Data(深圳大数据研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen International Center for Industrial and Applied Mathematics(深圳国际工业与应用数学中心)
Reasoning While Asking: Transforming Reasoning Large Language Models from Passive Solvers to Proactive Inquirers
National Key Laboratory for Novel Software Technology(新型软件技术国家重点实验室);Artificial Intelligence Research Institute(人工智能研究院);Shenzhen Institutes of Advanced Technology(深圳先进技术研究院);University of California, Santa Cruz(加州大学圣克鲁兹分校);University of Texas, Dallas(德克萨斯大学达拉斯分校);University of Minnesota(明尼苏达大学)
Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Zhejiang University(浙江大学)
Know Your Limits : On the Faithfulness of LLMs as Solvers and Autoformalizers in Legal Reasoning
UC Santa Cruz(加州大学圣克鲁兹分校);Univ. Helsinki(赫尔辛基大学);CodeX, Stanford(斯坦福大学CodeX中心);Stanford University(斯坦福大学);Canyon Crest Academy(峡谷峰学院);Monta Vista High School(蒙塔维斯塔高中);Los Altos High School(洛斯阿尔托斯高中)
Absurd World: A Simple Yet Powerful Method to Absurdify the Real-world for Probing LLM Reasoning Capabilities
The Nueva School(新维学校);University of Southern California(南加州大学);Notre Dame College(诺特大学);Arizona State University(亚利桑那州立大学);National University of Singapore(新加坡国立大学)
R-Zero: Self-Evolving Reasoning LLM from Zero Data
Tencent AI Seattle Lab(腾讯AI西雅图实验室);Washington University in St. Louis(华盛顿大学圣路易斯分校);University of Maryland, College Park(马里兰大学科廷分校);The University of Texas at Dallas(德克萨斯大学达拉斯分校)
Teaching Algorithmic Reasoning via In-context Learning
Mila(米拉研究所);Université de Montreal(蒙特利尔大学);Google Research(谷歌研究院)
大语言模型(LLM)通过扩展模型和数据大小,显示出越来越多的上下文学习能力。尽管取得了这些进展,LLM仍然无法解决算法推理问题。虽然为最终答案提供了一个基本原理,导致了多步推理问题的进一步改进,但Anil et al. 2022表明,即使是简单的算法推理任务,如奇偶校验,也远未解决。在这项工作中,我们确定并研究了成功教授算法推理的四个关键阶段:(1)将算法制定为技能,(2)同时教授多种技能(技能积累),(3)教授如何组合技能(技能组合)和(4)教授如何使用技能作为工具。我们表明,它是可以教算法推理LLMs通过上下文学习,我们称之为算法提示。我们评估我们的方法在各种算术和定量推理任务,并表现出显着的提升性能比现有的提示技术。特别是,对于长奇偶校验、加法、乘法和减法,与最佳可用基线相比,我们分别实现了约10倍、9倍、5倍和2倍的误差减少。
RIMO: An Easy-to-Evaluate, Hard-to-Solve Olympiad Benchmark for Advanced Mathematical Reasoning
Artificial Intelligence Thrust, Information Hub, Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China(人工智能 thrust,信息中心,香港科学与技术大学(广州),广州,中国)
GAMR: A Guided Attention Model for (visual) Reasoning
Carney Institute for Brain Science(卡尼脑科学研究所);Brown University(布朗大学);Centre de Recherche Cerveau et Cognition(脑与认知研究中心);CNRS(法国国家科学研究中心);Université de Toulouse(图卢兹大学);Artificial and Natural Intelligence Toulouse Institute(图卢兹人工智能与自然智能研究所)
CAP-CoT: Cycle Adversarial Prompt for Improving Chain of Thoughts in LLM Reasoning
Department of Electronic Engineering, Kyung Hee University(韩国庆熙大学电子工程系);School of Computer Science and Engineering, University of Electronic Science and Technology of China(中国电子科技大学计算机科学与工程学院);Henan Polytechnic University(河南理工大学);School of Computing, Kyung Hee University(韩国庆熙大学计算机学院)