arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.25832cs.LGcs.AI
收藏

MiniOpt: 在有限资源下推理建模与求解通用优化问题

MiniOpt: Reasoning to Model and Solve General Optimization Problems with Limited Resources

East China Normal University(华东师范大学);AntGroup(蚂蚁集团);Southern University of Science and Technology(南方科技大学);Nanjing University(南京大学)

Ke Zhao, Zixiang Di, Hong Qian, Xiang Shu, Yaolin Wen, Qitao Shi, Bingdong Li, Xingyu Lu, Xiangfeng Wang, Jun Zhou, Ke Tang, Yang Yu

AI总结:

提出MiniOpt强化学习框架,通过“推理-建模-求解”范式,结合层次化奖励函数OptReward和优化导向策略优化,使3B参数模型在多种优化问题上达到强泛化,且训练资源需求低。

摘要:

在多样化优化问题上实现强优化泛化,同时需要有限的训练资源,对于面向优化的大型语言模型(LLM)来说仍然是一个具有挑战性的问题。现有方法通常依赖大规模监督数据集、昂贵的推理标注和昂贵的中间步骤验证,导致大量训练开销。为了解决这些挑战,我们提出了MiniOpt,一个强化学习框架,通过“推理-建模-求解”范式学习解决优化问题。MiniOpt将优化推理分解为结构化的优化建模和可执行的求解器生成。基于这一范式,我们引入了OptReward,一个具有层次化评分结构的奖励函数,联合评估公式和解决方案,从而无需专家演示即可实现有效的策略学习。我们进一步开发了一种面向优化的策略优化方法,提高了探索效率,并稳定了紧凑模型的强化学习。大量实验表明,MiniOpt-3B在各种优化类型、问题场景和任务领域上表现出强大的优化泛化能力。对于参数少于10B的模型,MiniOpt系列实现了最高的平均求解准确率(SA)。对于参数超过10B的模型,MiniOpt仍然表现出有竞争力的性能。这些结果表明,面向优化的奖励设计和强化学习为开发具有强优化泛化能力的紧凑型优化专用语言模型提供了一条有效途径。代码可在以下网址获取:https://this URL。

英文摘要:

Achieving strong optimization generalization across diverse optimization problems while requiring limited training resources remains a challenging problem for optimization-oriented large language models (LLMs). Existing learning-based methods, however, typically require large-scale supervised data, expensive intermediate step verification, and extensive computational resources, while exhibiting limited optimization generalization across diverse optimization structures. In this paper, we propose \textsc{MiniOpt}, a lightweight optimization-specialized LLM trained through a reasoning-to-model-and-solve paradigm and a two-stage training framework with reinforcement learning with verifiable rewards (RLVR). MiniOpt encourages structure-oriented optimization learning by reinforcing the completeness of optimization formulations and optimization outcomes. Extensive experiments on eight optimization benchmarks demonstrate that MiniOpt consistently outperforms existing learning-based optimization models while requiring substantially fewer parameters and training data. To further understand optimization generalization, we introduce a complementary parameter-decoupled evaluation protocol that disentangles optimization structure understanding from numerical parameter processing. The results reveal that MiniOpt preserves optimization capability under parameter-decoupled settings significantly better than previous optimization-specialized models, suggesting that its performance gains primarily originate from stronger optimization structure understanding. These findings demonstrate an effective pathway toward efficient and generalizable optimization-specialized LLMs. The code is available at https://github.com/Hsiang-1/MiniOpt.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑