BOPO:基于最佳锚定与目标引导偏好优化的神经组合优化
BOPO: Neural Combinatorial Optimization via Best-anchored and Objective-guided Preference Optimization
AI总结:
本文提出BOPO训练范式,通过最佳锚定偏好对构建和目标引导的成对损失函数,利用解的目标值差异提升神经组合优化的样本效率与推理性能,在JSP、TSP和FJSP上显著缩小最优性差距。
摘要:
神经组合优化(NCO)已成为解决NP难问题的一种有前景的方法。然而,主流的基于强化学习的方法由于奖励稀疏和解决方案利用不足而面临样本效率低下的问题。我们提出最佳锚定与目标引导偏好优化(BOPO),这是一种通过目标值利用解决方案偏好的训练范式。该方法引入:(1)一种最佳锚定的偏好对构建方式,以更好地探索和利用解决方案;(2)一种目标引导的成对损失函数,通过目标差异自适应地缩放梯度,从而摆脱对奖励模型或参考策略的依赖。在作业车间调度问题(JSP)、旅行商问题(TSP)和柔性作业车间调度问题(FJSP)上的实验表明,BOPO优于最先进的神经方法,在高效推理的同时显著缩小了最优性差距。BOPO与架构无关,能够与现有NCO模型无缝集成,并将偏好优化确立为组合优化的一个原则性框架。
英文摘要:
Neural Combinatorial Optimization (NCO) has emerged as a promising approach for NP-hard problems. However, prevailing RL-based methods suffer from low sample efficiency due to sparse rewards and underused solutions. We propose Best-anchored and Objective-guided Preference Optimization (BOPO), a training paradigm that leverages solution preferences via objective values. It introduces: (1) a best-anchored preference pair construction for better explore and exploit solutions, and (2) an objective-guided pairwise loss function that adaptively scales gradients via objective differences, removing reliance on reward models or reference policies. Experiments on Job-shop Scheduling Problem (JSP), Traveling Salesman Problem (TSP), and Flexible Job-shop Scheduling Problem (FJSP) show BOPO outperforms state-of-the-art neural methods, reducing optimality gaps impressively with efficient inference. BOPO is architecture-agnostic, enabling seamless integration with existing NCO models, and establishes preference optimization as a principled framework for combinatorial optimization.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。