arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.03002cs.AI
收藏

零训练开销的组合问题学习算法

Zero Training Overhead Portfolios for Learning to Solve Combinatorial Problems

Cornell University(康奈尔大学)

Yiwei Bai, Wenting Zhao, Carla P. Gomes

摘要:

近年来,利用深度学习来解决组合优化(CO)问题的兴趣越来越大。典型的CO深度学习方法利用模型架构中的问题结构。尽管如此,模型选择仍然主要基于传统的机器学习设置。由于CO问题的离散性,单个模型不太可能完全学习问题。我们介绍ZTop,它代表零训练开销组合,一个简单而有效的模型选择和集成机制,用于学习解决组合问题。ZTop的灵感来自算法组合,一种流行的CO集成策略,特别是重启组合,它周期性地重启随机CO算法,实际上是用不同的算法探索搜索空间。我们已经观察到,在相同的训练轨迹中获得的经过良好训练的模型,具有相似的顶级验证性能,在非常不同的验证实例中表现良好。根据这一观察结果,ZTop集成了一组经过良好训练的模型,每个模型都提供了一个独特的启发式算法,并且可以顺序或并行地应用它们来解决测试实例。我们展示了ZTopping,即,使用ZTop集成策略与给定的深度学习方法,可以显著提高当前最先进的深度学习方法在三个原型CO域、最难的唯一解数独实例、具有挑战性的路由问题和图最大割问题上的性能,以及在多标签分类(具有大组合标签空间的机器学习任务)上的性能。

英文摘要:

There has been an increasing interest in harnessing deep learning to tackle combinatorial optimization (CO) problems in recent years. Typical CO deep learning approaches leverage the problem structure in the model architecture. Nevertheless, the model selection is still mainly based on the conventional machine learning setting. Due to the discrete nature of CO problems, a single model is unlikely to learn the problem entirely. We introduce ZTop, which stands for Zero Training Overhead Portfolio, a simple yet effective model selection and ensemble mechanism for learning to solve combinatorial problems. ZTop is inspired by algorithm portfolios, a popular CO ensembling strategy, particularly restart portfolios, which periodically restart a randomized CO algorithm, de facto exploring the search space with different heuristics. We have observed that well-trained models acquired in the same training trajectory, with similar top validation performance, perform well on very different validation instances. Following this observation, ZTop ensembles a set of well-trained models, each providing a unique heuristic with zero training overhead, and applies them, sequentially or in parallel, to solve the test instances. We show how ZTopping, i.e., using a ZTop ensemble strategy with a given deep learning approach, can significantly improve the performance of the current state-of-the-art deep learning approaches on three prototypical CO domains, the hardest unique-solution Sudoku instances, challenging routing problems, and the graph maximum cut problem, as well as on multi-label classification, a machine learning task with a large combinatorial label space.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑