发表机构
BCG X AI Science Institute; Zapata Quantum（波士顿咨询X人工智能科学研究院; Zapata Quantum公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出一种自回归可微方法求解0-1整数规划，通过transformer预测比特并利用拉格朗日惩罚与Gumbel-softmax探索可行集，在非凸二次背包问题上优于开源求解器，并发现隧道效应现象。
AI 中文摘要
我们提出了一种自回归可微方法来解决0-1整数规划问题。我们固定二元变量的任意顺序，并训练一个transformer在保持可行集内的情况下预测下一个比特。我们的方法首先在由任意求解器提供的可行初始解上进行训练，从而允许我们在可行集内初始化transformer。随后，我们的过程采用拉格朗日惩罚项来惩罚不可行解，并且transformer通过使用Gumbel-softmax激活函数在松弛目标上进一步训练以探索可行集。我们在二次背包问题的非凸实例上测试了我们的方法，并展示了在多达10,000个二元变量的稠密问题上，相对于最先进的开源求解器的一致改进。特别是，我们实证地展示了一种类似于隧道效应的现象，即该方法实现的从二元变量到transformer连续权重的有效变量转换，使得能够跨越松弛目标景观中的障碍。
英文摘要
We introduce an autoregressive differentiable method to solve 0-1 integer programs. We fix an arbitrary order of the binary variables and we train a transformer to predict the next bit while remaining in the feasible set. Our method is first trained on feasible incumbents provided by any solver, thus allowing us to initialize the transformer in the feasible set. Our procedure then implements a Lagrangian penalty to penalize infeasible solutions, and the transformer is further trained to explore the feasible set using Gumbel-softmax activations on the relaxed objective. We have tested our method on non-convex instances of quadratic knapsack problem and demonstrated consistent improvement upon state-of-the-art open-source solvers for dense problems up to 10,000 binary variables. In particular, we empirically demonstrate a phenomenon akin to a tunneling effect where the effective change of variables from binary variable to the continuous weights of the transformer that the method implements enables crossing barriers in the relaxed objective landscape.
