arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.17320math.OCcs.LG
收藏

通过学习优化加速多块约束优化

Accelerating Multi-Block Constrained Optimization Through Learning to Optimize

University of Maryland(马里兰大学)

Ling Liang, Cameron Austin, Haizhao Yang

AI总结:

针对多块ADMM不保证收敛及MPALM对惩罚参数敏感的问题,提出基于监督学习的L2O方法自适应选择超参数,在Lasso和最优传输问题上优于现有替代方案,适用于一般线性约束复合优化。

摘要:

学习优化(L2O)方法,包括算法展开、即插即用方法和超参数学习,已引起广泛关注,并成功应用于交替方向乘子法(ADMM)及其变体。然而,将L2O自然扩展到多块ADMM类方法在很大程度上仍未被探索。这种扩展至关重要,因为多块方法利用优化问题的可分离结构,大幅降低了每次迭代的复杂度。鉴于经典的多块ADMM不保证收敛,与多块ADMM形式相似且能确保收敛的Majorized Proximal Augmented Lagrangian Method(MPALM)在此设定下更为合适。尽管具有理论优势,MPALM的性能对惩罚参数的选择高度敏感。为了解决这一局限性,我们提出了一种新颖的L2O方法,通过监督学习自适应地选择该超参数。我们将该方法应用于Lasso问题和最优传输问题,证明了其通用性和有效性。我们的数值结果表明,所提出的框架优于流行的替代方案。鉴于其适用于一般的线性约束复合优化问题,这项工作为广泛的潜在现实世界应用打开了大门。

英文摘要:

Learning to Optimize (L2O) approaches, including algorithm unrolling, plug-and-play methods, and hyperparameter learning, have garnered significant attention and have been successfully applied to the Alternating Direction Method of Multipliers (ADMM) and its variants. However, the natural extension of L2O to multi-block ADMM-type methods remains largely unexplored. Such an extension is critical, as multi-block methods leverage the separable structure of optimization problems, offering substantial reductions in per-iteration complexity. Given that classical multi-block ADMM does not guarantee convergence, the Majorized Proximal Augmented Lagrangian Method (MPALM), which shares a similar form with multi-block ADMM and ensures convergence, is more suitable in this setting. Despite its theoretical advantages, MPALM's performance is highly sensitive to the choice of penalty parameters. To address this limitation, we propose a novel L2O approach that adaptively selects this hyperparameter using supervised learning. We demonstrate the versatility and effectiveness of our method by applying it to the Lasso problem and the optimal transport problem. Our numerical results show that the proposed framework outperforms popular alternatives. Given its applicability to generic linearly constrained composite optimization problems, this work opens the door to a wide range of potential real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑