两块与多块ADMM：收敛性的经验评估
Two-block vs. Multi-block ADMM: An empirical evaluation of convergence
摘要：
交替方向乘子法（ADMM）已成为一种广泛使用的凸优化方法，特别是在数据挖掘中经常遇到大型优化问题。ADMM具有几个理想的属性，包括将大型问题分解为较小的易处理子问题的能力和易于并行化，这在这些场景中是必不可少的。ADMM最常见的形式是两块，其中两组原始变量交替更新。近年来，多块ADMM取得了进展，它顺序更新两个以上的原始变量块。在本文中，我们研究的经验问题：{ \em是两块ADMM总是可比的顺序多块ADMM解决一个等价的问题？}在多任务学习中出现的优化问题的背景下，通过一组全面的实验，我们令人惊讶地表明，多块ADMM在优化性能上始终优于两块ADMM，因此在所有数据集和整个双步长范围内的预测性能。我们的研究结果有一个重要的实际意义：而不是简单地使用流行的两块ADMM，一个可以大大受益于实验与多块ADMM应用到一个等效的问题。
英文摘要：
Alternating Direction Method of Multipliers (ADMM) has become a widely used optimization method for convex problems, particularly in the context of data mining in which large optimization problems are often encountered. ADMM has several desirable properties, including the ability to decompose large problems into smaller tractable sub-problems and ease of parallelization, that are essential in these scenarios. The most common form of ADMM is the two-block, in which two sets of primal variables are updated alternatingly. Recent years have seen advances in multi-block ADMM, which update more than two blocks of primal variables sequentially. In this paper, we study the empirical question: {\em Is two-block ADMM always comparable with sequential multi-block ADMM solving an equivalent problem?} In the context of optimization problems arising in multi-task learning, through a comprehensive set of experiments we surprisingly show that multi-block ADMM consistently outperformed two-block ADMM on optimization performance, and as a consequence on prediction performance, across all datasets and for the entire range of dual step sizes. Our results have an important practical implication: rather than simply using the popular two-block ADMM, one may considerably benefit from experimenting with multi-block ADMM applied to an equivalent problem.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
