arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1612.09171math.OCcs.DScs.GTmath.DS
收藏

分析异步坐标下降和同步的统一方法

A Unified Approach to Analyzing Asynchronous Coordinate Descent and Tatonnement

Yun Kuen Cheung, Richard Cole

摘要:

本文关注迭代过程中的价格波动,重点关注梯度下降和tatonnement,这是一个基本的价格动态。梯度下降法是求解凸函数极小化问题的一类重要迭代算法。传统上,梯度下降是一个顺序和同步的过程,尽管自20世纪80年代以来已经研究了分布式和异步变体。坐标下降是梯度下降的一种常见研究版本。在本文中,我们专注于异步坐标下降的凸函数$F:\mathbb{R}^n\rightarrow\mathbb{R}$的形式$F(x) = f(x) + \sum_{k=1}^n Ψ_k(x_k)$,其中$f:\mathbb{R}^n\rightarrow\mathbb{R}$是一个光滑的凸函数,和每个$Ψ_k:\mathbb{R}\rightarrow\mathbb{R}$是一个单变量,可能是非光滑的凸函数。这些函数出现在许多数据分析和机器学习问题中。我们给出了新的分析循环坐标下降,并行异步随机坐标下降,而一般的最坏情况下的并行异步坐标下降。对于所有这些,我们要么获得大幅改善的界限,或提供第一个分析。我们的分析都使用一个共同的摊销框架。这个框架的应用程序的异步随机版本需要一些新的想法,因为它是不明显的,如何确保一个均匀的分布,它是在面对异步行动,可能会撤消均匀性需要。我们相信,我们的方法可能适用于其他迭代异步随机过程的分析。我们扩展的框架表明,异步版本的tatonnement,一个基本的价格动态广泛研究的一般均衡理论,收敛到一个市场均衡的费雪市场与CES公用事业或Leontief公用事业,tatonnement是相当于坐标下降。

英文摘要:

This paper concerns asynchrony in iterative processes, focusing on gradient descent and tatonnement, a fundamental price dynamic. Gradient descent is an important class of iterative algorithms for minimizing convex functions. Classically, gradient descent has been a sequential and synchronous process, although distributed and asynchronous variants have been studied since the 1980s. Coordinate descent is a commonly studied version of gradient descent. In this paper, we focus on asynchronous coordinate descent on convex functions $F:\mathbb{R}^n\rightarrow\mathbb{R}$ of the form $F(x) = f(x) + \sum_{k=1}^n Ψ_k(x_k)$, where $f:\mathbb{R}^n\rightarrow\mathbb{R}$ is a smooth convex function, and each $Ψ_k:\mathbb{R}\rightarrow\mathbb{R}$ is a univariate and possibly non-smooth convex function. Such functions occur in many data analysis and machine learning problems. We give new analyses of cyclic coordinate descent, a parallel asynchronous stochastic coordinate descent, and a rather general worst-case parallel asynchronous coordinate descent. For all of these, we either obtain sharply improved bounds, or provide the first analyses. Our analyses all use a common amortized framework. The application of this framework to the asynchronous stochastic version requires some new ideas, for it is not obvious how to ensure a uniform distribution where it is needed in the face of asynchronous actions that may undo uniformity. We believe that our approach may well be applicable to the analysis of other iterative asynchronous stochastic processes. We extend the framework to show that an asynchronous version of tatonnement, a fundamental price dynamic widely studied in general equilibrium theory, converges toward a market equilibrium for Fisher markets with CES utilities or Leontief utilities, for which tatonnement is equivalent to coordinate descent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑