arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1209.1076cs.DC
收藏

分布式优化中的通信/计算权衡

Communication/Computation Tradeoffs in Consensus-Based Distributed Optimization

Konstantinos I. Tsianos, Sean Lawlor, Michael G. Rabbat

摘要:

我们研究基于共识的分布式优化算法的可扩展性,考虑两个问题:有多少处理器,我们应该使用一个给定的问题,以及多久应该沟通时,通信是不自由的。我们分析的核心是一个问题特定的值$r$,它量化了通信/计算的权衡。我们表明,组织节点之间的通信作为一个$k$ -定期扩展图(Reingold,Vadhan,和Wigderson,2002年)产生加速,而当所有对节点通信(如在一个完整的图),有一个最佳的处理器数量取决于$r$。令人惊讶的是,在达到固定精度水平的时间方面,随着计算的进行,通信的频率越来越低,可以获得加速。一个真实的集群解决度量学习和非光滑凸最小化任务的实验证明了理论和实践之间的高度一致。

英文摘要:

We study the scalability of consensus-based distributed optimization algorithms by considering two questions: How many processors should we use for a given problem, and how often should they communicate when communication is not free? Central to our analysis is a problem-specific value $r$ which quantifies the communication/computation tradeoff. We show that organizing the communication among nodes as a $k$-regular expander graph (Reingold, Vadhan, and Wigderson, 2002) yields speedups, while when all pairs of nodes communicate (as in a complete graph), there is an optimal number of processors that depends on $r$. Surprisingly, a speedup can be obtained, in terms of the time to reach a fixed level of accuracy, by communicating less and less frequently as the computation progresses. Experiments on a real cluster solving metric learning and non-smooth convex minimization tasks demonstrate strong agreement between theory and practice.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑