高效的分布式分类和优化协议
Efficient Protocols for Distributed Classification and Optimization
摘要：
在分布式学习中，目标是以最少（昂贵）的通信对分布在多个节点上的数据执行学习任务。先前的工作（Daume III等人，2012）提出了一种通用模型，该模型限制了学习分类器所需的通信，同时允许$\eps$在跨节点逆向分布的线性可分离数据上的训练误差。在这项工作中，我们开发了这个基本模型的关键改进和扩展。我们的第一个结果是一个两方的乘法权重更新为基础的协议，使用$O(d^2 \log{1/\eps})$字的通信分类分布式数据在任意维度$d$，$\eps$ -最佳。这很容易扩展到对具有$O(kd^2 \log{1/\eps})$通信字的$k$节点的分类。我们提出的协议是简单的实现，是相当有效的基线相比，我们的实证结果表明。此外，我们说明了一般的算法设计范例做有效的学习分布式数据。我们展示了如何有效地解决固定维和高维线性规划在分布式环境中的约束可能分布在节点。由于许多学习问题可以被视为凸优化问题，其中约束由各个点生成，因此可以对许多典型的分布式学习场景进行建模。我们的技术利用了一种新的连接多通道流，以及适应乘法权重更新框架更普遍的分布式设置。因此，我们的方法扩展到使用这些技术可解决的问题的广泛范围。
英文摘要：
In distributed learning, the goal is to perform a learning task over data distributed across multiple nodes with minimal (expensive) communication. Prior work (Daume III et al., 2012) proposes a general model that bounds the communication required for learning classifiers while allowing for $\eps$ training error on linearly separable data adversarially distributed across nodes. In this work, we develop key improvements and extensions to this basic model. Our first result is a two-party multiplicative-weight-update based protocol that uses $O(d^2 \log{1/\eps})$ words of communication to classify distributed data in arbitrary dimension $d$, $\eps$-optimally. This readily extends to classification over $k$ nodes with $O(kd^2 \log{1/\eps})$ words of communication. Our proposed protocol is simple to implement and is considerably more efficient than baselines compared, as demonstrated by our empirical results. In addition, we illustrate general algorithm design paradigms for doing efficient learning over distributed data. We show how to solve fixed-dimensional and high dimensional linear programming efficiently in a distributed setting where constraints may be distributed across nodes. Since many learning problems can be viewed as convex optimization problems where constraints are generated by individual points, this models many typical distributed learning scenarios. Our techniques make use of a novel connection from multipass streaming, as well as adapting the multiplicative-weight-update framework more generally to a distributed setting. As a consequence, our methods extend to the wide range of problems solvable using these techniques.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
