When the data are stored in a distributed manner, direct application of traditional statistical inference procedures is often prohibitive due to communication cost and privacy concerns. This paper develops and investigates two Communication-Efficient Accurate Statistical Estimators (CEASE), implemented through iterative algorithms for distributed optimization. In each iteration, node machines carry out computation in parallel and communicate with the central processor, which then broadcasts aggregated information to node machines for new updates. The algorithms adapt to the similarity among loss functions on node machines, and converge rapidly when each node machine has large enough sample size. Moreover, they do not require good initialization and enjoy linear converge guarantees under general conditions. The contraction rate of optimization errors is presented explicitly, with dependence on the local sample size unveiled. In addition, the improved statistical accuracy per iteration is derived. By regarding the proposed method as a multi-step statistical estimator, we show that statistical efficiency can be achieved in finite steps in typical statistical applications. In addition, we give the conditions under which the one-step CEASE estimator is statistically efficient. Extensive numerical experiments on both synthetic and real data validate the theoretical results and demonstrate the superior performance of our algorithms.
Communication Lower Bounds for Statistical Estimation Problems via a Distributed Data Processing Inequality
Princeton University（普林斯顿大学）；Toyota Technological Institute at Chicago（芝加哥丰田技术研究院）；IBM Research Almaden（IBM阿尔马登研究中心）
我们研究了高维分布式统计估计问题的统计误差和通信成本之间的权衡。在分布式稀疏高斯均值估计问题中，每个$m$机器接收来自具有未知均值$θ$的$d$维高斯分布的$n$数据点，该均值被承诺为$k$ -稀疏的。机器通过消息传递进行通信，目的是估计平均值$θ$。我们提供了一个紧密的（对数因子）之间的权衡估计误差和机器之间通信的位数。这直接导致了分布式\textit{sparse linear regression}问题的下限：为了实现统计极大极小误差，总通信至少是$Ω(\min\{n,d\}m)$，其中$n$是每台机器接收的观测数，$d$是环境维度。通过允许多轮迭代通信模型，这些较低的结果改进了[Sha14，SD'14]。我们也给出了第一个最佳的同时协议在稠密的情况下，均值估计。作为我们的主要技术，我们证明了一个\textit{distributed data processing inequality}，作为一般的数据处理不等式的推广，这可能是独立的兴趣和有用的其他问题。
SIMCom: Statistical Sniffing of Inter-Module Communications for Run-time Hardware Trojan Detection
Technische Universität Wien (TU Wien)（维也纳工业大学（TU Wien））；Tennessee Tech University（田纳西理工大学）；National University of Sciences and Technology (NUST)（国立科技大学（NUST））
Communication-Efficient and Byzantine-Robust Distributed Learning with Error Feedback
Halicioglu Data Science Institute, UC San Diego（哈利乔格鲁数据科学研究所，加州大学圣迭戈分校）；UC Berkeley（加州大学伯克利分校）；UMass Amherst（马萨诸塞大学阿默斯特分校）；EECS department, UC Berkeley（加州大学伯克利分校电子工程与计算机科学系）
