Two parties observe independent copies of a $d$-dimensional vector and a scalar. They seek to test if their data is correlated or not, namely they seek to test if the norm $\|ρ\|_2$ of the correlation vector $ρ$ between their observations exceeds $τ$ or is it $0$. To that end, they communicate interactively and declare the output of the test. We show that roughly order $d/τ^2$ bits of communication are sufficient and necessary for resolving the distributed correlation testing problem above. Furthermore, we establish a lower bound of roughly $d^2/τ^2$ bits for communication needed for distributed correlation estimation, rendering the estimate-and-test approach suboptimal in communication required for distributed correlation testing. For the one-dimensional case with one-way communication, our bounds are tight even in the constant and provide a precise dependence of communication complexity on the probabilities of error of two types.
We investigate the quantize and binning scheme, known as the Shimokawa-Han-Amari (SHA) scheme, for the distributed hypothesis testing. We develop tools to evaluate the critical rate attainable by the SHA scheme. For a product of binary symmetric double sources, we present a sequential scheme that improves upon the SHA scheme.
This paper characterizes the optimal type-II error exponent for a distributed hypothesis testing-against-independence problem when the \emph{expected} rate of the sensor-detector link is constrained. Unlike for the well-known Ahlswede-Csiszar result that holds under a \emph{maximum} rate constraint and where a strong converse holds, here the optimal exponent depends on the allowed type-I error exponent. Specifically, if the type-I error probability is limited by $ε$, then the optimal type-II error exponent under an \emph{expected} rate constraint $R$ coincides with the optimal type-II error ex…
The distributed hypothesis testing problem with full side-information is studied. The trade-off (reliability function) between the two types of error exponents under limited rate is studied in the following way. First, the problem is reduced to the problem of determining the reliability function of channel codes designed for detection (in analogy to a similar result which connects the reliability function of distributed lossless compression and ordinary channel codes). Second, a single-letter random-coding bound based on a hierarchical ensemble, as well as a single-letter expurgated bound, are…
The problem of distributed binary hypothesis testing in the Gray-Wyner network with side information is studied in this paper. An observer has access to a discrete memoryless and stationary source and describes its observation to two detectors via one common and two private channels. The channels are considered error-free but rate-limited. Each detector also has access to its own discrete memoryless and stationary source, i.e., the side information. The goal is to perform two distinct binary hypothesis testings on the joint distribution of observations at detectors. Additionally, the observer …
In this paper, we consider sequential testing over a single-sensor, a single-decision center setup. At each time instant $t$, the sensor gets $k$ samples $(k>0)$ and describes the observed sequence until time $t$ to the decision center over a zero-rate noiseless link. The decision center sends a single bit of feedback to the sensor to request for more samples for compression/testing or to stop the transmission. We have characterized the optimal exponent of type-II error probability under the constraint that type-I error probability does not exceed a given threshold $ε\in (0,1)$ and also when t…
This paper investigates distributed computing systems where computations are split into "Map" and "Reduce" functions. A new coded scheme, called distributed computing and coded communication (D3C), is proposed, and its communication load is analyzed as a function of the available storage space and the number of intermediate values (IVA) to be computed. D3C achieves the smallest possible communication load for a given storage space, while a smaller number of IVAs need to be computed compared to Li et al.'s coded distributed computing (CDC) scheme. More generally, our scheme can flexibly trade b…
We study distributed binary hypothesis testing with a single sensor and two remote decision centers that are also equipped with local sensors. The communication between the sensor and the two decision centers takes place over three links: a shared link to both centers and an individual link to each of the two centers. All communication links are subject to expected rate constraints. This paper characterizes the optimal exponents region of the type-II error for given type-I error thresholds at the two decision centers and further simplifies the expressions in the special case of having only the…
We revisit the distributed hypothesis testing (or hypothesis testing with communication constraints) problem from the viewpoint of privacy. Instead of observing the raw data directly, the transmitter observes a sanitized or randomized version of it. We impose an upper bound on the mutual information between the raw and randomized data. Under this scenario, the receiver, which is also provided with side information, is required to make a decision on whether the null or alternative hypothesis is in effect. We first provide a general lower bound on the type-II exponent for an arbitrary pair of hy…
We consider a MapReduce-like distributed computing system. We derive a lower bound on the communication cost for any given storage and computation costs. This lower bound matches the achievable bound we proposed recently. As a result, we completely characterize the optimal tradeoff between the storage, the computation, and the communication. Our result generalizes the previous one by Li et al. to also account for the number of computed intermediate values.
Coded distributed computing can alleviate the communication load by leveraging the redundant storage and computation resources with coding techniques in distributed computing. In this paper, we study a MapReduce-type distributed computing framework over star topological network, where all the workers exchange information through a common access point. The optimal tradeoff among the normalized number of the stored files (storage load), computed intermediate values (computation load) and transmitted bits in the uplink and downlink (communication loads) are characterized. A coded computing scheme…
We propose a novel distributed iterative linear inverse solver method. Our method, PolyLin, has significantly lower communication cost, both in terms of number of rounds as well as number of bits, in comparison with the state of the art at the cost of higher computational complexity and storage. Our algorithm also has a built-in resilience to straggling and faulty computation nodes. We develop a natural variant of our main algorithm that trades off communication cost for computational complexity. Our method is inspired by ideas in error correcting codes.
A distributed binary hypothesis testing (HT) problem involving two parties, one referred to as the observer and the other as the detector is studied. The observer observes a discrete memoryless source (DMS) and communicates its observations to the detector over a discrete memoryless channel (DMC). The detector observes another DMS correlated with that at the observer, and performs a binary HT on the joint distribution of the two DMS's using its own observed data and the information received from the observer. The trade-off between the type I error probability and the type II error-exponent of …
In this paper we study the problem of signal detection in Gaussian noise in a distributed setting where the local machines in the star topology can communicate a single bit of information. We derive a lower bound on the Euclidian norm that the signal needs to have in order to be detectable. Moreover, we exhibit optimal distributed testing strategies that attain the lower bound.
In this paper we explore fundamental problems in randomized communication complexity such as computing Set Intersection on sets of size $k$ and Equality Testing between vectors of length $k$. Sağlam and Tardos and Brody et al. showed that for these types of problems, one can achieve optimal communication volume of $O(k)$ bits, with a randomized protocol that takes $O(\log^* k)$ rounds. Aside from rounds and communication volume, there is a \emph{third} parameter of interest, namely the \emph{error probability} $p_{\mathrm{err}}$. It is straightforward to show that protocols for Set Intersectio…
We study a distributed hypothesis testing setup where peripheral nodes send quantized data to the fusion center in a memoryless fashion. The \emph{expected} number of bits sent by each node under the null hypothesis is kept limited. We characterize the optimal decay rate of the mis-detection (type-II error) probability provided that false alarms (type-I error) are rare, and study the tradeoff between the communication rate and maximal type-II error decay rate. We resort to rate-distortion methods to provide upper bounds to the tradeoff curve and show that at high rates lattice quantization ach…
To process and transfer large amounts of data in emerging wireless services, it has become increasingly appealing to exploit distributed data communication and learning. Specifically, edge learning (EL) enables local model training on geographically disperse edge nodes and minimizes the need for frequent data exchange. However, the current design of separating EL deployment and communication optimization does not yet reap the promised benefits of distributed signal processing, and sometimes suffers from excessive signalling overhead, long processing delay, and unstable learning convergence. In…
Penn State Behrend（宾夕法尼亚州立大学贝伦德分校）；University of California, Davis（加州大学戴维斯分校）；University of Iowa（爱荷华大学）
Due to the big size of data and limited data storage volume of a single computer or a single server, data are often stored in a distributed manner. Thus, performing large-scale machine learning operations with the distributed datasets through communication networks is often required. In this paper, we study the convergence rate of the distributed dual coordinate ascent for distributed machine learning problems in a general tree-structured network. Since a tree network model can be understood as the generalization of a star network model, our algorithm can be thought of as the generalization of…
We consider the problem of distributed hypothesis testing (or social learning) where a network of agents seeks to identify the true state of the world from a finite set of hypotheses, based on a series of stochastic signals that each agent receives. Prior work on this problem has provided distributed algorithms that guarantee asymptotic learning of the true state, with corresponding efforts to improve the rate of learning. In this paper, we first argue that one can readily modify existing asymptotic learning algorithms to enable learning in finite time, effectively yielding arbitrarily large (…
Deakin University（迪肯大学）；University of Oulu（奥卢大学）；Korea University（高丽大学）；Yonsei University（延世大学）；CentraleSupélec, Université Paris-Saclay（巴黎-萨克雷大学中央理工-高等电力学院）
Machine learning (ML) is a promising enabler for the fifth generation (5G) communication systems and beyond. By imbuing intelligence into the network edge, edge nodes can proactively carry out decision-making, and thereby react to local environmental changes and disturbances while experiencing zero communication latency. To achieve this goal, it is essential to cater for high ML inference accuracy at scale under time-varying channel and network dynamics, by continuously exchanging fresh data and ML model updates in a distributed way. Taming this new kind of data traffic boils down to improving…
We study distributed estimation of a Gaussian mean under communication constraints in a decision theoretical framework. Minimax rates of convergence, which characterize the tradeoff between the communication costs and statistical accuracy, are established in both the univariate and multivariate settings. Communication-efficient and statistically optimal procedures are developed. In the univariate case, the optimal rate depends only on the total communication budget, so long as each local machine has at least one bit. However, in the multivariate case, the minimax rate depends on the specific a…
