A distributed binary hypothesis testing (HT) problem over a noisy (discrete and memoryless) channel studied previously by the authors is investigated from the perspective of the strong converse property. It was shown by Ahlswede and Csiszár that a strong converse holds in the above setting when the channel is rate-limited and noiseless. Motivated by this observation, we show that the strong converse continues to hold in the noisy channel setting for a special case of HT known as testing against independence (TAI). The proof utilizes the blowing up lemma and the recent change of measure technique of Tyagi and Watanabe as the key tools.
By proving a strong converse, we strengthen the weak converse result by Salehkalaibar, Wigger and Wang (2017) concerning hypothesis testing against independence over a two-hop network with communication constraints. Our proof follows by judiciously combining two recently proposed techniques for proving strong converse theorems, namely the strong converse technique via reverse hypercontractivity by Liu, van Handel, and Verdú (2017) and the strong converse technique by Tyagi and Watanabe (2018), in which the authors used a change-of-measure technique and replaced hard Markov constraints with sof…
We study a class of $K$-encoder hypothesis testing against conditional independence problems. Under the criterion that stipulates minimization of the Type II error subject to a (constant) upper bound $ε$ on the Type I error, we characterize the set of encoding rates and exponent for both discrete memoryless and memoryless vector Gaussian settings. For the DM setting, we provide a converse proof and show that it is achieved using the Quantize-Bin-Test scheme of Rahman and Wagner. For the memoryless vector Gaussian setting, we develop a tight outer bound by means of a technique that relies on th…
We consider the group testing problem, in which one seeks to identify a subset of defective items within a larger set of items based on a number of noisy tests. While matching achievability and converse bounds are known in several cases of interest for i.i.d.~measurement matrices, less is known regarding converse bounds for arbitrary measurement matrices. We address this by presenting two converse bounds for arbitrary matrices and general noise models. First, we provide a strong converse bound ($\mathbb{P}[\mathrm{error}] \to 1$) that matches existing achievability bounds in several cases of i…
In this paper, we develop new test statistics for private hypothesis testing. These statistics are designed specifically so that their asymptotic distributions, after accounting for noise added for privacy concerns, match the asymptotics of the classical (non-private) chi-square tests for testing if the multinomial data parameters lie in lower dimensional manifolds (examples include goodness of fit and independence testing). Empirically, these new test statistics outperform prior work, which focused on noisy versions of existing statistics.
Volume-Refined Achievability and Converse Approximations for Noisy Permutation Channels
School of Electronics, Peking University（北京大学电子学院）；National Computer Network Emergency Response Technical Team/Coordination Center of China（国家计算机网络应急技术处理协调中心）
In this paper, an achievability region and a converse region for the two-user Gaussian interference channel with noisy channel-output feedback (G-IC-NOF) are presented. The achievability region is obtained using a random coding argument and three well-known techniques: rate splitting, superposition coding and backward decoding. The converse region is obtained using some of the existing perfect-output feedback outer-bounds as well as a set of new outer-bounds that are obtained by using genie-aided models of the original G-IC-NOF. Finally, it is shown that the achievability region and the conver…
In this research report, an achievability region and a converse region for the two-user Gaussian interference channel with noisy channel-output feedback (G-IC-NOF) are presented. The achievability region is obtained using a random coding argument and three well-known techniques: rate splitting, superposition coding and backward decoding. The converse region is obtained using some of the existing perfect-output feedback outer-bounds as well as a set of new outer-bounds that are obtained by using genie-aided models of the original G-IC-NOF. Finally, it is shown that the achievability region and …
The main contribution of this paper is a strong converse result for $K$-hop distributed hypothesis testing against independence with multiple (intermediate) decision centers under a Markov condition. Our result shows that the set of type-II error exponents that can simultaneously be achieved at all the terminals does not depend on the maximum permissible type-I error probabilities. Our strong converse proof is based on a change of measure argument and on the asymptotic proof of specific Markov chains. This proof method can also be used for other converse proofs, and is appealing because it doe…
University College London（伦敦大学学院）；University of Cambridge（剑桥大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Institute of Statistical Mathematics（统计数学研究所）
We provide a unifying framework linking two classes of statistics used in two-sample and independence testing: on the one hand, the energy distances and distance covariances from the statistics literature; on the other, maximum mean discrepancies (MMD), that is, distances between embeddings of distributions to reproducing kernel Hilbert spaces (RKHS), as established in machine learning. In the case where the energy distance is computed with a semimetric of negative type, a positive definite kernel, termed distance kernel, may be defined such that the MMD corresponds exactly to the energy dista…
We consider the problem of distributed estimation, where local processors observe independent samples conditioned on a common random parameter of interest, map the observations to a finite number of bits, and send these bits to a remote estimator over independent noisy channels. We derive converse results for this problem, such as lower bounds on Bayes risk. The main technical tools include a lower bound on the Bayes risk via mutual information and small ball probability, as well as strong data processing inequalities for the relative entropy. Our results can recover and improve some existing …
In this paper, we formally define and analyze the class of noisy permutation channels. The noisy permutation channel model constitutes a standard discrete memoryless channel (DMC) followed by an independent random permutation that reorders the output codeword of the DMC. While coding theoretic aspects of this model have been studied extensively, particularly in the context of reliable communication in network settings where packets undergo transpositions, and closely related models of DNA based storage systems have also been analyzed recently, we initiate an information theoretic study of this…
We study a hypothesis testing problem with a privacy constraint over a noisy channel and derive the performance of optimal tests under the Neyman-Pearson criterion. The fundamental limit of interest is the privacy-utility tradeoff (PUT) between the exponent of the type-II error probability and the leakage of the information source subject to a constant constraint on the type-I error probability. We provide an exact characterization of the asymptotic PUT for any non-vanishing type-I error probability. Our result implies that tolerating a larger type-I error probability cannot improve the PUT. S…
In this paper, the capacity region of the linear deterministic interference channel with noisy channel-output feedback (LD-IC-NF) is fully characterized. The proof of achievability is based on random coding arguments and rate splitting; block-Markov superposition coding; and backward decoding. The proof of converse reuses some of the existing outer bounds and includes new ones obtained using genie-aided models. Following the insight gained from the analysis of the LD-IC-NF, an achievability region and a converse region for the two-user Gaussian interference channel with noisy channel-output fe…
A distributed binary hypothesis testing problem is studied with one observer and two decision centers. Achievable type-II error exponents are derived for testing against conditional independence when the observer communicates with the two decision centers over one common and two individual noise-free bit pipes and when it communicates with them over a noisy broadcast channel (BC). The results are based on a coding and testing scheme that splits the observations into subblocks, so that transmitter and receivers can independently apply to each subblock either Gray-Wyner coordination coding with …
The paper presents exponentially-strong converses for source-coding, channel coding, and hypothesis testing problems. More specifically, it presents alternative proofs for the well-known exponentially-strong converse bounds for almost lossless source-coding with side-information and for channel coding over a discrete memoryless channel (DMC). These alternative proofs are solely based on a change of measure argument on the sets of conditionally or jointly typical sequences that result in a correct decision, and on the analysis of these measures in the asymptotic regime of infinite blocklengths.…
A novel blind estimate of the number of sources from noisy, linear mixtures is proposed. Based on Székely et al.'s distance correlation measure, we define the Sources' Dependency Criterion (SDC), from which our estimate arises. Unlike most previously proposed estimates, the SDC estimate exploits the full independence of the sources and noise, as well as the non-Gaussianity of the sources (as opposed to the Gaussianity of the noise), via implicit use of high-order statistics. This leads to a more robust, resilient and stable estimate w.r.t. the mixing matrix and the noise covariance structure. …
The achievable and converse regions for sparse representation of white Gaussian noise based on an overcomplete dictionary are derived in the limit of large systems. Furthermore, the marginal distribution of such sparse representations is also inferred. The results are obtained via the Replica method which stems from statistical mechanics. A direct outcome of these results is the introduction of sharp threshold for $\ell_{0}$-norm decoding in noisy compressed sensing, and its mean-square error for underdetermined Gaussian vector channels.
Error correction, authentication, and false acceptance, probabilities for communication over noisy quantum channels: converse upper bounds on the bit transmission rate
In this paper, we consider the group testing problem with adaptive test designs and noisy outcomes. We propose a computationally efficient four-stage procedure with components including random binning, identification of bins containing defective items, 1-sparse recovery via channel codes, and a "clean-up" step to correct any errors from the earlier stages. We prove that the asymptotic required number of tests comes very close to the best known information-theoretic achievability bound (which is based on computationally intractable decoding), and approaches a capacity-based converse bound in th…
The fully quantum reverse Shannon theorem establishes the optimal rate of noiseless classical communication required for simulating the action of many instances of a noisy quantum channel on an arbitrary input state, while also allowing for an arbitrary amount of shared entanglement of an arbitrary form. Turning this theorem around establishes a strong converse for the entanglement-assisted classical capacity of any quantum channel. This paper proves the strong converse for entanglement-assisted capacity by a completely different approach and identifies a bound on the strong converse exponent …
This paper presents an analysis of the concept of capacity for noisy com- putations, i.e. functions implemented by unreliable or random devices. An information theoretic model of noisy computation of a perfect function f (measurable function between sequence spaces) thanks to an unreliable device (random channel) F is given: a noisy computation is a product fxF of channels. A model of reliable computation based on input encoding and output decoding is also proposed. These models extend those of noisy communication channel and of reliable communication through a noisy channel. The capacity of a…
The problem of distributed testing against independence with variable-length coding is considered when the \emph{average} and not the \emph{maximum} communication load is constrained as in previous works. The paper characterizes the optimum type-II error exponent of a single sensor single decision center system given a maximum type-I error probability when communication is either over a noise-free rate-$R$ link or over a noisy discrete memoryless channel (DMC) with stop-feedback. Specifically, let $ε$ denote the maximum allowed type-I error probability. Then the optimum exponent of the system …
We establish the classical capacity of optical quantum channels as a sharp transition between two regimes---one which is an error-free regime for communication rates below the capacity, and the other in which the probability of correctly decoding a classical message converges exponentially fast to zero if the communication rate exceeds the classical capacity. This result is obtained by proving a strong converse theorem for the classical capacity of all phase-insensitive bosonic Gaussian channels, a well-established model of optical quantum communication channels, such as lossy optical fibers, …
In this work, we consider $q$-ary signature codes of length $k$ and size $n$ for a noisy adder multiple access channel. A signature code in this model has the property that any subset of codewords can be uniquely reconstructed based on any vector that is obtained from the sum (over integers) of these codewords. We show that there exists an algorithm to construct a signature code of length $k = \frac{2n\log{3}}{(1-2τ)\left(\log{n} + (q-1)\log{\fracπ{2}}\right)} +\mathcal{O}\left(\frac{n}{\log{n}(q+\log{n})}\right)$ capable of correcting $τk$ errors at the channel output, where $0\le τ< \frac{q-…
The group testing problem consists of determining a small set of defective items from a larger set of items based on a number of possibly-noisy tests, and is relevant in applications such as medical testing, communication protocols, pattern matching, and many more. One of the defining features of the group testing problem is the distinction between the non-adaptive and adaptive settings: In the non-adaptive case, all tests must be designed in advance, whereas in the adaptive case, each test can be designed based on the previous outcomes. While tight information-theoretic limits and near-optima…
This paper characterizes the optimal type-II error exponent for a distributed hypothesis testing-against-independence problem when the \emph{expected} rate of the sensor-detector link is constrained. Unlike for the well-known Ahlswede-Csiszar result that holds under a \emph{maximum} rate constraint and where a strong converse holds, here the optimal exponent depends on the allowed type-I error exponent. Specifically, if the type-I error probability is limited by $ε$, then the optimal type-II error exponent under an \emph{expected} rate constraint $R$ coincides with the optimal type-II error ex…
In this note we study Shannon capacity of channels in the context of classical Ramsey numbers. We overview some of the results on capacity of noisy channels modelled by graphs, and how some constructions may contribute to our knowledge of this capacity. We present an improvement to the constructions by Abbott and Song and thus establish new lower bounds for a special type of multicolor Ramsey numbers. We prove that our construction implies that the supremum of the Shannon capacity over all graphs with independence number 2 cannot be achieved by any finite graph power. This can be generalized t…
The group testing problem consists of determining a small set of defective items from a larger set of items based on a number of tests, and is relevant in applications such as medical testing, communication protocols, pattern matching, and more. In this paper, we revisit an efficient algorithm for noisy group testing in which each item is decoded separately (Malyutov and Mateev, 1980), and develop novel performance guarantees via an information-theoretic framework for general noise models. For the special cases of no noise and symmetric noise, we find that the asymptotic number of tests requir…
We consider the problem of reliable communication over multiple-access channels (MAC) where the channel is driven by an independent and identically distributed state process and the encoders and the decoder are provided with various degrees of asymmetric noisy channel state information (CSI). For the case where the encoders observe causal, asymmetric noisy CSI and the decoder observes complete CSI, we provide inner and outer bounds to the capacity region, which are tight for the sum-rate capacity. We then observe that, under a Markov assumption, similar capacity results also hold in the case w…
In this paper we consider the identification (ID) via multiple access channels (MACs). In the general MAC the ID capacity region includes the ordinary transmission (TR) capacity region. In this paper we discuss the converse coding theorem. We estimate two types of error probabilities of identification for rates outside capacity region, deriving some function which serves as a lower bound of the sum of two error probabilities of identification. This function has a property that it tends to zero as $n\to \infty$ for noisy channels satisfying the strong converse property. Using this property, we …
We study a special class of diamond channels which was introduced by Schein in 2001. In this special class, each diamond channel consists of a transmitter, a noisy relay, a noiseless relay and a receiver. We prove the capacity of this class of diamond channels by providing an achievable scheme and a converse. The capacity we show is strictly smaller than the cut-set bound. Our result also shows the optimality of a combination of decode-and-forward (DAF) and compress-and-forward (CAF) at the noisy relay node. This is the first example where a combination of DAF and CAF is shown to be capacity a…
Coding and testing schemes for binary hypothesis testing over noisy networks are proposed and their corresponding type-II error exponents are derived. When communication is over a discrete memoryless channel (DMC), our scheme combines Shimokawa-Han-Amari's hypothesis testing scheme with Borade's unequal error protection (UEP) for channel coding. A separate source channel coding architecture is employed. The resulting exponent is optimal for the newly introduced class of \emph{generalized testing against conditional independence}. When communication is over a MAC or a BC, our scheme combines hy…
Massachusetts Institute of Technology（麻省理工学院）；Harvard University（哈佛大学）；Broad Institute of MIT and Harvard（麻省理工学院和哈佛大学布罗德研究所）
In exploratory data analysis, we are often interested in identifying promising pairwise associations for further analysis while filtering out weaker, less interesting ones. This can be accomplished by computing a measure of dependence on all variable pairs and examining the highest-scoring pairs, provided the measure of dependence used assigns similar scores to equally noisy relationships of different types. This property, called equitability, is formalized in Reshef et al. [2015b]. In addition to equitability, measures of dependence can also be assessed by the power of their corresponding ind…
We revisit the distributed hypothesis testing (or hypothesis testing with communication constraints) problem from the viewpoint of privacy. Instead of observing the raw data directly, the transmitter observes a sanitized or randomized version of it. We impose an upper bound on the mutual information between the raw and randomized data. Under this scenario, the receiver, which is also provided with side information, is required to make a decision on whether the null or alternative hypothesis is in effect. We first provide a general lower bound on the type-II exponent for an arbitrary pair of hy…
