Improved Bounds and Algorithms for Sparsity-Constrained Group Testing
Nelvin Tan, Jonathan Scarlett
备注：
This paper has been merged with concurrent work to form arXiv:2004.11860. See v2 (arXiv:2004.03119v2) for a 5-page ISIT version with the adaptive setting only
In group testing, the goal is to identify a subset of defective items within a larger set of items based on tests whose outcomes indicate whether any defective item is present. This problem is relevant in areas such as medical testing, data science, communications, and many more. Motivated by physical considerations, we consider a sparsity-based constrained setting (Gandikota et al., 2019) in which the testing procedure is subject to one of the following two constraints: items are finitely divisible and thus may participate in at most $γ$ tests; or tests are size-constrained to pool no more than $ρ$ items per test. While information-theoretic limits and algorithms are known for the non-adaptive setting, relatively little is known in the adaptive setting. We address this gap by providing an information-theoretic converse that holds even in the adaptive setting, as well as a near-optimal noiseless adaptive algorithm for $γ$-divisible items. In broad scaling regimes, our upper and lower bounds asymptotically match up to a factor of $e$. We also present a simple asymptotically optimal adaptive algorithm for $ρ$-sized tests. In addition, in the non-adaptive setting with $γ$-divisible items, we use the Definite Defectives (DD) decoding algorithm and study bounds on the required number of tests for vanishing error probability under the random near-constant test-per-item design. We show that the number of tests required can be significantly less than the Combinatorial Orthogonal Matching Pursuit (COMP) decoding algorithm, and is asymptotically optimal in broad scaling regimes.
The group testing problem consists of determining a small set of defective items from a larger set of items based on a number of tests, and is relevant in applications such as medical testing, communication protocols, pattern matching, and more. In this paper, we revisit an efficient algorithm for noisy group testing in which each item is decoded separately (Malyutov and Mateev, 1980), and develop novel performance guarantees via an information-theoretic framework for general noise models. For the special cases of no noise and symmetric noise, we find that the asymptotic number of tests requir…
In this paper, we consider the problem of noiseless non-adaptive probabilistic group testing, in which the goal is high-probability recovery of the defective set. We show that in the case of $n$ items among which $k$ are defective, the smallest possible number of tests equals $\min\{ C_{k,n} k \log n, n\}$ up to lower-order asymptotic terms, where $C_{k,n}$ is a uniformly bounded constant (varying depending on the scaling of $k$ with respect to $n$) with a simple explicit expression. The algorithmic upper bound follows from a minor adaptation of an existing analysis of the Definite Defectives …
In group testing, the goal is to identify a subset of defective items within a larger set of items based on tests whose outcomes indicate whether at least one defective item is present. This problem is relevant in areas such as medical testing, DNA sequencing, communication protocols, and many more. In this paper, we study (i) a sparsity-constrained version of the problem, in which the testing procedure is subjected to one of the following two constraints: items are finitely divisible and thus may participate in at most $γ$ tests; or tests are size-constrained to pool no more than $ρ$ items pe…
This paper treats the problem of minimizing a general continuously differentiable function subject to sparsity constraints. We present and analyze several different optimality criteria which are based on the notions of stationarity and coordinate-wise optimality. These conditions are then used to derive three numerical algorithms aimed at finding points satisfying the resulting optimality criteria: the iterative hard thresholding method and the greedy and partial sparse-simplex methods. The first algorithm is essentially a gradient projection method while the remaining two algorithms are of co…
Multi-cell cooperative processing with limited backhaul traffic is studied for cellular uplinks. Aiming at reduced backhaul overhead, a sparsity-regularized multi-cell receive-filter design problem is formulated. Both unstructured distributed cooperation as well as clustered cooperation, in which base station groups are formed for tight cooperation, are considered. Dynamic clustered cooperation, where the sparse equalizer and the cooperation clusters are jointly determined, is solved via alternating minimization based on spectral clustering and group-sparse regression. Furthermore, decentraliz…
In probabilistic nonadaptive group testing (PGT), we aim to characterize the number of pooled tests necessary to identify a random $k$-sparse vector of defectives with high probability. Recent work has shown that $n$ tests are necessary when $k =ω(n/\log n)$. It is also known that $O(k \log n)$ tests are necessary and sufficient in other regimes. This leaves open the important sparsity regime where the probability of a defective item is $\sim 1/\log n$ (or $k = Θ(n/\log n)$) where the number of tests required is linear in $n$. In this work we aim to exactly characterize the number of tests in …
In network tomography, one goal is to identify a small set of failed links in a network, by sending a few packets through the network and seeing which reach their destination. This problem can be seen as a variant of combinatorial group testing, which has been studied before under the moniker "graph-constrained group testing." The main contribution of this work is to show that for most graphs, the "constraints" imposed by the underlying network topology are no constraint at all. That is, the number of tests required to identify the failed links in "graph-constrained" group testing is near-opti…
In this paper, we consider the group testing problem with adaptive test designs and noisy outcomes. We propose a computationally efficient four-stage procedure with components including random binning, identification of bins containing defective items, 1-sparse recovery via channel codes, and a "clean-up" step to correct any errors from the earlier stages. We prove that the asymptotic required number of tests comes very close to the best known information-theoretic achievability bound (which is based on computationally intractable decoding), and approaches a capacity-based converse bound in th…
Recent advances in noiseless non-adaptive group testing have led to a precise asymptotic characterization of the number of tests required for high-probability recovery in the sublinear regime $k = n^θ$ (with $θ\in (0,1)$), with $n$ individuals among which $k$ are infected. However, the required number of tests may increase substantially under real-world practical constraints, notably including bounds on the maximum number $Δ$ of tests an individual can be placed in, or the maximum number $Γ$ of individuals in a given test. While previous works have given recovery guarantees for these settings,…
We present novel information-theoretic limits on detecting sparse changes in Ising models, a problem that arises in many applications where network changes can occur due to some external stimuli. We show that the sample complexity for detecting sparse changes, in a minimax sense, is no better than learning the entire model even in settings with local sparsity. This is a surprising fact in light of prior work rooted in sparse recovery methods, which suggest that sample complexity in this context scales only with the number of network changes. To shed light on when change detection is easier tha…
Dept. of ECE and CSL, UIUC（伊利诺伊大学厄巴纳-香槟分校电气与计算机工程系及协调科学实验室）
This paper concerns the construction of tests for universal hypothesis testing problems, in which the alternate hypothesis is poorly modeled and the observation space is large. The mismatched universal test is a feature-based technique for this purpose. In prior work it is shown that its finite-observation performance can be much better than the (optimal) Hoeffding test, and good performance depends crucially on the choice of features. The contributions of this paper include: 1) We obtain bounds on the number of εdistinguishable distributions in an exponential family. 2) This motivates a new f…
Instituto de Telecomunicações（电信研究所）；Instituto Superior Técnico, Technical University of Lisbon（里斯本技术大学高等技术学院）
We propose a new method, {\it binary fused compressive sensing} (BFCS), to recover sparse piece-wise smooth signals from 1-bit compressive measurements. The proposed algorithm is a modification of the previous {\it binary iterative hard thresholding} (BIHT) algorithm, where, in addition to the sparsity constraint, the total-variation of the recovered signal is upper constrained. As in BIHT, the data term of the objective function is an one-sided $\ell_1$ (or $\ell_2$) norm. Experiments on the recovery of sparse piece-wise smooth signals show that the proposed algorithm is able to take advantag…
Non-adaptive group testing involves grouping arbitrary subsets of $n$ items into different pools. Each pool is then tested and defective items are identified. A fundamental question involves minimizing the number of pools required to identify at most $d$ defective items. Motivated by applications in network tomography, sensor networks and infection propagation, a variation of group testing problems on graphs is formulated. Unlike conventional group testing problems, each group here must conform to the constraints imposed by a graph. For instance, items can be associated with vertices and each …
Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting
研究了在高斯测量集成下，恢复一个固定但未知的向量$β^* \in \real^p based on a set of $ n $ noisy observations arises in a variety of settings, including subset selection in regression, graphical model selection, signal denoising, compressive sensing, and constructive approximation. Of interest are conditions on the model dimension $ p $, the sparsity index $ s $ (number of non-zero entries in $ β^* $), and the number of observations $ n $ that are necessary and/or sufficient to ensure asymptotically perfect recovery of the sparsity pattern. This paper focuses on the information-theoretic limits of sparsity recovery: in…
