In phase retrieval problems, a signal of interest (SOI) is reconstructed based on the magnitude of a linear transformation of the SOI observed with additive noise. The linear transform is typically referred to as a measurement matrix. Many works on phase retrieval assume that the measurement matrix is a random Gaussian matrix, which, in the noiseless scenario with sufficiently many measurements, guarantees invertability of the transformation between the SOI and the observations, up to an inherent phase ambiguity. However, in many practical applications, the measurement matrix corresponds to an underlying physical setup, and is therefore deterministic, possibly with structural constraints. In this work we study the design of deterministic measurement matrices, based on maximizing the mutual information between the SOI and the observations. We characterize necessary conditions for the optimality of a measurement matrix, and analytically obtain the optimal matrix in the low signal-to-noise ratio regime. Practical methods for designing general measurement matrices and masked Fourier measurements are proposed. Simulation tests demonstrate the performance gain achieved by the proposed techniques compared to random Gaussian measurements for various phase recovery algorithms.
For discrete random variables X_1,..., X_n we construct an n by n matrix. In the (i,j) entry we put the mutual information I(X_i;X_j) between X_i and X_j. In particular, in the (i,i) entry we put the entropy H(X_i)=I(X_i;X_i) of X_i. This matrix, called the mutual information matrix of (X_1,...,X_n), has been conjectured to be positive semi-definite. In this note, we give counterexamples to the conjecture, and show that the conjecture holds for up to three random variables.
Private Information Retrieval (PIR) schemes allow a user to retrieve a record from the server without revealing any information on which record is being downloaded. In this paper, we consider PIR schemes where the database is stored using Minimum Storage Regenerating (MSR) codes which is a class of optimal regenerating codes providing efficient repair when a node failure occurs in the system. We analyse the relationship between the costs of privacy, storage and repair, and also construct an explicit PIR scheme that uses the MSR codes from [3] to achieve the optimal curve of the trade-off.
Coded caching is a promising technique to smooth out network traffic by storing part of the library content at the users' local caches. The seminal work on coded caching for single file retrieval by Maddah-Ali and Niesen (MAN) showed the existence of a global caching gain that scales with the total memory in the system, in addition to the known local caching gain in uncoded systems. This paper formulates a novel cache-aided matrix multiplication retrieval problem, relevant for data analytics and machine learning applications. In the considered problem, each cache-aided user requests the produc…
In this paper, we develop a framework of generalized phase retrieval in which one aims to reconstruct a vector ${\mathbf x}$ in ${\mathbb R}^d$ or ${\mathbb C}^d$ through quadratic samples ${\mathbf x}^*A_1{\mathbf x}, \dots, {\mathbf x}^*A_N{\mathbf x}$. The generalized phase retrieval includes as special cases the standard phase retrieval as well as the phase retrieval by orthogonal projections. We first explore the connections among generalized phase retrieval, low-rank matrix recovery and nonsingular bilinear form. Motivated by the connections, we present results on the minimal measurement…
Multi-message private information retrieval (MPIR) is an interesting variation of PIR which allows a user to download multiple messages from the database without revealing the identity of the desired messages. Obviously, the user can repeatly use a single-message PIR scheme, but we wish for a more efficient way to reduce the download cost. In [1], Banawan and Ukulus investigate the multi-message PIR problem with replicated database. In this paper, we consider multi-message PIR schemes where the database is stored using minimum storage regenerating (MSR) or minimum bandwidth regenerating (MBR) …
In this contribution, an algorithm for evaluating the capacity-achieving input covariance matrices for frequency selective Rayleigh MIMO channels is proposed. In contrast with the flat fading Rayleigh case, no closed-form expressions for the eigenvectors of the optimum input covariance matrix are available. Classically, both the eigenvectors and eigenvalues are computed numerically and the corresponding optimization algorithms remain computationally very demanding. In this paper, it is proposed to optimize (w.r.t. the input covariance matrix) a large system approximation of the average mutual …
Shannon's mutual information of a random multiple antenna and multipath channel is studied in the general case where the channel impulse response is an ergodic and stationary process which is assumed to be available at the receiver. From this viewpoint, the channel is represented by an ergodic self-adjoint block-Jacobi operator, which is close in many aspects to a block version of a random Schrödinger operator. The mutual information is then related to the so-called density of states of this operator. In this paper, it is shown that under the weakest assumptions on the channel, the mutual info…
We study the problem of weakly private information retrieval (W-PIR), where a user wishes to retrieve a desired message from $N$ non-colluding servers in a way that the privacy leakage regarding the desired message's identity is less than or equal to a threshold. We propose a new code construction which significantly improves upon the best known result in the literature, based on the following critical observation. In previous constructions, for the extreme case of minimum download, the retrieval pattern is to download the message directly from $N-1$ servers; however this causes leakage to all…
In this paper, {the goal is to design deterministic sampling patterns on the sphere and the rotation group} and, thereby, construct sensing matrices for sparse recovery of band-limited functions. It is first shown that random sensing matrices, which consists of random samples of Wigner D-functions, satisfy the Restricted Isometry Property (RIP) with proper preconditioning and can be used for sparse recovery on the rotation group. The mutual coherence, however, is used to assess the performance of deterministic and regular sensing matrices. We show that many of widely used regular sampling patt…
We consider the estimation of a n-dimensional vector x from the knowledge of noisy and possibility non-linear element-wise measurements of xxT , a very generic problem that contains, e.g. stochastic 2-block model, submatrix localization or the spike perturbation of random matrices. We use an interpolation method proposed by Guerra and later refined by Korada and Macris. We prove that the Bethe mutual information (related to the Bethe free energy and conjectured to be exact by Lesieur et al. on the basis of the non-rigorous cavity method) always yields an upper bound to the exact mutual informa…
A new computational private information retrieval (PIR) scheme based on random linear codes is presented. A matrix of messages from a McEliece scheme is used to query the server with carefully chosen errors. The server responds with the sum of the scalar multiple of the rows of the query matrix and the files. The user recovers the desired file by erasure decoding the response. Contrary to code-based cryptographic systems, the scheme presented here enables to use truly random codes, not only codes disguised as such. Further, we show the relation to the so-called error subspace search problem an…
We study the low-rank phase retrieval problem, where we try to recover a $d_1\times d_2$ low-rank matrix from a series of phaseless linear measurements. This is a fourth-order inverse problem, as we are trying to recover factors of matrix that have been put through a quadratic nonlinearity after being multiplied together. We propose a solution to this problem using the recently introduced technique of anchored regression. This approach uses two different types of convex relaxations: we replace the quadratic equality constraints for the phaseless measurements by a search over a polytope, and en…
A private information retrieval (PIR) protocol guarantees that a user can privately retrieve files stored in a database without revealing any information about the identity of the requested file. Existing information-theoretic PIR protocols ensure perfect privacy, i.e., zero information leakage to the servers storing the database, but at the cost of high download. In this work, we present weakly-private information retrieval (WPIR) schemes that trade off perfect privacy to improve the download cost when the database is stored on a single server. We study the tradeoff between the download cost …
The purpose of this paper is to introduce a communication model for Large Intelligent Surfaces (LIS). A LIS is modelled as a collection of tiny closely spaced antenna elements. Due to the proximity of the elements, mutual coupling arises. An optimal transmitter design depends on the mutual coupling matrix. For single user communication, the optimal transmitter uses the inverse of the mutual coupling matrix in a filter matched to the channel vector. We give the expression of the mutual coupling for two types of planar arrays. The conditioning number of the mutual coupling matrix is unbounded as…
The capacity-achieving input covariance matrices for coherent block-fading correlated MIMO Rician channels are determined. In this case, no closed-form expressions for the eigenvectors of the optimum input covariance matrix are available. An approximation of the average mutual information is evaluated in this paper in the asymptotic regime where the number of transmit and receive antennas converge to $+\infty$. New results related to the accuracy of the corresponding large system approximation are provided. An attractive optimization algorithm of this approximation is proposed and we establish…
Previous criteria for the feasibility of reconstructing phase information from intensity measurements, both in x-ray crystallography and more recently in coherent x-ray imaging, have been based on the Maxwell constraint counting principle. We propose a new criterion, based on Shannon's mutual information, that is better suited for noisy data or contrast that has strong priors not well modeled by continuous variables. A natural application is magnetic domain imaging, where the criterion for uniqueness in the reconstruction takes the form that the number of photons, per pixel of contrast in the …
Exploiting Mutual Coupling Structure for Channel Estimation of Active RIS-Assisted Links
Technische Universität Berlin（柏林工业大学）；University of British Columbia, Kelowna（不列颠哥伦比亚大学（基洛纳校区））；American University of Beirut (AUB)（贝鲁特美国大学）；King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）
In the phase retrieval problem one seeks to recover an unknown $n$ dimensional signal vector $\mathbf{x}$ from $m$ measurements of the form $y_i = |(\mathbf{A} \mathbf{x})_i|$, where $\mathbf{A}$ denotes the sensing matrix. Many algorithms for this problem are based on approximate message passing. For these algorithms, it is known that if the sensing matrix $\mathbf{A}$ is generated by sub-sampling $n$ columns of a uniformly random (i.e., Haar distributed) orthogonal matrix, in the high dimensional asymptotic regime ($m,n \rightarrow \infty, n/m \rightarrow κ$), the dynamics of the algorithm a…
Stanford University（斯坦福大学）；Technion, Israel Institute of Technology（以色列理工学院）；University of California at Davis（加州大学戴维斯分校）；University of California at Berkeley（加州大学伯克利分校）
We rigorously derive a single-letter variational expression for the mutual information of the asymmetric two-groups stochastic block model in the dense graph regime. Existing proofs in the literature are indirect, as they involve mapping the model to a rank-one matrix estimation problem whose mutual information is then determined by a combination of methods (e.g., interpolation, cavity, algorithmic, spatial coupling). In this contribution we provide a self-contained direct method using only the recently introduced adaptive interpolation method.
There has been definite progress recently in proving the variational single-letter formula given by the heuristic replica method for various estimation problems. In particular, the replica formula for the mutual information in the case of noisy linear estimation with random i.i.d. matrices, a problem with applications ranging from compressed sensing to statistics, has been proven rigorously. In this contribution we go beyond the restrictive i.i.d. matrix assumption and discuss the formula proposed by Takeda, Uda, Kabashima and later by Tulino, Verdu, Caire and Shamai who used the replica metho…
We study the problem of community detection when there is covariate information about the node labels and one observes multiple correlated networks. We provide an asymptotic upper bound on the per-node mutual information as well as a heuristic analysis of a multivariate performance measure called the MMSE matrix. These results show that the combined effects of seemingly very different types of information can be characterized explicitly in terms of formulas involving low-dimensional estimation problems in additive Gaussian noise. Our analysis is supported by numerical simulations.
Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；CNRS（法国国家科学研究中心）；PSL Universités et Ecole Normale Supérieure（巴黎文理研究大学和巴黎高等师范学院）；Sorbonne Universités et Université Pierre & Marie Curie（索邦大学和皮埃尔和玛丽居里大学）；CEA（法国原子能和替代能源委员会）；Université Paris-Saclay（巴黎萨克雷大学）
Factorizing low-rank matrices has many applications in machine learning and statistics. For probabilistic models in the Bayes optimal setting, a general expression for the mutual information has been proposed using heuristic statistical physics computations, and proven in few specific cases. Here, we show how to rigorously prove the conjectured formula for the symmetric rank-one case. This allows to express the minimal mean-square-error and to characterize the detectability phase transitions in a large set of estimation problems ranging from community detection to sparse PCA. We also show that…
We consider the processing of statistical samples $X\sim P_θ$ by a channel $p(y|x)$, and characterize how the statistical information from the samples for estimating the parameter $θ\in\mathbb{R}^d$ can scale with the mutual information or capacity of the channel. We show that if the statistical model has a sub-Gaussian score function, then the trace of the Fisher information matrix for estimating $θ$ from $Y$ can scale at most linearly with the mutual information between $X$ and $Y$. We apply this result to obtain minimax lower bounds in distributed statistical estimation problems, and obtain…
We consider rank-one non-symmetric tensor estimation and derive simple formulas for the mutual information. We start by the order 2 problem, namely matrix factorization. We treat it completely in a simpler fashion than previous proofs using a new type of interpolation method developed in [1]. We then show how to harness the structure in "layers" of tensor estimation in order to obtain a formula for the mutual information for the order 3 problem from the knowledge of the formula for the order 2 problem, still using the same kind of interpolation. Our proof technique straightforwardly generalize…
International Center for Theoretical Physics (ICTP)（国际理论物理中心）；Ecole Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
We consider increasingly complex models of matrix denoising and dictionary learning in the Bayes-optimal setting, in the challenging regime where the matrices to infer have a rank growing linearly with the system size. This is in contrast with most existing literature concerned with the low-rank (i.e., constant-rank) regime. We first consider a class of rotationally invariant matrix denoising problems whose mutual information and minimum mean-square error are computable using techniques from random matrix theory. Next, we analyze the more challenging models of dictionary learning. To do so we …
Recent years have seen a flurry of activities in designing provably efficient nonconvex procedures for solving statistical estimation problems. Due to the highly nonconvex nature of the empirical loss, state-of-the-art procedures often require proper regularization (e.g. trimming, regularized cost, projection) in order to guarantee fast convergence. For vanilla procedures such as gradient descent, however, prior theory either recommends highly conservative learning rates to avoid overshooting, or completely lacks performance guarantees. This paper uncovers a striking phenomenon in nonconvex op…
