Various alignment problems arising in cryo-electron microscopy, community detection, time synchronization, computer vision, and other fields fall into a common framework of synchronization problems over compact groups such as Z/L, U(1), or SO(3). The goal of such problems is to estimate an unknown vector of group elements given noisy relative observations. We present an efficient iterative algorithm to solve a large class of these problems, allowing for any compact group, with measurements on multiple 'frequency channels' (Fourier modes, or more generally, irreducible representations of the group). Our algorithm is a highly efficient iterative method following the blueprint of approximate message passing (AMP), which has recently arisen as a central technique for inference problems such as structured low-rank estimation and compressed sensing. We augment the standard ideas of AMP with ideas from representation theory so that the algorithm can work with distributions over compact groups. Using standard but non-rigorous methods from statistical physics we analyze the behavior of our algorithm on a Gaussian noise model, identifying phases where the problem is easy, (computationally) hard, and (statistically) impossible. In particular, such evidence predicts that our algorithm is information-theoretically optimal in many cases, and that the remaining cases show evidence of statistical-to-computational gaps.
Carnegie Mellon University（卡内基梅隆大学）；University of Alberta（阿尔伯塔大学）
Boolean matrix factorization and Boolean matrix completion from noisy observations are desirable unsupervised data-analysis methods due to their interpretability, but hard to perform due to their NP-hardness. We treat these problems as maximum a posteriori inference problems in a graphical model and present a message passing approach that scales linearly with the number of observations and factors. Our empirical study demonstrates that message passing is able to recover low-rank Boolean matrices, in the boundaries of theoretically possible recovery and compares favorably with state-of-the-art …
University of Minnesota（明尼苏达大学）；School of Mathematics, University of Minnesota（明尼苏达大学数学学院）
We propose an efficient algorithm for solving group synchronization under high levels of corruption and noise, while we focus on rotation synchronization. We first describe our recent theoretically guaranteed message passing algorithm that estimates the corruption levels of the measured group ratios. We then propose a novel reweighted least squares method to estimate the group elements, where the weights are initialized and iteratively updated using the estimated corruption levels. We demonstrate the superior performance of our algorithm over state-of-the-art methods for rotation synchronizati…
We propose a general framework for solving the group synchronization problem, where we focus on the setting of adversarial or uniform corruption and sufficiently small noise. Specifically, we apply a novel message passing procedure that uses cycle consistency information in order to estimate the corruption levels of group ratios and consequently solve the synchronization problem in our setting. We first explain why the group cycle consistency information is essential for effectively solving group synchronization problems. We then establish exact recovery and linear convergence guarantees for t…
This paper addresses the reconstruction of sparse signals from spatially coupled, linear, and noisy measurements. A unified framework of rigorous state evolution is established for developing long-memory message-passing (LM-MP) in spatially coupled systems. LM-MP utilizes all previous messages to compute the current message while conventional MP only uses the latest messages. The unified framework is utilized to propose orthogonal approximate message-passing (OAMP) for spatially coupled systems. The framework for LM-MP is used as a technical tool to prove the convergence of state evolution for…
For the problem of multi-class linear classification and feature selection, we propose approximate message passing approaches to sparse multinomial logistic regression (MLR). First, we propose two algorithms based on the Hybrid Generalized Approximate Message Passing (HyGAMP) framework: one finds the maximum a posteriori (MAP) linear classifier and the other finds an approximation of the test-error-rate minimizing linear classifier. Then we design computationally simplified variants of these two algorithms. Next, we detail methods to tune the hyperparameters of their assumed statistical models…
Variational message passing (VMP), belief propagation (BP) and expectation propagation (EP) have found their wide applications in complex statistical signal processing problems. In addition to viewing them as a class of algorithms operating on graphical models, this paper unifies them under an optimization framework, namely, Bethe free energy minimization with differently and appropriately imposed constraints. This new perspective in terms of constraint manipulation can offer additional insights on the connection between different message passing algorithms and is valid for a generic statistic…
This paper is concerned with the problem of reconstructing an unknown rank-one matrix with prior structural information from noisy observations. While computing the Bayes-optimal estimator seems intractable in general due to its nonconvex nature, Approximate Message Passing (AMP) emerges as an efficient first-order method to approximate the Bayes-optimal estimator. However, the theoretical underpinnings of AMP remain largely unavailable when it starts from random initialization, a scheme of critical practical utility. Focusing on a prototypical model called $\mathbb{Z}_{2}$ synchronization, we…
This paper proves the convergence of Bayes-optimal orthogonal/vector approximate message-passing (AMP) to a fixed point in the large system limit. The proof is based on Bayes-optimal long-memory (LM) message-passing (MP) that is guaranteed to converge systematically. The dynamics of Bayes-optimal LM-MP is analyzed via an existing state evolution framework. The obtained state evolution recursions are proved to converge. The convergence of Bayes-optimal orthogonal/vector AMP is proved by confirming an exact reduction of the state evolution recursions to those for Bayes-optimal orthogonal/vector …
A Local Tree Structure is NOT Sufficient for the Local Optimality of Message-Passing Decoding in Low Density Parity Check Codes
我们解决的问题，“是一个局部树结构足够的局部最优性的消息传递算法在低密度奇偶检验码？结果表明，答案是否定的。利用这一观察，我们指出了Thomas J. Richardson和Rüdiger L.Urbanke \cite{RUCapacity}的论文“消息传递解码下低密度奇偶校验码的容量”中定理1证明的一个缺陷。我们进一步提供了一个新的证明该定理的基础上不同的论点。
A ternary message passing (TMP) decoding algorithm for low-density parity-check codes is developed. All messages exchanged between variable and check nodes have a ternary alphabet, and the variable nodes exploit soft information from the channel. A density evolution analysis is developed for unstructured and protograph-based ensembles. For unstructured ensembles the stability condition is derived. Optimized ensembles for TMP decoding show asymptotic gains of up to 0.6 dB with respect to ensembles optimized for binary message passing decoding. Finite length simulations of codes from TMP-optimiz…
Localization and synchronization are very important in many wireless applications such as monitoring and vehicle tracking. Utilizing the same time of arrival (TOA) measurements for simultaneous localization and synchronization is challenging. In this paper, we present a factor graph (FG) representation of the joint localization and time synchronization problem based on TOA measurements, in which the non-line-of-sight measurements are also taken into consideration. On this FG, belief propagation (BP) message passing and variational message passing (VMP) are applied to derive two fully distribut…
This letter proposes a novel message-passing algorithm for signal recovery in compressed sensing. The proposed algorithm solves the disadvantages of approximate message-passing (AMP) and orthogonal/vector AMP, and realizes their advantages. AMP converges only in a limited class of sensing matrices while it has low complexity. Orthogonal/vector AMP requires a high-complexity matrix inversion while it is applicable for a wide class of sensing matrices. The key feature of the proposed algorithm is the so-called Onsager correction via a convolution of messages in all preceding iterations while the…
This paper presents a unified framework to understand the dynamics of message-passing algorithms in compressed sensing. State evolution is rigorously analyzed for a general error model that contains the error model of approximate message-passing (AMP), as well as that of orthogonal AMP. As a by-product, AMP is proved to converge asymptotically if the sensing matrix is orthogonally invariant and if the moment sequence of its asymptotic singular-value distribution coincide with that of the Marchenko-Pastur distribution up to the order that is at most twice as large as the maximum number of itera…
We consider a broad class of Approximate Message Passing (AMP) algorithms defined as a Lipschitzian functional iteration in terms of an $n\times n$ random symmetric matrix $A$. We establish universality in noise for this AMP in the $n$-limit and validate this behavior in a number of AMPs popularly adapted in compressed sensing, statistical inferences, and optimizations in spin glasses.
We present a novel decoding algorithm for q-ary low-density parity-check codes, termed symbol message passing. The proposed algorithm can be seen as a generalization of Gallager B and the binary message passing algorithm by Lechner et al. to q-ary codes. We derive density evolution equations for the q-ary symmetric channel, compute thresholds for a number of regular low-density parity-check code ensembles, and verify those by Monte Carlo simulations of long channel codes. The proposed algorithm shows performance advantages with respect to an algorithm of comparable complexity from the literatu…
The denoising-based approximate message passing (D-AMP) methodology, recently proposed by Metzler, Maleki, and Baraniuk, allows one to plug in sophisticated denoisers like BM3D into the AMP algorithm to achieve state-of-the-art compressive image recovery. But AMP diverges with small deviations from the i.i.d.-Gaussian assumption on the measurement matrix. Recently, the vector AMP (VAMP) algorithm has been proposed to fix this problem. In this work, we show that the benefits of VAMP extend to D-VAMP.
This work considers millimeter-wave channel estimation in a setting where parameters of the underlying mmWave channels are varying dynamically over time and there is a single drifting path. In this setting, channel estimates at time block $k$ can be used as side information (SI) when estimating the channel at block $k+1$. To estimate channel parameters, we employ an SI-aided (complex) approximate message passing algorithm and compare its performance to a benchmark based on orthogonal matching pursuit.
The Consultative Committee for Space Data Systems (CCSDS) 141.11-O-1 Line Product Code (LPC) provides a rare opportunity to compare maximum-likelihood decoding and message passing. The LPC considered in this paper is intended to serve as the inner code in conjunction with a (255,239) Reed Solomon (RS) code whose symbols are bytes of data. This paper represents the 141.11-O-1 LPC as a bipartite graph and uses that graph to formulate both maximum likelihood (ML) and message passing algorithms. ML decoding must, of course, have the best frame error rate (FER) performance. However, a fixed point i…
Recently, several promising approximate message passing (AMP) based algorithms have been developed for bilinear recovery with model $\boldsymbol{Y}=\sum_{k=1}^K b_k \boldsymbol{A}_k \boldsymbol{C} +\boldsymbol{W} $, where $\{b_k\}$ and $\boldsymbol{C}$ are jointly recovered with known $\boldsymbol{A}_k$ from the noisy measurements $\boldsymbol{Y}$. The bilinear recover problem has many applications such as dictionary learning, self-calibration, compressive sensing with matrix uncertainty, etc. In this work, we propose a new bilinear recovery algorithm based on AMP with unitary transformation. …
Cloud Radio Access Network (C-RAN) is a promising architecture for unprecedented capacity enhancement in next-generation wireless networks thanks to the centralization and virtualization of base station processing. However, centralized signal processing in C-RANs involves high computational complexity that quickly becomes unaffordable when the network grows to a huge size. Among the first, this paper endeavours to design a scalable uplink signal detection algorithm, in the sense that both the complexity per unit network area and the total computation time remain constant when the network size …
A decoding algorithm for $q$-ary low-density parity-check codes over the $q$-ary symmetric channel is introduced. The exchanged messages are lists of symbols from $\Fq$. A density evolution analysis for maximum list sizes $1$ and $2$ is developed. Thresholds for selected regular low-density parity-check code ensembles are computed showing gains with respect to a similar algorithm in the literature. Finite-length simulation results confirm the asymptotic analysis.
With a unified belief propagation (BP) and mean field (MF) framework, we propose an iterative message passing receiver, which performs joint channel state and noise precision (the reciprocal of noise variance) estimation and decoding for OFDM systems. The recently developed generalized approximate message passing (GAMP) is incorporated to the BP-MF framework, where MF is used to handle observation factor nodes with unknown noise precision and GAMP is used for channel estimation in the time-frequency domain. Compared to state-of-the-art algorithms in the literature, the proposed algorithm eithe…
Over the last decade or so, Approximate Message Passing (AMP) algorithms have become extremely popular in various structured high-dimensional statistical problems. The fact that the origins of these techniques can be traced back to notions of belief propagation in the statistical physics literature lends a certain mystique to the area for many statisticians. Our goal in this work is to present the main ideas of AMP from a statistical perspective, to illustrate the power and flexibility of the AMP framework. Along the way, we strengthen and unify many of the results in the existing literature.
In this paper, we propose a new combined message passing algorithm which allows belief propagation (BP) and mean filed (MF) applied on a same factor node, so that MF can be applied to hard constraint factors. Based on the proposed message passing algorithm, a iterative receiver is designed for MIMO-OFDM systems. Both BP and MF are exploited to deal with the hard constraint factor nodes involving the multiplication of channel coefficients and data symbols to reduce the complexity of the only BP used. The numerical results show that the BER performance of the proposed low complexity receiver clo…
Low complexity decoding algorithms are necessary to meet data rate requirements in excess of 1 Tbps. In this paper, we study one and two bit message passing algorithms for belief propagation decoding of low-density parity-check (LDPC) codes and analyze them by density evolution. The variable nodes (VNs) exploit soft information from the channel output. To decrease the data flow, the messages exchanged between check nodes (CNs) and VNs are represented by one or two bits. The newly proposed quaternary message passing (QMP) algorithm is compared asymptotically and in finite length simulations to …
In this project, the behavior of Generalized Approximate Message-Passing Decoder for BSC and Z Channel is studied using i.i.d matrices for constructing the codewords. The performance of GAMP in AWGN Channel is already evaluated in the previous scientific work of Jean Barbier, therefore, this project mainly focuses on the performance of GAMP decoder for BSC and Z Channel. We evaluate the performance of the GAMP decoder for sparse superposition codes at various settings and compare the performance of decoder for different channels and parameters.
The topological interference management (TIM) problem studies partially-connected interference networks with no channel state information except for the network topology (i.e., connectivity graph) at the transmitters. In this paper, we consider a similar problem in the uplink cellular networks, while message passing is enabled at the receivers (e.g., base stations), so that the decoded messages can be routed to other receivers via backhaul links to help further improve network performance. For this TIM problem with decoded message passing (TIM-MP), we model the interference pattern by conflict…
In this paper, we address the problem of recovering complex-valued signals from a set of complex-valued linear measurements. Approximate message passing (AMP) is one state-of-the-art algorithm to recover real-valued sparse signals. However, the extension of AMP to complex-valued case is nontrivial and no detailed and rigorous derivation has been explicitly presented. To fill this gap, we extend AMP to complex Bayesian approximate message passing (CB-AMP) using expectation propagation (EP). This novel perspective leads to a concise derivation of CB-AMP without sophisticated transformations betw…
The ubiquity of approximately sparse data has led a variety of com- munities to great interest in compressed sensing algorithms. Although these are very successful and well understood for linear measurements with additive noise, applying them on real data can be problematic if imperfect sensing devices introduce deviations from this ideal signal ac- quisition process, caused by sensor decalibration or failure. We propose a message passing algorithm called calibration approximate message passing (Cal-AMP) that can treat a variety of such sensor-induced imperfections. In addition to deriving the…
In this work, based on the hybrid generalized approximate message passing (HyGAMP) algorithm, we propose the message-scheduling GAMP (MSGAMP) algorithm in order to address the problem of joint active device detection and channel estimation in an uplink grant-free massive MIMO system scenario. In MSGAMP, we apply three different scheduling techniques based on the Residual Belief Propagation (RBP) in which messages are generated using the latest available information. With a much lower computational cost than the state-of-the-art algorithms, MSGAMP-type schemes exhibits good performance in terms…
Zhejiang University（浙江大学）；Ocean College, Zhejiang University（浙江大学海洋学院）；The University of Tokyo（东京大学）；Institute for Physics of Intelligence, The University of Tokyo（东京大学智能物理研究所）；University of Wollongong（伍伦贡大学）
We consider the problem of recovering an unknown signal ${\mathbf x}\in {\mathbb R}^n$ from general nonlinear measurements obtained through a generalized linear model (GLM), i.e., ${\mathbf y}= f\left({\mathbf A}{\mathbf x}+{\mathbf w}\right)$, where $f(\cdot)$ is a componentwise nonlinear function. Based on the unitary transform approximate message passing (UAMP) and expectation propagation, a unitary transform based generalized approximate message passing (GUAMP) algorithm is proposed for general measurement matrices $\bf{A}$, in particular highly correlated matrices. Experimental results on…
In this paper, a communication-efficient multi-processor compressed sensing framework based on the approximate message passing algorithm is proposed. We perform lossy compression on the data being communicated between processors, resulting in a reduction in communication costs with a minor degradation in recovery quality. In the proposed framework, a new state evolution formulation takes the quantization error into account, and analytically determines the coding rate required in each iteration. Two approaches for allocating the coding rate, an online back-tracking heuristic and an optimal allo…
