This paper presents a unified framework to understand the dynamics of message-passing algorithms in compressed sensing. State evolution is rigorously analyzed for a general error model that contains the error model of approximate message-passing (AMP), as well as that of orthogonal AMP. As a by-product, AMP is proved to converge asymptotically if the sensing matrix is orthogonally invariant and if the moment sequence of its asymptotic singular-value distribution coincide with that of the Marchenko-Pastur distribution up to the order that is at most twice as large as the maximum number of iterations.
It is shown how expectation maximization (EM) may be viewed as a message passing algorithm in factor graphs. In particular, a general EM message computation rule is identified. As a factor graph tool, EM may be used to break cycles in a factor graph, and tractable messages may in some cases be obtained where the sum-product messages are unwieldy. As an exemplary application, the paper considers linear Gaussian state space models. Unknown coefficients in such models give rise to multipliers in the corresponding factor graph. A main attraction of EM in such cases is that it results in purely Gau…
In this paper, we show how to construct a factor graph from a network code. This provides a systematic framework for decoding using message passing algorithms. The proposed message passing decoder exploits knowledge of the underlying communications network topology to simplify decoding. For uniquely decodeable linear network codes on networks with error-free links, only the message supports (rather than the message values themselves) are required to be passed. This proposed simplified support message algorithm is an instance of the sum-product algorithm. Our message-passing framework provides …
This paper addresses the reconstruction of sparse signals from spatially coupled, linear, and noisy measurements. A unified framework of rigorous state evolution is established for developing long-memory message-passing (LM-MP) in spatially coupled systems. LM-MP utilizes all previous messages to compute the current message while conventional MP only uses the latest messages. The unified framework is utilized to propose orthogonal approximate message-passing (OAMP) for spatially coupled systems. The framework for LM-MP is used as a technical tool to prove the convergence of state evolution for…
We introduce an iterative solution to the problem of interference alignment (IA) over MIMO channels based on a message-passing formulation. We propose a parameterization of the messages that enables the computation of IA precoders by a min-sum algorithm over continuous variable spaces -- under this parameterization, suitable approximations of the messages can be computed in closed-form. We show that the iterative leakage minimization algorithm of Cadambe et al. is a special case of our message-passing algorithm, obtained for a particular schedule. Finally, we show that the proposed algorithm c…
Variational message passing (VMP), belief propagation (BP) and expectation propagation (EP) have found their wide applications in complex statistical signal processing problems. In addition to viewing them as a class of algorithms operating on graphical models, this paper unifies them under an optimization framework, namely, Bethe free energy minimization with differently and appropriately imposed constraints. This new perspective in terms of constraint manipulation can offer additional insights on the connection between different message passing algorithms and is valid for a generic statistic…
This paper proves the convergence of Bayes-optimal orthogonal/vector approximate message-passing (AMP) to a fixed point in the large system limit. The proof is based on Bayes-optimal long-memory (LM) message-passing (MP) that is guaranteed to converge systematically. The dynamics of Bayes-optimal LM-MP is analyzed via an existing state evolution framework. The obtained state evolution recursions are proved to converge. The convergence of Bayes-optimal orthogonal/vector AMP is proved by confirming an exact reduction of the state evolution recursions to those for Bayes-optimal orthogonal/vector …
A Local Tree Structure is NOT Sufficient for the Local Optimality of Message-Passing Decoding in Low Density Parity Check Codes
我们解决的问题，“是一个局部树结构足够的局部最优性的消息传递算法在低密度奇偶检验码？结果表明，答案是否定的。利用这一观察，我们指出了Thomas J. Richardson和Rüdiger L.Urbanke \cite{RUCapacity}的论文“消息传递解码下低密度奇偶校验码的容量”中定理1证明的一个缺陷。我们进一步提供了一个新的证明该定理的基础上不同的论点。
A ternary message passing (TMP) decoding algorithm for low-density parity-check codes is developed. All messages exchanged between variable and check nodes have a ternary alphabet, and the variable nodes exploit soft information from the channel. A density evolution analysis is developed for unstructured and protograph-based ensembles. For unstructured ensembles the stability condition is derived. Optimized ensembles for TMP decoding show asymptotic gains of up to 0.6 dB with respect to ensembles optimized for binary message passing decoding. Finite length simulations of codes from TMP-optimiz…
In this paper, we propose a finite alphabet message passing algorithm for LDPC codes that replaces the standard min-sum variable node update rule by a mapping based on generic look-up tables. This mapping is designed in a way that maximizes the mutual information between the decoder messages and the codeword bits. We show that our decoder can deliver the same error rate performance as the conventional decoder with a much smaller message bit-width. Finally, we use the proposed algorithm to design a fully unrolled LDPC decoder hardware architecture.
This letter proposes a novel message-passing algorithm for signal recovery in compressed sensing. The proposed algorithm solves the disadvantages of approximate message-passing (AMP) and orthogonal/vector AMP, and realizes their advantages. AMP converges only in a limited class of sensing matrices while it has low complexity. Orthogonal/vector AMP requires a high-complexity matrix inversion while it is applicable for a wide class of sensing matrices. The key feature of the proposed algorithm is the so-called Onsager correction via a convolution of messages in all preceding iterations while the…
We present a novel decoding algorithm for q-ary low-density parity-check codes, termed symbol message passing. The proposed algorithm can be seen as a generalization of Gallager B and the binary message passing algorithm by Lechner et al. to q-ary codes. We derive density evolution equations for the q-ary symmetric channel, compute thresholds for a number of regular low-density parity-check code ensembles, and verify those by Monte Carlo simulations of long channel codes. The proposed algorithm shows performance advantages with respect to an algorithm of comparable complexity from the literatu…
The denoising-based approximate message passing (D-AMP) methodology, recently proposed by Metzler, Maleki, and Baraniuk, allows one to plug in sophisticated denoisers like BM3D into the AMP algorithm to achieve state-of-the-art compressive image recovery. But AMP diverges with small deviations from the i.i.d.-Gaussian assumption on the measurement matrix. Recently, the vector AMP (VAMP) algorithm has been proposed to fix this problem. In this work, we show that the benefits of VAMP extend to D-VAMP.
This work considers millimeter-wave channel estimation in a setting where parameters of the underlying mmWave channels are varying dynamically over time and there is a single drifting path. In this setting, channel estimates at time block $k$ can be used as side information (SI) when estimating the channel at block $k+1$. To estimate channel parameters, we employ an SI-aided (complex) approximate message passing algorithm and compare its performance to a benchmark based on orthogonal matching pursuit.
The Consultative Committee for Space Data Systems (CCSDS) 141.11-O-1 Line Product Code (LPC) provides a rare opportunity to compare maximum-likelihood decoding and message passing. The LPC considered in this paper is intended to serve as the inner code in conjunction with a (255,239) Reed Solomon (RS) code whose symbols are bytes of data. This paper represents the 141.11-O-1 LPC as a bipartite graph and uses that graph to formulate both maximum likelihood (ML) and message passing algorithms. ML decoding must, of course, have the best frame error rate (FER) performance. However, a fixed point i…
Recently, several promising approximate message passing (AMP) based algorithms have been developed for bilinear recovery with model $\boldsymbol{Y}=\sum_{k=1}^K b_k \boldsymbol{A}_k \boldsymbol{C} +\boldsymbol{W} $, where $\{b_k\}$ and $\boldsymbol{C}$ are jointly recovered with known $\boldsymbol{A}_k$ from the noisy measurements $\boldsymbol{Y}$. The bilinear recover problem has many applications such as dictionary learning, self-calibration, compressive sensing with matrix uncertainty, etc. In this work, we propose a new bilinear recovery algorithm based on AMP with unitary transformation. …
In this work we aim to solve the compressed sensing problem for the case of a complex unknown vector by utilizing the Bayesian-optimal structured signal approximate message passing (BOSSAMP) algorithm on the jointly sparse real and imaginary parts of the unknown. By introducing a latent activity variable, BOSSAMP separates the tasks of activity detection and value estimation to overcome the problem of detecting different supports in the real and imaginary parts. We complement the recovery algorithm by two novel support detection schemes that utilize the updated auxiliary variables of BOSSAMP. …
We address noisy message-passing decoding of lowdensity parity-check (LDPC) codes over additive white Gaussian noise channels. Message-passing decoders in which certain processing units iteratively exchange messages are common for decoding LDPC codes. The exchanged messages are in general subject to internal noise in hardware implementation of these decoders. We model the internal decoder noise as additive white Gaussian noise (AWGN) degrading exchanged messages. Using Gaussian approximation of the exchanged messages, we perform a two-dimensional density evolution analysis for the noisy LDPC d…
Cloud Radio Access Network (C-RAN) is a promising architecture for unprecedented capacity enhancement in next-generation wireless networks thanks to the centralization and virtualization of base station processing. However, centralized signal processing in C-RANs involves high computational complexity that quickly becomes unaffordable when the network grows to a huge size. Among the first, this paper endeavours to design a scalable uplink signal detection algorithm, in the sense that both the complexity per unit network area and the total computation time remain constant when the network size …
Approximate message-passing (AMP) method is a simple and efficient framework for the linear inverse problems. In this letter, we propose a faster AMP to solve the \emph{$L_1$-Split-Analysis} for the 2D sparsity separation, which is referred to as \emph{MixAMP}. We develop the MixAMP based on the factor graphical modeling and the min-sum message-passing. Then, we examine MixAMP for two types of the sparsity separation: separation of the direct-and-group sparsity, and that of the direct-and-finite-difference sparsity. This case study shows that the MixAMP method offers computational advantages o…
A decoding algorithm for $q$-ary low-density parity-check codes over the $q$-ary symmetric channel is introduced. The exchanged messages are lists of symbols from $\Fq$. A density evolution analysis for maximum list sizes $1$ and $2$ is developed. Thresholds for selected regular low-density parity-check code ensembles are computed showing gains with respect to a similar algorithm in the literature. Finite-length simulation results confirm the asymptotic analysis.
With a unified belief propagation (BP) and mean field (MF) framework, we propose an iterative message passing receiver, which performs joint channel state and noise precision (the reciprocal of noise variance) estimation and decoding for OFDM systems. The recently developed generalized approximate message passing (GAMP) is incorporated to the BP-MF framework, where MF is used to handle observation factor nodes with unknown noise precision and GAMP is used for channel estimation in the time-frequency domain. Compared to state-of-the-art algorithms in the literature, the proposed algorithm eithe…
In this paper, we propose a new combined message passing algorithm which allows belief propagation (BP) and mean filed (MF) applied on a same factor node, so that MF can be applied to hard constraint factors. Based on the proposed message passing algorithm, a iterative receiver is designed for MIMO-OFDM systems. Both BP and MF are exploited to deal with the hard constraint factor nodes involving the multiplication of channel coefficients and data symbols to reduce the complexity of the only BP used. The numerical results show that the BER performance of the proposed low complexity receiver clo…
Low complexity decoding algorithms are necessary to meet data rate requirements in excess of 1 Tbps. In this paper, we study one and two bit message passing algorithms for belief propagation decoding of low-density parity-check (LDPC) codes and analyze them by density evolution. The variable nodes (VNs) exploit soft information from the channel output. To decrease the data flow, the messages exchanged between check nodes (CNs) and VNs are represented by one or two bits. The newly proposed quaternary message passing (QMP) algorithm is compared asymptotically and in finite length simulations to …
The paper proposes a new message passing algorithm for cycle-free factor graphs. The proposed "entropy message passing" (EMP) algorithm may be viewed as sum-product message passing over the entropy semiring, which has previously appeared in automata theory. The primary use of EMP is to compute the entropy of a model. However, EMP can also be used to compute expressions that appear in expectation maximization and in gradient descent algorithms.
In this project, the behavior of Generalized Approximate Message-Passing Decoder for BSC and Z Channel is studied using i.i.d matrices for constructing the codewords. The performance of GAMP in AWGN Channel is already evaluated in the previous scientific work of Jean Barbier, therefore, this project mainly focuses on the performance of GAMP decoder for BSC and Z Channel. We evaluate the performance of the GAMP decoder for sparse superposition codes at various settings and compare the performance of decoder for different channels and parameters.
We consider the optimal quantization of compressive sensing measurements following the work on generalization of relaxed belief propagation (BP) for arbitrary measurement channels. Relaxed BP is an iterative reconstruction scheme inspired by message passing algorithms on bipartite graphs. Its asymptotic error performance can be accurately predicted and tracked through the state evolution formalism. We utilize these results to design mean-square optimal scalar quantizers for relaxed BP signal reconstruction and empirically demonstrate the superior error performance of the resulting quantizers.
The topological interference management (TIM) problem studies partially-connected interference networks with no channel state information except for the network topology (i.e., connectivity graph) at the transmitters. In this paper, we consider a similar problem in the uplink cellular networks, while message passing is enabled at the receivers (e.g., base stations), so that the decoded messages can be routed to other receivers via backhaul links to help further improve network performance. For this TIM problem with decoded message passing (TIM-MP), we model the interference pattern by conflict…
A message-passing algorithm for counting short cycles in a graph is presented. For bipartite graphs, which are of particular interest in coding, the algorithm is capable of counting cycles of length g, g +2,..., 2g - 2, where g is the girth of the graph. For a general (non-bipartite) graph, cycles of length g; g + 1, ..., 2g - 1 can be counted. The algorithm is based on performing integer additions and subtractions in the nodes of the graph and passing extrinsic messages to adjacent nodes. The complexity of the proposed algorithm grows as $O(g|E|^2)$, where $|E|$ is the number of edges in the …
In this paper, we address the problem of recovering complex-valued signals from a set of complex-valued linear measurements. Approximate message passing (AMP) is one state-of-the-art algorithm to recover real-valued sparse signals. However, the extension of AMP to complex-valued case is nontrivial and no detailed and rigorous derivation has been explicitly presented. To fill this gap, we extend AMP to complex Bayesian approximate message passing (CB-AMP) using expectation propagation (EP). This novel perspective leads to a concise derivation of CB-AMP without sophisticated transformations betw…
Inspired by distributed resource allocation problems in dynamic topology networks, we initiate the study of distributed consensus with finite messaging passing. We first find a sufficient condition on the network graph for which no distributed protocol can guarantee a conflict-free allocation after $R$ rounds of message passing. Secondly we fully characterize the conflict minimizing zero-round protocol for path graphs, namely random allocation, which partitions the graph into small conflict groups. Thirdly, we enumerate all one-round protocols for path graphs and show that the best one further…
The ubiquity of approximately sparse data has led a variety of com- munities to great interest in compressed sensing algorithms. Although these are very successful and well understood for linear measurements with additive noise, applying them on real data can be problematic if imperfect sensing devices introduce deviations from this ideal signal ac- quisition process, caused by sensor decalibration or failure. We propose a message passing algorithm called calibration approximate message passing (Cal-AMP) that can treat a variety of such sensor-induced imperfections. In addition to deriving the…
We consider the problem of recovering a block (or group) sparse signal from an underdetermined set of random linear measurements, which appear in compressed sensing applications such as radar and imaging. Recent results of Donoho, Johnstone, and Montanari have shown that approximate message passing (AMP) in combination with Stein's shrinkage outperforms group LASSO for large block sizes. In this paper, we prove that, for a fixed block size and in the strong undersampling regime (i.e., having very few measurements compared to the ambient dimension), AMP cannot improve upon group LASSO, thereby …
This paper proposes a fast approximate message-passing (AMP) algorithm for solving compressed sensing (CS) recovery problems with 1D-finite-difference sparsity in term of MMSE estimation. The proposed algorithm, named ssAMP-BGFD, is low-computational with its fast convergence and cheap per-iteration cost, providing phase transition nearly approaching to the state-of-the-art. The proposed algorithm is originated from a sum-product message-passing rule, applying a Bernoulli-Gaussian (BG) prior, seeking an MMSE solution. The algorithm construction includes not only the conventional AMP technique …
The goal of hyperspectral unmixing is to decompose an electromagnetic spectral dataset measured over M spectral bands and T pixels into N constituent material spectra (or "end-members") with corresponding spatial abundances. In this paper, we propose a novel approach to hyperspectral unmixing based on loopy belief propagation (BP) that enables the exploitation of spectral coherence in the endmembers and spatial coherence in the abundances. In particular, we partition the factor graph into spectral coherence, spatial coherence, and bilinear subgraphs, and pass messages between them using a "tur…
In this work we propose a novel iterative estimation algorithm for linear observation systems called S-AMP whose fixed points are the stationary points of the exact Gibbs free energy under a set of (first- and second-) moment consistency constraints in the large system limit. S-AMP extends the approximate message-passing (AMP) algorithm to general matrix ensembles. The generalization is based on the S-transform (in free probability) of the spectrum of the measurement matrix. Furthermore, we show that the optimality of S-AMP follows directly from its design rather than from solving a separate o…
Localization and synchronization are very important in many wireless applications such as monitoring and vehicle tracking. Utilizing the same time of arrival (TOA) measurements for simultaneous localization and synchronization is challenging. In this paper, we present a factor graph (FG) representation of the joint localization and time synchronization problem based on TOA measurements, in which the non-line-of-sight measurements are also taken into consideration. On this FG, belief propagation (BP) message passing and variational message passing (VMP) are applied to derive two fully distribut…
The max-product algorithm, a local message-passing scheme that attempts to compute the most probable assignment (MAP) of a given probability distribution, has been successfully employed as a method of approximate inference for applications arising in coding theory, computer vision, and machine learning. However, the max-product algorithm is not guaranteed to converge to the MAP assignment, and if it does, is not guaranteed to recover the MAP assignment. Alternative convergent message-passing schemes have been proposed to overcome these difficulties. This work provides a systematic study of suc…
In this paper we consider Basis Pursuit De-Noising (BPDN) problems in which the sparse original signal is drawn from a finite alphabet. To solve this problem we propose an iterative message passing algorithm, which capitalises not only on the sparsity but by means of a prior distribution also on the discrete nature of the original signal. In our numerical experiments we test this algorithm in combination with a Rademacher measurement matrix and a measurement matrix derived from the random demodulator, which enables compressive sampling of analogue signals. Our results show in both cases signif…
Superposition codes are efficient for the Additive White Gaussian Noise channel. We provide here a replica analysis of the performances of these codes for large signals. We also consider a Bayesian Approximate Message Passing decoder based on a belief-propagation approach, and discuss its performance using the density evolution technic. Our main findings are 1) for the sizes we can access, the message-passing decoder outperforms other decoders studied in the literature 2) its performance is limited by a sharp phase transition and 3) while these codes reach capacity as $B$ (a crucial parameter …
For the problem of multi-class linear classification and feature selection, we propose approximate message passing approaches to sparse multinomial logistic regression (MLR). First, we propose two algorithms based on the Hybrid Generalized Approximate Message Passing (HyGAMP) framework: one finds the maximum a posteriori (MAP) linear classifier and the other finds an approximation of the test-error-rate minimizing linear classifier. Then we design computationally simplified variants of these two algorithms. Next, we detail methods to tune the hyperparameters of their assumed statistical models…
We study the behavior of Approximate Message-Passing, a solver for linear sparse estimation problems such as compressed sensing, when the i.i.d matrices -for which it has been specifically designed- are replaced by structured operators, such as Fourier and Hadamard ones. We show empirically that after proper randomization, the structure of the operators does not significantly affect the performances of the solver. Furthermore, for some specially designed spatially coupled operators, this allows a computationally fast and memory efficient reconstruction in compressed sensing up to the informati…
In this work, based on the hybrid generalized approximate message passing (HyGAMP) algorithm, we propose the message-scheduling GAMP (MSGAMP) algorithm in order to address the problem of joint active device detection and channel estimation in an uplink grant-free massive MIMO system scenario. In MSGAMP, we apply three different scheduling techniques based on the Residual Belief Propagation (RBP) in which messages are generated using the latest available information. With a much lower computational cost than the state-of-the-art algorithms, MSGAMP-type schemes exhibits good performance in terms…
Zhejiang University（浙江大学）；Ocean College, Zhejiang University（浙江大学海洋学院）；The University of Tokyo（东京大学）；Institute for Physics of Intelligence, The University of Tokyo（东京大学智能物理研究所）；University of Wollongong（伍伦贡大学）
We consider the problem of recovering an unknown signal ${\mathbf x}\in {\mathbb R}^n$ from general nonlinear measurements obtained through a generalized linear model (GLM), i.e., ${\mathbf y}= f\left({\mathbf A}{\mathbf x}+{\mathbf w}\right)$, where $f(\cdot)$ is a componentwise nonlinear function. Based on the unitary transform approximate message passing (UAMP) and expectation propagation, a unitary transform based generalized approximate message passing (GUAMP) algorithm is proposed for general measurement matrices $\bf{A}$, in particular highly correlated matrices. Experimental results on…
In this paper, a communication-efficient multi-processor compressed sensing framework based on the approximate message passing algorithm is proposed. We perform lossy compression on the data being communicated between processors, resulting in a reduction in communication costs with a minor degradation in recovery quality. In the proposed framework, a new state evolution formulation takes the quantization error into account, and analytically determines the coding rate required in each iteration. Two approaches for allocating the coding rate, an online back-tracking heuristic and an optimal allo…
The orthogonal-time-frequency-space (OTFS) modulation has emerged as a promising modulation scheme for high mobility wireless communications. To harvest the time and frequency diversity promised by OTFS, some promising detectors, especially message passing based ones, have been developed by taking advantage of the sparsity of the channel in the delay-Doppler domain. However, when the number of channel paths is relatively large or fractional Doppler {shifts have} to be considered, the complexity of existing detectors is a concern, and the message passing based detectors may suffer from performa…
We consider a class of approximated message passing (AMP) algorithms and characterize their high-dimensional behavior in terms of a suitable state evolution recursion. Our proof applies to Gaussian matrices with independent but not necessarily identically distributed entries. It covers --in particular-- the analysis of generalized AMP, introduced by Rangan, and of AMP reconstruction in compressed sensing with spatially coupled sensing matrices. The proof technique builds on the one of [BM11], while simplifying and generalizing several steps.
We consider dictionary learning and blind calibration for signals and matrices created from a random ensemble. We study the mean-squared error in the limit of large signal dimension using the replica method and unveil the appearance of phase transitions delimiting impossible, possible-but-hard and possible inference regions. We also introduce an approximate message passing algorithm that asymptotically matches the theoretical performance, and show through numerical tests that it performs very well, for the calibration problem, for tractable system sizes.
We consider a broad class of Approximate Message Passing (AMP) algorithms defined as a Lipschitzian functional iteration in terms of an $n\times n$ random symmetric matrix $A$. We establish universality in noise for this AMP in the $n$-limit and validate this behavior in a number of AMPs popularly adapted in compressed sensing, statistical inferences, and optimizations in spin glasses.
Over the last decade or so, Approximate Message Passing (AMP) algorithms have become extremely popular in various structured high-dimensional statistical problems. The fact that the origins of these techniques can be traced back to notions of belief propagation in the statistical physics literature lends a certain mystique to the area for many statisticians. Our goal in this work is to present the main ideas of AMP from a statistical perspective, to illustrate the power and flexibility of the AMP framework. Along the way, we strengthen and unify many of the results in the existing literature.
We propose a general framework for solving the group synchronization problem, where we focus on the setting of adversarial or uniform corruption and sufficiently small noise. Specifically, we apply a novel message passing procedure that uses cycle consistency information in order to estimate the corruption levels of group ratios and consequently solve the synchronization problem in our setting. We first explain why the group cycle consistency information is essential for effectively solving group synchronization problems. We then establish exact recovery and linear convergence guarantees for t…
