This paper presents the Gradient Flow (GF) decoding for LDPC codes. GF decoding, a continuous-time methodology based on gradient flow, employs a potential energy function associated with bipolar codewords of LDPC codes. The decoding process of the GF decoding is concisely defined by an ordinary differential equation and thus it is well suited to an analog circuit implementation. We experimentally demonstrate that the decoding performance of the GF decoding for AWGN channels is comparable to that of the multi-bit mode gradient descent bit flipping algorithm. We further introduce the negative log-likelihood function of the channel for generalizing the GF decoding. The proposed method is shown to be tensor-computable, which means that the gradient of the objective function can be evaluated with the combination of basic tensor computations. This characteristic is well-suited to emerging AI accelerators, potentially applicable in wireless signal processing. The paper assesses the decoding performance of the generalized GF decoding in LDPC-coded MIMO channels. Our numerical experiments reveal that the decoding performance rivals that of established techniques like MMSE + BP. Furthermore, an exploration of score-based channel learning for capturing statistical properties is also provided.
The power consumption of the integrated circuit is becoming a significant burden, particularly for large-scale signal processing tasks requiring high throughput. The decoding process of LDPC codes is such a heavy signal processing task that demands power efficiency and higher decoding throughput. A promising approach to reducing both power and latency of a decoding process is to use an analog circuit instead of a digital circuit. This paper investigates a continuous-time gradient flow-based approach for decoding LDPC codes, which employs a potential energy function similar to the objective fun…
In this paper, we propose a Gradient Descent Bit-Flipping (GDBF) decoding with momentum, which considers past updates to provide inertia to the decoding process. We show that GDBF or randomized GDBF decoders with momentum may closely approach the floating-point Belief-Propagation decoding performance, and even outperform it in the error-floor region, especially for graphs with high connectivity degree.
We present a novel optimization-based decoding algorithm for LDPC codes that is suitable for hardware architectures specialized to feed-forward neural networks. The algorithm is based on the projected gradient descent algorithm with a penalty function for solving a non-convex minimization problem. The proposed algorithm has several internal parameters such as step size parameters, a softness parameter, and the penalty coefficients. We use a standard tool set of deep learning, i.e., back propagation and stochastic gradient descent (SGD) type algorithms, to optimize these parameters. Several num…
We propose a novel optimization-based decoding algorithm for LDPC-coded massive MIMO channels. The proposed decoding algorithm is based on a proximal gradient method for solving an approximate maximum a posteriori (MAP) decoding problem. The key idea is the use of a code-constraint polynomial penalizing a vector far from a codeword as a regularizer in the approximate MAP objective function. The code proximal operator is naturally derived from code-constraint polynomials. The proposed algorithm, called proximal decoding, can be described by a simple recursion consisting of the gradient descent …
We propose an improved soft-aided decoding scheme for product codes that approaches the decoding performance of conventional soft-decision TPD with only a 0.2 dB gap while keeping the complexity and internal decoder data flow similarly low as in hard decision decoders.
We propose a binary message passing decoding algorithm for product codes based on generalized minimum distance decoding (GMDD) of the component codes, where the last stage of the GMDD makes a decision based on the Hamming distance metric. The proposed algorithm closes half of the gap between conventional iterative bounded distance decoding (iBDD) and turbo product decoding based on the Chase--Pyndiah algorithm, at the expense of some increase in complexity. Furthermore, the proposed algorithm entails only a limited increase in data flow compared to iBDD.
Imperial College London（帝国理工学院）；University of Maryland（马里兰大学）
Coded computation techniques provide robustness against straggling servers in distributed computing, with the following limitations: First, they increase decoding complexity. Second, they ignore computations carried out by straggling servers; and they are typically designed to recover the full gradient, and thus, cannot provide a balance between the accuracy of the gradient and per-iteration completion time. Here we introduce a hybrid approach, called coded partial gradient computation (CPGC), that benefits from the advantages of both coded and uncoded computation schemes, and reduces both the…
We propose a novel binary message passing decoding algorithm for product-like codes based on bounded distance decoding (BDD) of the component codes. The algorithm, dubbed iterative BDD with scaled reliability (iBDD-SR), exploits the channel reliabilities and is therefore soft in nature. However, the messages exchanged by the component decoders are binary (hard) messages, which significantly reduces the decoder data flow. The exchanged binary messages are obtained by combining the channel reliability with the BDD decoder output reliabilities, properly conveyed by a scaling factor applied to the…
Signal Space Alignment for an Encryption Message and Successive Network Code Decoding on the MIMO K-way Relay Channel
本文研究了多输入多输出（MIMO）高斯无线网络的$K$ -用户和一个中间中继$M$天线的网络信息流问题。在该网络中，每个用户打算向所有其他用户传送多播消息，同时经由中间中继从其他用户接收$K-1$独立消息。该网络信息流被称为MIMO高斯$K$路中继信道。对于这个通道，我们表明，$\frac{K}{2}$自由度是可以实现的，如果$M=K-1$。为了证明这一点，我们提出了一个编码和解码策略的灵感来自密码学理论。所提出的编码和解码策略涉及用于多址接入阶段（MAC）的\textit{signal space alignment for an encryption message}和用于广播（BC）阶段的\textit{zero forcing with successive network code decoding}。\emph{signal space alignment for an encryption message}的思想是所有用户合作地选择预编码向量来发送消息，使得中继器可以接收具有特殊结构\textit{network code chain structure}的适当加密消息。在BC阶段期间，\emph{zero forcing combined with successive network code decoding}使所有用户能够解密来自中继的加密消息，尽管事实上他们…
We derive new gradient flows of divergence functions in the probability space embedded with a class of Riemannian metrics. The Riemannian metric tensor is built from the transported Hessian operator of an entropy function. The new gradient flow is a generalized Fokker-Planck equation and is associated with a stochastic differential equation that depends on the reference measure. Several examples of Hessian transport gradient flows and the associated stochastic differential equations are presented, including the ones for the reverse Kullback--Leibler divergence, alpha-divergence, Hellinger dist…
In this paper, we propose a new coded computing technique called "substitute decoding" for general iterative distributed computation tasks. In the first part of the paper, we use PageRank as a simple example to show that substitute decoding can make the computation of power iterations solving PageRank on sparse matrices robust to erasures in distributed systems. For these sparse matrices, codes with dense generator matrices can significantly increase storage costs and codes with low-density generator matrices (LDGM) are preferred. Surprisingly, we show through both theoretical analysis and sim…
Distributed implementations of gradient-based methods, wherein a server distributes gradient computations across worker machines, need to overcome two limitations: delays caused by slow running machines called 'stragglers', and communication overheads. Recently, Ye and Abbe [ICML 2018] proposed a coding-theoretic paradigm to characterize a fundamental trade-off between computation load per worker, communication overhead per worker, and straggler tolerance. However, their proposed coding schemes suffer from heavy decoding complexity and poor numerical stability. In this paper, we develop a comm…
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；MIT（麻省理工学院）
This paper considers the problem of implementing large-scale gradient descent algorithms in a distributed computing setting in the presence of {\em straggling} processors. To mitigate the effect of the stragglers, it has been previously proposed to encode the data with an erasure-correcting code and decode at the master server at the end of the computation. We, instead, propose to encode the second-moment of the data with a low density parity-check (LDPC) code. The iterative decoding algorithms for LDPC codes have very low computational overhead and the number of decoding iterations can be mad…
