In this paper, we investigate the convergence speed of the Arimoto-Blahut algorithm. For many channel matrices the convergence is exponential, but for some channel matrices it is slower than exponential. By analyzing the Taylor expansion of the defining function of the Arimoto-Blahut algorithm, we will make the conditions clear for the exponential or slower convergence. The analysis of the slow convergence is new in this paper. Based on the analysis, we will compare the convergence speed of the Arimoto-Blahut algorithm numerically with the values obtained in our theorems for several channel matrices. The purpose of this paper is a complete understanding of the convergence speed of the Arimoto-Blahut algorithm.
The paper first recalls the Blahut Arimoto algorithm for computing the capacity of arbitrary discrete memoryless channels, as an example of an iterative algorithm working with probability density estimates. Then, a geometrical interpretation of this algorithm based on projections onto linear and exponential families of probabilities is provided. Finally, this understanding allows also to propose to write the Blahut-Arimoto algorithm, as a true proximal point algorithm. it is shown that the corresponding version has an improved convergence rate, compared to the initial algorithm, as well as in …
Improved Capacity Upper Bounds for the Deletion Channel using a Parallelized Blahut-Arimoto Algorithm
Instituto Superior Técnico, Universidade de Lisboa（里斯本技术学院，里斯本大学）；Instituto de Telecomunicações and Departamento de Matemática, Instituto Superior Técnico, Universidade de Lisboa（电信研究所与数学系，里斯本技术学院，里斯本大学）
We extend the Blahut-Arimoto algorithm for maximizing Massey's directed information. The algorithm can be used for estimating the capacity of channels with delayed feedback, where the feedback is a deterministic function of the output. In order to do so, we apply the ideas from the regular Blahut-Arimoto algorithm, i.e., the alternating maximization procedure, onto our new problem. We provide both upper and lower bound sequences that converge to the optimum value. Our main insight in this paper is that in order to find the maximum of the directed information over causal conditioning probabilit…
The Arimoto--Blahut algorithm for computing the capacity of a discrete memoryless channel is revisited. A so-called ``squeezing'' strategy is used to design algorithms that preserve its simplicity and monotonic convergence properties, but have provably better rates of convergence.
In this paper, we present iterative algorithms that numerically compute the rate-distortion regions of two problems: the two-encoder multiterminal source coding problem and the Chief Executive Officer (CEO) problem, both under logarithmic loss distortion measure.With the clear connection of these models with the distributed information bottleneck method, the proposed algorithms may find usefulness in a variety of applications, such as clustering, pattern recognition and learning. We illustrate the efficiency of our algorithms through some numerical examples.
In this report, we study communication over an additive white Gaussian noise channel with a fixed signal constellation. We measure how much information we can send through this channel and how to improve the rate of communication by changing the input probability distribution. More precisely, we study the mutual information obtained from the Maxwell-Boltzmann distribution, the Blahut-Arimoto algorithm and a constrained version of the Blahut-Arimoto algorithm. We emphasise the fact that the Maxwell--Boltzmann distribution is not optimal. We also observe that the Blahut-Arimoto algorithm does no…
The source coding problem with action-dependent side information at the decoder has recently been introduced to model data acquisition in resource-constrained systems. In this paper, an efficient algorithm for numerical computation of the rate-distortion-cost function for this problem is proposed, and a convergence proof is provided. Moreover, a two-stage code design based on multiplexing is put forth, whereby the first stage encodes the actions and the second stage is composed of an array of classical Wyner-Ziv codes, one for each action. Specific coding/decoding strategies are designed based…
In [8] (Nakagawa, et.al., IEEE Trans. IT, 2021), we investigated the convergence speed of the Arimoto-Blahut algorithm. In [8], the convergence of the order $O(1/N)$ was analyzed by focusing on the second-order nonlinear recurrence formula consisting of the first- and second-order terms of the Taylor expansion of the defining function of the Arimoto-Blahut algorithm. However, in [8], an infinite number of inequalities were assumed as a "conjecture," and proofs were given based on the conjecture. In this paper, we report a proof of the convergence of the order $O(1/N)$ for a class of channel ma…
Based on Arimoto's work in 1978, we propose an iterative algorithm for computing the capacity of a discrete memoryless classical-quantum channel with a finite input alphabet and a finite dimensional output, which we call the Blahut-Arimoto algorithm for classical-quantum channel, and an input cost constraint is considered. We show that to reach $\varepsilon$ accuracy, the iteration complexity of the algorithm is up bounded by $\frac{\log n\log\varepsilon}{\varepsilon}$ where $n$ is the size of the input alphabet. In particular, when the output state $\{ρ_x\}_{x\in \mathcal{X}}$ is linearly ind…
This paper recalls the proximal point method. We study two iterative algorithms: the Blahut-Arimoto algorithm for computing the capacity of arbitrary discrete memoryless channels, as an example of an iterative algorithm working with probability density estimates and the iterative decoding of the Bit Interleaved Coded Modulation (BICM-ID). For these iterative algorithms, we apply the proximal point method which allows new interpretations with improved convergence rate.
This paper studies a scalar Gaussian wiretap channel where instead of an average input power constraint, we consider a peak amplitude constraint on the input. The goal is to obtain insights into the secrecy-capacity and the structure of the secrecy-capacity-achieving distribution. Capitalizing on the recent theoretical progress on the structure of the secrecy-capacity-achieving distribution, this paper develops a numerical procedure, based on the gradient ascent algorithm and a version of the Blahut-Arimoto algorithm, for computing the secrecy-capacity and the secrecy-capacity-achieving input …
We study the beamforming optimization for an intelligent reflecting surface (IRS)-aided full-duplex (FD) communication system in this letter. Specifically, we maximize the sum rate of bi-directional transmissions by jointly optimizing the transmit beamforming and the beamforming of the IRS reflection. A fast converging alternating algorithm is developed to tackle this problem. In each iteration of the proposed algorithm, the solutions to the transmit beamforming and the IRS reflect beamforming are obtained in a semi-closed form and a closed form, respectively. Compared to an existing method ba…
This paper considers a discrete time-Poisson noise channel which is used to model pulse-amplitude modulated optical communication with a direct-detection receiver. The goal of this paper is to obtain insights into the capacity and the structure of the capacity-achieving distribution for the channel under the amplitude constraint $\mathsf{A}$ and in the presence of dark current $λ$. Using recent theoretical progress on the structure of the capacity-achieving distribution, this paper develops a numerical algorithm, based on the gradient ascent and Blahut-Arimoto algorithms, for computing the cap…
