We present novel bounds on the capacity of the independent and identically distributed binary deletion channel. Four upper bounds are obtained by providing the transmitter and the receiver with genie-aided information on suitably-defined random processes. Since some of the proposed bounds involve infinite series, we also introduce provable inequalities that lead to more manageable results. For most values of the deletion probability, these bounds improve the existing ones and significantly narrow the gap with the available lower bounds. Exploiting the same auxiliary processes, we also derive, as a by-product, a couple of very simple lower bounds on the channel capacity, which, for low values of the deletion probability, are almost as good as the best existing lower bounds.
We derive an upper bound on the capacity of non-binary deletion channels. Although binary deletion channels have received significant attention over the years, and many upper and lower bounds on their capacity have been derived, such studies for the non-binary case are largely missing. The state of the art is the following: as a trivial upper bound, capacity of an erasure channel with the same input alphabet as the deletion channel can be used, and as a lower bound the results by Diggavi and Grossglauser are available. In this paper, we derive the first non-trivial non-binary deletion channel …
The one-bit deletion and duplication channel is investigated. An input to this channel consists of a block of bits which experiences either a deletion, or a duplication, or remains unchanged. For this channel a capacity expression is obtained in a certain asymptotic regime where the deletion and duplication probabilities tend to zero. As a corollary, we obtain an asymptotic expression for the capacity of the segmented deletion and duplication channel where the input now consists of several blocks and each block independently experiences either a deletion, or a duplication, or remains unchanged…
We consider the class of channels formed from the concatenation of a deletion channel and a finite-state channel. For this class of channels, we show that the operational capacity is equal to the stationary capacity, which can be approached by a sequence of Markov processes with increasing Markovian orders. As a by-product, we show that the polar coding scheme constructed by Tal, Pfister, Fazeli and Vardy achieves the capacity of the deletion channel.
Improved Capacity Upper Bounds for the Deletion Channel using a Parallelized Blahut-Arimoto Algorithm
Instituto Superior Técnico, Universidade de Lisboa（里斯本技术学院，里斯本大学）；Instituto de Telecomunicações and Departamento de Matemática, Instituto Superior Técnico, Universidade de Lisboa（电信研究所与数学系，里斯本技术学院，里斯本大学）
The deletion channel is the simplest point-to-point communication channel that models lack of synchronization. Despite significant effort, little is known about its capacity, and even less about optimal coding schemes. In this paper we intiate a new systematic approach to this problem, by demonstrating that capacity can be computed in a series expansion for small deletion probability. We compute two leading terms of this expansion, and show that capacity is achieved, up to this order, by i.i.d. uniform random distribution of the input. We think that this strategy can be useful in a number of c…
Let $C(d)$ be the capacity of the binary deletion channel with deletion probability $d$. It was proved by Drinea and Mitzenmacher that, for all $d$, $C(d)/(1-d)\geq 0.1185 $. Fertonani and Duman recently showed that $\limsup_{d\to 1}C(d)/(1-d)\leq 0.49$. In this paper, it is proved that $\lim_{d\to 1}C(d)/(1-d)$ exists and is equal to $\inf_{d}C(d)/(1-d)$. This result suggests the conjecture that the curve $C(d)$ my be convex in the interval $d\in [0,1]$. Furthermore, using currently known bounds for $C(d)$, it leads to the upper bound $\lim_{d\to 1}C(d)/(1-d)\leq 0.4143$.
Memoryless channels with deletion errors as defined by a stochastic channel matrix allowing for bit drop outs are considered in which transmitted bits are either independently deleted with probability $d$ or unchanged with probability $1-d$. Such channels are information stable, hence their Shannon capacity exists. However, computation of the channel capacity is formidable, and only some upper and lower bounds on the capacity exist. In this paper, we first show a simple result that the parallel concatenation of two different independent deletion channels with deletion probabilities $d_1$ and $…
The deletion channel is the simplest point-to-point communication channel that models lack of synchronization. Input bits are deleted independently with probability d, and when they are not deleted, they are not affected by the channel. Despite significant effort, little is known about the capacity of this channel, and even less about optimal coding schemes. In this paper we develop a new systematic approach to this problem, by demonstrating that capacity can be computed in a series expansion for small deletion probability. We compute three leading terms of this expansion, and find an input di…
We generalize Helberg's number-theoretic construction of multiple insertion-deletion correcting binary codes to non-binary alphabets and describe a linear decoding algorithm for correcting multiple deletions.
We first give a construction of binary $t_1$-deletion-$t_2$-insertion-burst correcting codes with redundancy at most $\log(n)+(t_1-t_2-1)\log\log(n)+O(1)$, where $t_1\ge 2t_2$. Then we give an improved construction of binary codes capable of correcting a burst of $4$ non-consecutive deletions, whose redundancy is reduced from $7\log(n)+2\log\log(n)+O(1)$ to $4\log(n)+6\log\log(n)+O(1)$. Lastly, by connecting non-binary $b$-burst-deletion correcting codes with binary $2b$-deletion-$b$-insertion-burst correcting codes, we give a new construction of non-binary $b$-burst-deletion correcting codes …
Memoryless channels with synchronization errors as defined by a stochastic channel matrix allowing for symbol insertions and deletions in addition to random errors are considered. Such channels are information stable, hence their Shannon capacity exists. However, computation of the channel capacity is formidable, and only some upper and lower bounds on the capacity (for some special cases) exist. In this short paper, using a simple methodology, we prove that the channel capacity is a convex function of the stochastic channel matrix. Since the more widely studied model of an independent identic…
We develop several analytical lower bounds on the capacity of binary insertion and deletion channels by considering independent uniformly distributed (i.u.d.) inputs and computing lower bounds on the mutual information between the input and output sequences. For the deletion channel, we consider two different models: independent and identically distributed (i.i.d.) deletion-substitution channel and i.i.d. deletion channel with additive white Gaussian noise (AWGN). These two models are considered to incorporate effects of the channel noise along with the synchronization errors. For the insertio…
We consider the zero-error capacity of deletion channels. Specifically, we consider the setting where we choose a codebook ${\cal C}$ consisting of strings of $n$ bits, and our model of the channel corresponds to an adversary who may delete up to $pn$ of these bits for a constant $p$. Our goal is to decode correctly without error regardless of the actions of the adversary. We consider what values of $p$ allow non-zero capacity in this setting. We suggest multiple approaches, one of which makes use of the natural connection between this problem and the problem of finding the expected length of …
In this paper, we construct systematic $q$-ary two-deletion correcting codes and burst-deletion correcting codes, where $q\geq 2$ is an even integer. For two-deletion codes, our construction has redundancy $5\log n+O(\log q\log\log n)$ and has encoding complexity near-linear in $n$, where $n$ is the length of the message sequences. For burst-deletion codes, we first present a construction of binary codes with redundancy $\log n+9\log\log n+γ_t+o(\log\log n)$ bits $(γ_t$ is a constant that depends only on $t)$ and capable of correcting a burst of at most $t$ deletions, which improves the Lenz-P…
This paper constructs a non-binary code correcting a single $b$-burst of insertions or deletions with a large cardinality. This paper also proposes a decoding algorithm of this code and evaluates a lower bound of the cardinality of this code. Moreover, we evaluate an asymptotic upper bound on the cardinality of codes which correct a single burst of insertions or deletions.
We develop a systematic approach, based on convex programming and real analysis, for obtaining upper bounds on the capacity of the binary deletion channel and, more generally, channels with i.i.d. insertions and deletions. Other than the classical deletion channel, we give a special attention to the Poisson-repeat channel introduced by Mitzenmacher and Drinea (IEEE Transactions on Information Theory, 2006). Our framework can be applied to obtain capacity upper bounds for any repetition distribution (the deletion and Poisson-repeat channels corresponding to the special cases of Bernoulli and Po…
The problem of correcting deletions has received significant attention, partly because of the prevalence of these errors in DNA data storage. In this paper, we study the problem of correcting a consecutive burst of at most $t$ deletions in non-binary sequences. We first propose a non-binary code correcting a burst of at most 2 deletions for $q$-ary alphabets. Afterwards, we extend this result to the case where the length of the burst can be at most $t$ where $t$ is a constant. Finally, we consider the setup where the sequences that are transmitted are permutations. The proposed codes are the l…
The construction of deletion codes for the Levenshtein metric is reduced to the construction of codes over the integers for the Manhattan metric by run length coding. The latter codes are constructed by expurgation of translates of lattices. These lattices, in turn, are obtained from Construction~A applied to binary codes and $\Z_4-$codes. A lower bound on the size of our codes for the Manhattan distance are obtained through generalized theta series of the corresponding lattices.
We study the application of polar codes in deletion channels by analyzing the cascade of a binary erasure channel (BEC) and a deletion channel. We show how polar codes can be used effectively on a BEC with a single deletion, and propose a list decoding algorithm with a cyclic redundancy check for this case. The decoding complexity is $O(N^2\log N)$, where $N$ is the blocklength of the code. An important contribution is an optimization of the amount of redundancy added to minimize the overall error probability. Our theoretical results are corroborated by numerical simulations which show that th…
We consider a new formulation of a class of synchronization error channels and derive analytical bounds and numerical estimates for the capacity of these channels. For the binary channel with only deletions, we obtain an expression for the symmetric information rate in terms of subsequence weights which reduces to a tight lower bound for small deletion probabilities. We are also able to exactly characterize the Markov-1 rate for the binary channel with only replications. For a channel that introduces deletions as well as replications of input symbols, we design approximating channels that para…
We consider the problem of reliable communication over non-binary insertion/deletion channels where symbols are randomly deleted from or inserted in the transmitted sequence and all symbols are corrupted by additive white Gaussian noise. To this end, we utilize the inherent redundancy achievable in non-binary symbol sets by first expanding the symbol set and then allocating part of the bits associated with each symbol to watermark symbols. The watermark sequence, known at the receiver, is then used by a forward-backward algorithm to provide soft information for an outer code which decodes the …
Correcting insertions/deletions as well as substitution errors simultaneously plays an important role in DNA-based storage systems as well as in classical communications. This paper deals with the fundamental task of constructing codes that can correct a single insertion or deletion along with a single substitution. A non-asymptotic upper bound on the size of single-deletion single-substitution correcting codes is derived, showing that the redundancy of such a code of length $n$ has to be at least $2 \log n$. The bound is presented both for binary and non-binary codes while an extension to sin…
Suppose a binary string x = x_1...x_n is being broadcast repeatedly over a faulty communication channel. Each time, the channel delivers a fixed number m of the digits (m<n) with the lost digits chosen uniformly at random, and the order of the surviving digits preserved. How large does m have to be to reconstruct the message?
New bounds for covering codes under insertions or deletions
Institute of Mathematics and Interdisciplinary Sciences, Xidian University（西安电子科技大学数学与交叉科学研究院）；School of Mathematical Sciences, Capital Normal University（首都师范大学数学科学学院）
This paper presents the first proof of polarization for the deletion channel with a constant deletion rate and a regular hidden-Markov input distribution. A key part of this work involves representing the deletion channel using a trellis and describing the plus and minus polar-decoding operations on that trellis. In particular, the plus and minus operations can be seen as combining adjacent trellis stages to yield a new trellis with half as many stages. Using this viewpoint, we prove a weak polarization theorem for standard polar codes on the deletion channel. To achieve strong polarization, w…
We study codes that can detect the exact number of deletions and insertions in concatenated binary strings. We construct optimal codes for the case of detecting up to $\del$ deletions. We prove the optimality of these codes by deriving a converse result which shows that the redundancy of our codes is asymptotically optimal in $\del$ among all families of deletion detecting codes, and particularly optimal among all block-by-block decodable codes. For the case of insertions, we construct codes that can detect up to $2$ insertions in each concatenated binary string.
This work gives an explicit construction of a family of error correcting codes for the binary deletion channel and for the Poisson repeat channel. In the binary deletion channel with parameter $p$ (BDC$_p$) every bit is deleted independently with probability $p$. A lower bound of $(1-p)/9$ is known on the capacity of the BDC$_p$ \cite{mitzenmacher2006simple}, yet no explicit construction is known to achieve this rate. We give an explicit family of codes of rate $(1-p)/16$, for every $p$. This improves upon the work of Guruswami and Li \cite{guruswami2017efficiently} that gave a construction of…
Two of the most common models for channels with synchronisation errors are the Binary Deletion Channel with parameter $p$ ($\text{BDC}_p$) -- a channel where every bit of the codeword is deleted i.i.d with probability $p$, and the Poisson Repeat Channel with parameter $λ$ ($\text{PRC}_λ$) -- a channel where every bit of the codeword is repeated $\text{Poisson}(λ)$ times. Previous constructions based on synchronisation strings yielded codes with rates far lower than the capacities of these channels [CS19, GL18], and the only efficient construction to achieve capacity on the BDC at the time of w…
In the random deletion channel, each bit is deleted independently with probability $p$. For the random deletion channel, the existence of codes of rate $(1-p)/9$, and thus bounded away from $0$ for any $p < 1$, has been known. We give an explicit construction with polynomial time encoding and deletion correction algorithms with rate $c_0 (1-p)$ for an absolute constant $c_0 > 0$.
Motivated by the sequence reconstruction problem from traces in DNA-based storage, we consider the problem of designing codes for the deletion channel when multiple observations (or traces) are available to the decoder. We propose simple binary and non-binary codes based on Varshamov-Tenengolts (VT) codes. The proposed codes split the codeword in blocks and employ a VT code in each block. The availability of multiple traces helps the decoder to identify deletion-free copies of a block, and to avoid mis-synchronization while decoding. The encoding complexity of the proposed scheme is linear in …
The noise model of deletions poses significant challenges in coding theory, with basic questions like the capacity of the binary deletion channel still being open. In this paper, we study the harder model of worst-case deletions, with a focus on constructing efficiently decodable codes for the two extreme regimes of high-noise and high-rate. Specifically, we construct polynomial-time decodable codes with the following trade-offs (for any eps > 0): (1) Codes that can correct a fraction 1-eps of deletions with rate poly(eps) over an alphabet of size poly(1/eps); (2) Binary codes of rate 1-O~(sqr…
Motivated by applications in DNA-based storage and communication systems, we study deletion and insertion errors simultaneously in a burst. In particular, we study a type of error named $t$-deletion-$s$-insertion-burst ($(t,s)$-burst for short) which is a generalization of the $(2,1)$-burst error proposed by Schoeny {\it et. al}. Such an error deletes $t$ consecutive symbols and inserts an arbitrary sequence of length $s$ at the same coordinate. We provide a sphere-packing upper bound on the size of binary codes that can correct a $(t,s)$-burst error, showing that the redundancy of such codes …
Deletion-Correcting Codes for the $\ell$-Symbol Read Channel
Institute of Mathematics and Interdisciplinary Sciences, Xidian University（西安电子科技大学数学与交叉科学研究院）；School of Mathematical Sciences, Capital Normal University（首都师范大学数学科学学院）
Improved Lower Bounds on the Capacity of the Binary Deletion Channel via a Learning Approach to Run-Length Inputs
School of Engineering, The University of British Columbia, Okanagan Campus（不列颠哥伦比亚大学工程学院）；Department of Electrical and Electronics Engineering, Bilkent University（比尔肯特大学电气与电子工程系）
In this paper, we consider a synchronization problem between nodes $A$ and $B$ that are connected through a two--way communication channel. {Node $A$} contains a binary file $X$ of length $n$ and {node $B$} contains a binary file $Y$ that is generated by randomly deleting bits from $X$, by a small deletion rate $β$. The location of deleted bits is not known to either node $A$ or node $B$. We offer a deterministic synchronization scheme between nodes $A$ and $B$ that needs a total of $O(nβ\log \frac{1}β)$ transmitted bits and reconstructs $X$ at node $B$ with probability of error that is expone…
This paper presents a coding scheme for an insertion deletion substitution channel. We extend a previous scheme for the deletion channel where polar codes are modified by adding "guard bands" between segments. In the new scheme, each guard band is comprised of a middle segment of '1' symbols, and left and right segments of '0' symbols. Our coding scheme allows for a regular hidden-Markov input distribution, and achieves the information rate between the input and corresponding output of such a distribution. Thus, we prove that our scheme can be used to efficiently achieve the capacity of the ch…
This paper studies on the cardinality of perfect multi deletion binary codes. The lower bound for any perfect deletion code with the fixed code length and the number of deletions, and the asymptotic achievable of Levenshtein's upper bound are shown.
We study codes that are list-decodable under insertions and deletions. Specifically, we consider the setting where a codeword over some finite alphabet of size $q$ may suffer from $δ$ fraction of adversarial deletions and $γ$ fraction of adversarial insertions. A code is said to be $L$-list-decodable if there is an (efficient) algorithm that, given a received word, reports a list of $L$ codewords that include the original codeword. Using the concept of synchronization strings, introduced by the first two authors [STOC 2017], we show some surprising results. We show that for every $0\leqδ<1$, e…
We give a complete answer to the following basic question: "What is the maximal fraction of deletions or insertions tolerable by $q$-ary list-decodable codes with non-vanishing information rate?" This question has been open even for binary codes, including the restriction to the binary insertion-only setting, where the best-known result was that a $γ\leq 0.707$ fraction of insertions is tolerable by some binary code family. For any desired $ε> 0$, we construct a family of binary codes of positive rate which can be efficiently list-decoded from any combination of $γ$ fraction of insertions and …
We consider codes over fixed alphabets against worst-case symbol deletions. For any fixed $k \ge 2$, we construct a family of codes over alphabet of size $k$ with positive rate, which allow efficient recovery from a worst-case deletion fraction approaching $1-\frac{2}{k+\sqrt k}$. In particular, for binary codes, we are able to recover a fraction of deletions approaching $1/(\sqrt 2 +1)=\sqrt 2-1 \approx 0.414$. Previously, even non-constructively the largest deletion fraction known to be correctable with positive rate was $1-Θ(1/\sqrt{k})$, and around $0.17$ for the binary case. Our result pi…
This paper considers a binary channel with deletions and insertions, where each input bit is transformed in one of the following ways: it is deleted with probability d, or an extra bit is added after it with probability i, or it is transmitted unmodified with probability 1-d-i. A computable lower bound on the capacity of this channel is derived. The transformation of the input sequence by the channel may be viewed in terms of runs as follows: some runs of the input sequence get shorter/longer, some runs get deleted, and some new runs are added. It is difficult for the decoder to synchronize th…
List decoding of insertions and deletions in the Levenshtein metric is considered. The Levenshtein distance between two sequences is the minimum number of insertions and deletions needed to turn one of the sequences into the other. In this paper, a Johnson-like upper bound on the maximum list size when list decoding in the Levenshtein metric is derived. This bound depends only on the length and minimum Levenshtein distance of the code, the length of the received word, and the alphabet size. It shows that polynomial-time list decoding beyond half the Levenshtein distance is possible for many pa…
We give an explicit construction of length-$n$ binary codes capable of correcting the deletion of two bits that have size $2^n/n^{4+o(1)}$. This matches up to lower order terms the existential result, based on an inefficient greedy choice of codewords, that guarantees such codes of size $Ω(2^n/n^4)$. Our construction is based on augmenting the classic Varshamov-Tenengolts construction of single deletion codes with additional check equations. We also give an explicit construction of binary codes of size $Ω(2^n/n^{3+o(1)})$ that can be list decoded from two deletions using lists of size two. Pre…
Motivated by a greedy approach for generating {\it{information stable}} processes, we prove a universal maximum likelihood (ML) upper bound on the capacities of discrete information stable channels, including the binary erasure channel (BEC), the binary symmetric channel (BSC) and the binary deletion channel (BDC). The bound is derived leveraging a system of equations obtained via the Karush-Kuhn-Tucker conditions. Intriguingly, for some memoryless channels, e.g., the BEC and BSC, the resulting upper bounds are tight and equal to their capacities. For the BDC, the universal upper bound is rela…
