Analog coding decouples the tasks of protecting against erasures and noise. For erasure correction, it creates an "analog redundancy" by means of band-limited discrete Fourier transform (DFT) interpolation, or more generally, by an over-complete expansion based on a frame. We examine the analog coding paradigm for the dual setup of a source with "erasure" side-information (SI) at the encoder. The excess rate of analog coding above the rate-distortion function (RDF) is associated with the energy of the inverse of submatrices of the frame, where each submatrix corresponds to a possible erasure pattern. We give a partial theoretical as well as numerical evidence that a variety of structured frames, in particular DFT frames with difference-set spectrum and more general equiangular tight frames (ETFs), with a common MANOVA limiting spectrum, minimize the excess rate over all possible frames. However, they do not achieve the RDF even in the limit as the dimension goes to infinity.
In this paper, we investigate the optimal tradeoff between source and channel coding for channels with bit or packet erasure. Upper and Lower bounds on the optimal channel coding rate are computed to achieve minimal end-to-end distortion. The bounds are calculated based on a combination of sphere packing, straight line and expurgated error exponents and also high rate vector quantization theory. By modeling a packet erasure channel in terms of an equivalent bit erasure channel, we obtain bounds on the packet size for a specified limit on the distortion.
We consider a real-time streaming system where messages are created sequentially at the source, and are encoded for transmission to the receiver over a packet erasure link. Each message must subsequently be decoded at the receiver within a given delay from its creation time. The goal is to construct an erasure correction code that achieves the maximum message size when all messages must be decoded by their respective deadlines under a specified set of erasure patterns (erasure model). We present an explicit intrasession code construction that is asymptotically optimal under erasure models cont…
Data storage systems are more reliable than their individual components. In order to build highly reliable systems out of less reliable parts, systems introduce redundancy. In replicated systems, objects are simply copied several times with each copy residing on a different physical device. While such an approach is simple and direct, more elaborate approaches such as erasure coding can achieve equivalent levels of data protection while using less redundancy. This report examines the trade-offs in cost and performance between replicated and erasure encoded storage systems.
We assess the practicality of random network coding by illuminating the issue of overhead and considering it in conjunction with increasingly long packets sent over the erasure channel. We show that the transmission of increasingly long packets, consisting of either of an increasing number of symbols per packet or an increasing symbol alphabet size, results in a data rate approaching zero over the erasure channel. This result is due to an erasure probability that increases with packet length. Numerical results for a particular modulation scheme demonstrate a data rate of approximately zero for…
We consider a cache-aided communications system in which a transmitter communicates with many receivers over an erasure broadcast channel. The system serves as a basic model for communicating on-demand content during periods of high network congestion, where some content can be pre-placed in local caches near the receivers. We formulate the cache-aided communications problem as a joint cache-channel coding problem, and characterise some information-theoretic tradeoffs between reliable communications rates and cache sizes. We show that if the receivers experience different channel qualities, th…
We consider the delay of network coding compared to routing with retransmissions in packet erasure networks with probabilistic erasures. We investigate the sub-linear term in the block delay required for unicasting $n$ packets and show that there is an unbounded gap between network coding and routing. In particular, we show that delay benefit of network coding scales at least as $\sqrt{n}$. Our analysis of the delay function for the routing strategy involves a major technical challenge of computing the expectation of the maximum of two negative binomial random variables. This problem has been …
A pruned variant of polar coding is reinvented for all binary erasure channels. For small $\varepsilon>0$, we construct codes with block length $\varepsilon^{-5}$, code rate $\text{Capacity}-\varepsilon$, error probability $\varepsilon$, and encoding and decoding time complexity $O(N\log|\log\varepsilon|)$ per block, equivalently $O(\log|\log\varepsilon|)$ per information bit (Propositions 5 to 8). This result also follows if one applies systematic polar coding [Arıkan 10.1109/LCOMM.2011.061611.110862] with simplified successive cancelation decoding [Alamdar-Yazdi-Kschischang 10.1109/LCOMM.201…
Erasure codes are widely used in today's storage systems to cope with failures. Most of them use the finite field arithmetic. In this paper, we propose an implementation and a coding speed evaluation of an original method called PYRIT (PolYnomial RIng Transform) to perform operations between elements of a finite field into a bigger ring by using fast transforms between these two structures. Working in such a ring is much easier than working in a finite field. Firstly, it reduces the coding complexity by design. Secondly, it allows simple but efficient \texttt{xor}-based implementations by unro…
The complexity of software implementations of MDS erasure codes mainly depends on the efficiency of the finite field operations implementation. In this paper, we propose a method to reduce the complexity of the finite field multiplication by using fast transforms between a field and a ring to perform the multiplication in a ring. We show that moving to a ring reduces the complexity of the operations. Then, we show that this construction allows the use of simple scheduling to reduce the number of operations.
An erasure channel with a fixed alphabet size $q$, where $q \gg 1$, is studied. It is proved that over any erasure channel (with or without memory), Maximum Distance Separable (MDS) codes achieve the minimum probability of error (assuming maximum likelihood decoding). Assuming a memoryless erasure channel, the error exponent of MDS codes are compared with that of random codes and linear random codes. It is shown that the envelopes of all these exponents are identical for rates above the critical rate. Noting the optimality of MDS codes, it is concluded that both random codes and linear random …
Practical random network coding based schemes for multicast include a header in each packet that records the transformation between the sources and the terminal. The header introduces an overhead that can be significant in certain scenarios. In previous work, parity check matrices of error control codes along with error decoding were used to reduce this overhead. In this work we propose novel packet formats that allow us to use erasure decoding and list decoding. Both schemes have a smaller overhead compared to the error decoding based scheme, when the number of sources combined in a packet is…
We derive the optimum second-order coding rates, known as second-order capacities, for erasure and list decoding. For erasure decoding for discrete memoryless channels, we show that second-order capacity is $\sqrt{V}Φ^{-1}(ε_t)$ where $V$ is the channel dispersion and $ε_t$ is the total error probability, i.e., the sum of the erasure and undetected errors. We show numerically that the expected rate at finite blocklength for erasures decoding can exceed the finite blocklength channel coding rate. We also show that the analogous result also holds for lossless source coding with decoder side info…
In this paper, we investigate the impact of coding on the Age of Information (AoI) in a two-user broadcast symbol erasure channel with feedback. We assume each update consists of $K$ symbols and the source is able to broadcast one symbol in each time slot. Due to random channel noise, the intended symbol at each user will be erased according to an independent and identically distributed (i.i.d.) Bernoulli process. A user is able to successfully decode an update if it accumulates sufficient information and successfully decodes the $K$ symbols of the update. Assuming prefect feedback information…
Flash memory is a non-volatile computer memory comprised of blocks of cells, wherein each cell is implemented as either NAND or NOR floating gate. NAND flash is currently the most widely used type of flash memory. In a NAND flash memory, every block of cells consists of numerous pages; rewriting even a single page requires the whole block to be erased and reprogrammed. Block erasures determine both the longevity and the efficiency of a flash memory. Therefore, when data in a NAND flash memory are reorganized, minimizing the total number of block erasures required to achieve the desired data mo…
In this paper, we propose a new rateless coded cooperation scheme for a general multi-user cooperative wireless system. We develop cooperation methods based on Raptor codes with the assumption that the channels face erasure with specific erasure probabilities and transmitters have no channel state information. A fully coded cooperation (FCC) and a partially coded cooperation (PCC) strategy are developed to maximize the average system throughput. Both PCC and FCC schemes have been analyzed through AND-OR tree analysis and a linear programming optimization problem is then formulated to find the …
We study the problem of universal decoding for unknown discrete memoryless channels in the presence of erasure/list option at the decoder, in the random coding regime. Specifically, we harness a universal version of Forney's classical erasure/list decoder developed in earlier studies, which is based on the competitive minimax methodology, and guarantees universal achievability of a certain fraction of the optimum random coding error exponents. In this paper, we derive an exact single-letter expression for the maximum achievable fraction. Examples are given in which the maximal achievable fract…
In a distributed storage system, code symbols are dispersed across space in nodes or storage units as opposed to time. In settings such as that of a large data center, an important consideration is the efficient repair of a failed node. Efficient repair calls for erasure codes that in the face of node failure, are efficient in terms of minimizing the amount of repair data transferred over the network, the amount of data accessed at a helper node as well as the number of helper nodes contacted. Coding theory has evolved to handle these challenges by introducing two new classes of erasure codes,…
The conversational quality of voice over IP (VoIP) depends on packet-loss rates, burstiness of packet loss, and delays (or latencies). The benefits for conversational voice quality of erasure coding attributable to its reduction in packet loss rates are widely appreciated. When block erasure coding is used, our analysis shows how those benefits are reduced or even eliminated by increases in delays and in a measure of burstiness of packet loss. We nevertheless show that the net effect of those three factors is still positive over a wide range of network loss rates provided that block sizes are …
This paper considers the memoryless input-constrained binary erasure channel (BEC). The channel input constraint is the $(d,\infty)$-runlength limited (RLL) constraint, which mandates that any pair of successive $1$s in the input sequence be separated by at least $d$ $0$s. We consider a scenario where there is causal, noiseless feedback from the decoder. We demonstrate a simple, labelling-based, zero-error feedback coding scheme, which we prove to be feedback capacity-achieving, and, as a by-product, obtain an explicit characterization of the feedback capacity. Our proof is based on showing th…
A message composed of packets is transmitted using erasure and channel coding over a fading channel with no feedback. For this scenario, the paper explores the trade-off between the redundancies allocated to the packet-level erasure code and the channel code, along with an objective of a low probability of failure to recover the message. To this end, we consider a fading model that we term proportional-diversity block fading (PD block fading). For a fixed overall code rate and transmit power, we formulate an optimization problem to numerically find the optimal channel-coding rate (and thus the…
We study the application of polar codes in deletion channels by analyzing the cascade of a binary erasure channel (BEC) and a deletion channel. We show how polar codes can be used effectively on a BEC with a single deletion, and propose a list decoding algorithm with a cyclic redundancy check for this case. The decoding complexity is $O(N^2\log N)$, where $N$ is the blocklength of the code. An important contribution is an optimization of the amount of redundancy added to minimize the overall error probability. Our theoretical results are corroborated by numerical simulations which show that th…
The perceived quality of real-time media delivered over IP networks depends on both the rate and burstiness of packet loss. In this paper, we develop a new mathematical model for the residual burstiness of loss under erasure coding. We derive the expected number of consecutive losses in a burst as a function of erasure coding parameters and the network loss probability assuming a Bernoulli model for network losses.
The amount of digital data is rapidly growing. There is an increasing use of a wide range of computer systems, from mobile devices to large-scale data centers, and important for reliable operation of all computer systems is mitigating the occurrence and the impact of errors in digital data. The demand for new ultra-fast and highly reliable coding techniques for data at rest and for data in transit is a major research challenge. Reliability is one of the most important design requirements. The simplest way of providing a degree of reliability is by using data replication techniques. However, re…
We define a class of multi--hop erasure networks that approximates a wireless multi--hop network. The network carries unicast flows for multiple users, and each information packet within a flow is required to be decoded at the flow destination within a specified delay deadline. The allocation of coding rates amongst flows/users is constrained by network capacity. We propose a proportional fair transmission scheme that maximises the sum utility of flow throughputs. This is achieved by {\em jointly optimising the packet coding rates and the allocation of bits of coded packets across transmission…
In this paper we describe a network coding scheme for the Broadcast Erasure Channel with multiple unicast stochastic flows, in the case of a single source transmitting packets to $N$ users, where per-slot feedback is fed back to the transmitter in the form of ACK/NACK messages. This scheme performs only binary (XOR) operations and involves a network of queues, along with special rules for coding and moving packets among the queues, that ensure instantaneous decodability. The system under consideration belongs to a class of networks whose stability properties have been analyzed in earlier work,…
We design low-complexity error correction coding schemes for channels that introduce different types of errors and erasures: on the one hand, the proposed schemes can successfully deal with symbol errors and erasures, and, on the other hand, they can also successfully handle phased burst errors and erasures.
We present nonasymptotic achievability and converse bounds on the maximum coding rate (for a fixed average error probability and a fixed average blocklength) of variable-length full-feedback (VLF) and variable-length stop-feedback (VLSF) codes operating over a binary erasure channel (BEC). For the VLF setup, the achievability bound relies on a scheme that maps each message onto a variable-length Huffman codeword and then repeats each bit of the codeword until it is received correctly. The converse bound is inspired by the meta-converse framework by Polyanskiy, Poor, and Verdú (2010) and relies…
We present a closed-form expression for the minimal delay that is achievable in a setting that combines a buffer and an erasure code, used to mitigate the packet delay variance. The erasure code is modeled according to the recent information-theoretic results on finite block length codes. Evaluations reveal that accurate knowledge of the network parameters is essential for optimal operation. Moreover, it is shown that, when the network packet delay variance is large, the buffer delay becomes negligible. Therefore, in this case the delay budget should be spent mainly on the erasure code.
Based on the erasure channel FEC model as defined in multimedia wireless broadcast standards, we illustrate how doping mechanisms included in the design of erasure coding and decoding may improve the scalability of the packet throughput, decrease overall latency and potentially differentiate among classes of multimedia subscribers regardless of their signal quality. We describe decoding mechanisms that allow for linear complexity and give complexity bounds when feedback is available. We show that elaborate coding schemes which include pre-coding stages are inferior to simple Ideal Soliton base…
Previous works on age of information and erasure channels have dealt with specific models and computed the average age or average peak age for certain settings. In this paper, given a source that produces a letter every $T_s$ seconds and an erasure channel that can be used every $T_c$ seconds, we ask what is the coding strategy that minimizes the time-average age of information that an observer of the channel output incurs. We first analyze the case where the source alphabet and the channel-input alphabet have the same size. We show that a trivial coding strategy is optimal and a closed form e…
Motivated by applications of rateless coding, decision feedback, and ARQ, we study the problem of universal decoding for unknown channels, in the presence of an erasure option. Specifically, we harness the competitive minimax methodology developed in earlier studies, in order to derive a universal version of Forney's classical erasure/list decoder, which in the erasure case, optimally trades off between the probability of erasure and the probability of undetected error. The proposed universal erasure decoder guarantees universal achievability of a certain fraction $ξ$ of the optimum error expo…
We study sequential coding of Markov sources under an error propagation constraint. An encoder sequentially compresses a sequence of vector-sources that are spatially i.i.d. but temporally correlated according to a first-order Markov process. The channel erases up to B packets in a single burst, but reveals all other packets to the destination. The destination is required to reproduce all the source-vectors instantaneously and in a lossless manner, except those sequences that occur in an error propagation window of length B + W following the start of the erasure burst. We define the rate-recov…
One to Many communications are expected to be among the killer applications for the currently discussed 5G standard. The usage of coding mechanisms is impacting broadcasting standard quality, as coding is involved at several levels of the stack, and more specifically at the application layer where Rateless, LDPC, Reed Solomon codes and network coding schemes have been extensively studied, optimized and standardized in the past. Beyond reusing, extending or adapting existing application layer packet coding mechanisms based on previous schemes and designed for the foregoing LTE or other broadcas…
In this paper we study the problem of storing reliably an archive of versioned data. Specifically, we focus on systems where the differences (deltas) between subsequent versions rather than the whole objects are stored - a typical model for storing versioned data. For reliability, we propose erasure encoding techniques that exploit the sparsity of information in the deltas while storing them reliably in a distributed back-end storage system, resulting in improved I/O read performance to retrieve the whole versioned archive. Along with the basic techniques, we propose a few optimization heurist…
In this paper, we propose two novel physical layer aware adaptive network coding and coded modulation schemes for time variant channels. The proposed schemes have been applied to different satellite communications scenarios with different Round Trip Times (RTT). Compared to adaptive network coding, and classical non-adaptive network coding schemes for time variant channels, as benchmarks, the proposed schemes demonstrate that adaptation of packet transmission based on the channel variation and corresponding erasures allows for significant gains in terms of throughput, delay and energy efficien…
We consider the N-user broadcast erasure channel with public feedback and side information. Before the beginning of transmission, each receiver knows a function of the messages of some of the other receivers. This situation arises naturally in wireless and in particular cognitive networks where a node may overhear transmitted messages destined to other nodes before transmission over a given broadcast channel begins. We provide an upper bound to the capacity region of this system. Furthermore, when the side information is linear, we show that the bound is tight for the case of two-user broadcas…
We introduce a definition for \emph{Families of Optimal Binary Non-MDS Erasure Codes} for $[n, k]$ codes over $GF(2)$, and propose an algorithm for finding those families by using hill climbing techniques over Balanced XOR codes. Due to the hill climbing search, those families of codes have always better decoding probability than the codes generated in a typical Random Linear Network Coding scenario, i.e., random linear codes. We also show a surprising result that for small values of $k$, the decoding probability of our codes in $GF(2)$ is very close to the decoding probability of the codes ob…
Consider a lossy packet network of queues, communicating over a wireless medium. This paper presents a throughput-optimal transmission strategy for a unicast network when feedback is available, which has the following advantages: It requires a very limited form of acknowledgment feedback. It is completely distributed, and independent of the network topology. Finally, communication at the information theoretic cut-set rate requires no network coding and no rateless coding on the packets. This simple strategy consists of each node randomly choosing a packet from its buffer to transmit at each op…
We consider broadcasting a block of packets to multiple wireless receivers under random packet erasures using instantly decodable network coding (IDNC). The sender first broadcasts each packet uncoded once, then generates coded packets according to receivers' feedback about their missing packets. We focus on strict IDNC (S-IDNC), where each coded packet includes at most one missing packet of every receiver. But we will also compare it with general IDNC (G-IDNC), where this condition is relaxed. We characterize two fundamental performance limits of S-IDNC: 1) the number of transmissions to comp…
This work considers the smart repeater network where a single source $s$ wants to send two independent packet streams to destinations $\{d_1,d_2\}$ with the help of relay $r$. The transmission from $s$ or $r$ is modeled by packet erasure channels: For each time slot, a packet transmitted by $s$ may be received, with some probabilities, by a random subset of $\{d_1,d_2,r\}$; and those transmitted by $r$ will be received by a random subset of $\{d_1,d_2\}$. Interference is avoided by allowing at most one of $\{s,r\}$ to transmit in each time slot. One example of this model is any cellular networ…
The problem of error-control in random linear network coding is considered. A ``noncoherent'' or ``channel oblivious'' model is assumed where neither transmitter nor receiver is assumed to have knowledge of the channel transfer characteristic. Motivated by the property that linear network coding is vector-space preserving, information transmission is modelled as the injection into the network of a basis for a vector space $V$ and the collection by the receiver of a basis for a vector space $U$. A metric on the projective geometry associated with the packet space is introduced, and it is shown …
Distributed matrix multiplication is widely used in several scientific domains. It is well recognized that computation times on distributed clusters are often dominated by the slowest workers (called stragglers). Recent work has demonstrated that straggler mitigation can be viewed as a problem of designing erasure codes. For matrices $\mathbf A$ and $\mathbf B$, the technique essentially maps the computation of $\mathbf A^T \mathbf B$ into the multiplication of smaller (coded) submatrices. The stragglers are treated as erasures in this process. The computation can be completed as long as a cer…
We consider the problem of constructing an erasure code for storage over a network when the data sources are distributed. Specifically, we assume that there are n storage nodes with limited memory and k<n sources generating the data. We want a data collector, who can appear anywhere in the network, to query any k storage nodes and be able to retrieve the data. We introduce Decentralized Erasure Codes, which are linear codes with a specific randomized structure inspired by network coding on random bipartite graphs. We show that decentralized erasure codes are optimally sparse, and lead to reduc…
SD codes are erasure codes that address the mixed failure mode of current RAID systems. Rather than dedicate entire disks to erasure coding, as done in RAID-5, RAID-6 and Reed-Solomon coding, an SD code dedicates entire disks, plus individual sectors to erasure coding. The code then tolerates combinations of disk and sector errors, rather than solely disk errors. It is been an open problem to construct general codes that have the SD property, and previous work has relied on Monte Carlo searches. In this paper, we present two general constructions that address the cases with one disk and two se…
In this paper, we consider a few iterative decoding schemes for the joint source-channel coding of correlated sources. Specifically, we consider the joint source-channel coding of two erasure correlated sources with transmission over different erasure channels. Our main interest is in determining whether or not various code ensembles can achieve the capacity region universally over varying channel conditions. We consider two ensembles in the class of low-density generator-matrix (LDGM) codes known as Luby-Transform (LT) codes and one ensemble of low-density parity-check (LDPC) codes. We analyz…
Analog coding is a low-complexity method to combat erasures, based on linear redundancy in the signal space domain. Previous work examined "band-limited discrete Fourier transform (DFT)" codes for Gaussian channels with erasures or impulses. We extend this concept to source coding with "erasure side-information" at the encoder and show that the performance of band-limited DFT can be significantly improved using irregular spectrum, and more generally, using equiangular tight frames (ETF). Frames are overcomplete bases and are widely used in mathematics, computer science, engineering, and statis…
As sensing and instrumentation play an increasingly important role in systems controlled over wired and wireless networks, the need to better understand delay-sensitive communication becomes a prime issue. Along these lines, this article studies the operation of data links that employ incremental redundancy as a practical means to protect information from the effects of unreliable channels. Specifically, this work extends a powerful methodology termed sequential differential optimization to choose near-optimal block sizes for hybrid ARQ over erasure channels. In doing so, an interesting connec…
This paper proposes a fountain coding system which has lower space decoding complexity and lower decoding erasure rate than the Raptor coding systems. The main idea of the proposed fountain code is employing shift and exclusive OR to generate the output packets. This technique is known as the zigzag decodable code, which is efficiently decoded by the zigzag decoder. In other words, we propose a fountain code based on the zigzag decodable code in this paper. Moreover, we analyze the overhead for the received packets, decoding erasure rate, decoding complexity, and asymptotic overhead of the pro…
We study the effects of introducing a feedback channel in the erasure source-broadcast problem for the case of three receivers. In our problem formulation, we wish to transmit a binary source to three users over the erasure broadcast channel when causal feedback is universally available to the transmitter and all three receivers. Each receiver requires a certain fraction of the source sequence, and we are interested in the minimum latency, or transmission time, required to serve them all. We propose a queue-based hybrid digital-analog coding scheme that achieves optimal performance for the dur…
We consider an energy harvesting transmitter sending status updates to a receiver over an erasure channel, where each status update is of length $k$ symbols. The energy arrivals and the channel erasures are independent and identically distributed (i.i.d.) and Bernoulli distributed in each slot. In order to combat the effects of the erasures in the channel and the uncertainty in the energy arrivals, we use channel coding to encode the status update symbols. We consider two types of channel coding: maximum distance separable (MDS) codes and rateless erasure codes. For each of these models, we st…
The problem of error control in random linear network coding is addressed from a matrix perspective that is closely related to the subspace perspective of Kötter and Kschischang. A large class of constant-dimension subspace codes is investigated. It is shown that codes in this class can be easily constructed from rank-metric codes, while preserving their distance properties. Moreover, it is shown that minimum distance decoding of such subspace codes can be reformulated as a generalized decoding problem for rank-metric codes where partial information about the error is available. This partial i…
