It is shown that fully-parallel encoding and decoding schemes with asymptotic block error probability that scales as $O\left(f\left(n\right)\right)$ have Thompson energy that scales as $Ω\left(\sqrt{\ln f\left(n\right)}n\right)$. As well, it is shown that the number of clock cycles (denoted $T\left(n\right)$) required for any encoding or decoding scheme that reaches this bound must scale as $T\left(n\right)\ge\sqrt{\ln f\left(n\right)}$. Similar scaling results are extended to serialized computation.
The Grover information-friction energy model is generalized to three dimensions and the optimal energy of encoding or decoding schemes with probability of block error $P_\mathrm{e}$ is shown to be at least $Ω\left(n\left(\ln P_{\mathrm{e}}\left(n\right)\right)^{\frac{1}{3}}\right)$.
Blau and Michaeli recently introduced a novel concept for inverse problems of signal processing, that is, the perception-distortion tradeoff. We introduce their tradeoff into the rate distortion theory of variable-length lossy source coding in information theory, and clarify the tradeoff among information rate, distortion and perception for general information sources. We also discuss the fixed-length coding with average distortion criterion that was missing in the previous letter.
This paper studies the storage-latency tradeoff in the $3\times3$ wireless interference network with caches equipped at all transmitters and receivers. The tradeoff is characterized by the so-called fractional delivery time (FDT) at given normalized transmitter and receiver cache sizes. We first propose a generic cooperative transmitter/receiver caching strategy with adjustable file splitting ratios. Based on this caching strategy, we then design the delivery phase carefully to turn the considered interference channel opportunistically into broadcast channel, multicast channel, X channel, or a…
Narrowband IoT (NB-IoT) is the latest IoT connectivity solution presented by the 3GPP. NB-IoT introduces coverage classes and introduces a significant link budget improvement by allowing repeated transmissions by nodes that experience high path loss. However, those repetitions necessarily increase the energy consumption and the latency in the whole NB-IoT system. The extent to which the whole system is affected depends on the scheduling of the uplink and downlink channels. We address this question, not treated previously, by developing a tractable model of NB-IoT access protocol operation, com…
High-fidelity, real-time interactive applications are envisioned with the emergence of the Internet of Things (IoT) and tactile Internet by means of ultra-reliable low-latency communications (URLLC). Exploiting time diversity for fulfilling the URLLC requirements in an energy efficient manner is a challenging task due to the nontrivial interplay among packet size, retransmission rounds and delay, and transmit power. In this paper, we study the fundamental energy-latency tradeoff in URLLC systems employing incremental redundancy (IR) hybrid automatic repeat request (HARQ). We cast the average e…
Digital contents in large scale distributed storage systems may have different reliability and access delay requirements, and for this reason, erasure codes with different strengths need to be utilized to achieve the best storage efficiency. At the same time, in such large scale distributed storage systems, nodes fail on a regular basis, and the contents stored on them need to be regenerated and stored on other healthy nodes, the efficiency of which is an important factor affecting the overall quality of service. In this work, we formulate the problem of multilevel diversity coding with regene…
We study secure source-coding with causal disclosure, under the Gaussian distribution. The optimality of Gaussian auxiliary random variables is shown in various scenarios. We explicitly characterize the tradeoff between the rates of communication and secret key. This tradeoff is the result of a mutual information optimization under Markov constraints. As a corollary, we deduce a general formula for Wyner's Common Information in the Gaussian setting.
With the phenomenal growth of the Internet of Things (IoT), Ultra Reliable Low Latency Communications (URLLC) has potentially been the enabler to guarantee the stringent requirements on latency and reliability. However, how to achieve low latency and ultra-reliability with the random arrival remains open. In this paper, a queue-aware variable-length channel coding is presented over the single URLLC user link, in which the finite blocklength of channel coding is determined based on the random arrival. More particularly, a cross-layer approach is proposed for the URLLC user to establish the opti…
This paper studies the tradeoff in privacy and utility in a single-trial multi-terminal guessing (estimation) framework using a system model that is inspired by index coding. There are $n$ independent discrete sources at a data curator. There are $m$ legitimate users and one adversary, each with some side information about the sources. The data curator broadcasts a distorted function of sources to legitimate users, which is also overheard by the adversary. In terms of utility, each legitimate user wishes to perfectly reconstruct some of the unknown sources and attain a certain gain in the esti…
In the Cloud Radio Access Network (C-RAN) architecture, the baseband signals from multiple remote radio heads are processed in a centralized baseband unit (BBU) pool. This architecture allows network operators to adapt the BBU's computational resources to the aggregate access load experienced at the BBU, which can change in every air-interface access frame. The degree of savings that can be achieved by adapting the resources is a tradeoff between savings, adaptation frequency, and increased queuing time. If the time scale for adaptation of the resource multiplexing is greater than the access f…
We investigate the fundamental information theoretic limits of cache-aided wireless networks, in which edge nodes (or transmitters) are endowed with caches that can store popular content, such as multimedia files. This architecture aims to localize popular multimedia content by proactively pushing it closer to the edge of the wireless network, thereby alleviating backhaul load. An information theoretic model of such networks is presented, that includes the introduction of a new metric, namely normalized delivery time (NDT), which captures the worst case time to deliver any requested content to…
This paper studies the fundamental tradeoff between storage and latency in a general wireless interference network with caches equipped at all transmitters and receivers. The tradeoff is characterized by an information-theoretic metric, \emph{normalized delivery time} (NDT), which is the worst-case delivery time of the actual traffic load at a transmission rate specified by degrees of freedom (DoF) of a given channel. We obtain both an achievable upper bound and a theoretical lower bound of the minimum NDT for any number of transmitters, any number of receivers, and any feasible cache size tup…
Coded distributed computing can alleviate the communication load by leveraging the redundant storage and computation resources with coding techniques in distributed computing. In this paper, we study a MapReduce-type distributed computing framework over star topological network, where all the workers exchange information through a common access point. The optimal tradeoff among the normalized number of the stored files (storage load), computed intermediate values (computation load) and transmitted bits in the uplink and downlink (communication loads) are characterized. A coded computing scheme…
This paper considers the Gaussian multiple-access channel (MAC) in the asymptotic regime where the number of users grows linearly with the code length. We propose efficient coding schemes based on random linear models with approximate message passing (AMP) decoding and derive the asymptotic error rate achieved for a given user density, user payload (in bits), and user energy. The tradeoff between energy-per-bit and achievable user density (for a fixed user payload and target error rate) is studied, and it is demonstrated that in the large system limit, a spatially coupled coding scheme with AM…
We consider the storage-retrieval rate tradeoff in private information retrieval (PIR) systems using a Shannon-theoretic approach. Our focus is mostly on the canonical two-message two-database case, for which a coding scheme based on random codebook generation and the binning technique is proposed. This coding scheme reveals a hidden connection between PIR and the classic multiple description source coding problem. We first show that when the retrieval rate is kept optimal, the proposed non-linear scheme can achieve better performance over any linear scheme. Moreover, a non-trivial storage-ret…
Future autonomous systems require wireless connectivity able to support extremely stringent requirements on both latency and reliability. In this paper, we leverage recent developments in the field of finite-blocklength information theory to illustrate how to optimally design wireless systems in the presence of such stringent constraints. Focusing on a multi-antenna Rayleigh block-fading channel, we obtain bounds on the maximum number of bits that can be transmitted within given bandwidth, latency, and reliability constraints, using an orthogonal frequency-division multiplexing system similar …
Asymptotic energy-distortion performance of zero-delay communication scenarios under additive white Gaussian noise is investigated. Using high-resolution analysis for quantizer design, the higher-order term in the logarithm of the distortion (termed the {\em energy-distortion dispersion}) is optimized while keeping the leading term (i.e., {\em energy-distortion exponent}) at its optimal value. For uniform and Gaussian sources, significant gains are observed compared to naïvely performed quantization, i.e., aimed at optimizing the source coding performance instead of the end-to-end distortion i…
We propose a unified coded framework for distributed computing with straggling servers, by introducing a tradeoff between "latency of computation" and "load of communication" for some linear computation tasks. We show that the coded scheme of [1]-[3] that repeats the intermediate computations to create coded multicasting opportunities to reduce communication load, and the coded scheme of [4], [5] that generates redundant intermediate computations to combat against straggling servers can be viewed as special instances of the proposed framework, by considering two extremes of this tradeoff: mini…
This work focuses on the minimum transmission energy required for communicating a pair of correlated Gaussian sources over a two-user Gaussian broadcast channel with noiseless and causal channel output feedback (GBCF). We study the fundamental limit on the required transmission energy for broadcasting a pair of source samples, such that each source can be reconstructed at its respective receiver to within a target distortion, when the source-channel bandwidth ratio is not restricted. We derive a lower bound and three distinct upper bounds on the minimum required energy. For the upper bounds we…
An emerging trend of next generation communication systems is to provide network edges with additional capabilities such as storage resources in the form of caches to reduce file delivery latency. To investigate this aspect, we study the fundamental limits of a cache-aided broadcast-relay wireless network consisting of one central base station, $M$ cache-equipped transceivers and $K$ receivers from a latency-centric perspective. We use the normalized delivery time (NDT) to capture the per-bit latency for the worst-case file request pattern, normalized with respect to a reference interference-f…
Future networks are expected to support various ultra-reliable low-latency communications via wireless links. To avoid the loss of packets and keep the low latency, sliding network coding (SNC) is an emerging technology by generating redundant packets that are the linear combination of the original data packets from the current block and some previous blocks. However, how to take the advantage of re-transmission for SNC is still an open problem since higher reliability could be achieved at the expense of large latency caused by round-trip time (RTT). To deal with this issue, in this paper, we …
The Rate-Distortion-Perception-Classification Tradeoff: Joint Source Coding and Modulation via Inverse-Domain GANs
School of Informatics, Xiamen University（厦门大学信息学院）；School of Electronics and Information Engineering, Guangxi Normal University（广西师范大学电子与信息工程学院）；School of Engineering & Physical Sciences, Heriot-Watt University（赫瑞瓦特大学工程与物理科学学院）
To fully support vertical industries, 5G and its corresponding channel coding are expected to meet requirements of different applications. However, for applications of 5G and beyond 5G (B5G) such as URLLC, the transmission latency is required to be much shorter than that in eMBB. Therefore, the resulting channel code length reduces drastically. In this case, the traditional 1-D channel coding suffers a lot from the performance degradation and fails to deliver strong reliability with very low latency. To remove this bottleneck, new channel coding scheme beyond the existing 1-D one is in urgent …
To improve diversity gain in an interference channel and hence to maximize diversity multiplexing tradeoff (DMT), we propose on-off switched interference alignment (IA) where IA is intermittently utilized by switching IA on/off. For on-off switching, either IA with symbol extension or IA with Alamouti coding is adopted in this paper. Deriving and analyzing DMT of the proposed schemes, we reveal that the intermittent utilization of IA with simultaneous non-unique decoding can improve DMT in the 2-user X-channel with two antennas. Both the proposed schemes are shown to achieve diversity gain of …
Caching is an effective technique to improve user perceived experience for content delivery in wireless networks. Wireless caching differs from traditional web caching in that it can exploit the broadcast nature of wireless medium and hence opportunistically change the network topologies. This paper studies a cache-aided MIMO interference network with 3 transmitters each equipped with M antennas and 3 receivers each with N antennas. With caching at both the transmitter and receiver sides, the network is changed to hybrid forms of MIMO broadcast channel, MIMO X channel, and MIMO multicast chann…
Ultra reliable and low latency communication (URLLC) is a newly introduced service category in 5G to support delay-sensitive applications. In order to support this new service category, 3rd Generation Partnership Project (3GPP) sets an aggressive requirement that a packet should be delivered with 10^-5 packet error rate within 1 ms transmission period. Since the current wireless transmission scheme designed to maximize the coding gain by transmitting capacity achieving long codeblock is not relevant for this purpose, a new transmission scheme to support URLLC is required. In this paper, we pro…
Virtual reality (VR) over wireless is emerging as an important use case of 5G networks. Immersive VR experience requires the delivery of huge data at ultra-low latency, thus demanding ultra-high transmission rate. This challenge can be largely addressed by the recent network architecture known as mobile edge computing (MEC), which enables caching and computing capabilities at the edge of wireless networks. This paper presents a novel MEC-based mobile VR delivery framework that is able to cache parts of the field of views (FOVs) in advance and run certain post-processing procedures at the mobil…
In a distributed computing system operating according to the map-shuffle-reduce framework, coding data prior to storage can be useful both to reduce the latency caused by straggling servers and to decrease the inter-server communication load in the shuffling phase. In prior work, a concatenated coding scheme was proposed for a matrix multiplication task. In this scheme, the outer Maximum Distance Separable (MDS) code is leveraged to correct erasures caused by stragglers, while the inner repetition code is used to improve the communication efficiency in the shuffling phase by means of coded mul…
In the envisioned 5G, uplink grant-free multiple access will become the enabler of ultra-reliable low-latency communications (URLLC) services. By removing the forward scheduling request (SR) and backward scheduling grant (SG), pilot-based channel estimation and data transmission are launched in one-shot communications with the aim of maintaining the reliability of $99.999\% $ or more and latency of 1ms or less under 5G new radio (NR) numerologies. The problem is that channel estimation can easily suffer from pilot aware attack which significantly reduces the system reliability. To solve this, …
Ultra-reliability and low-latency are pivotal requirements of the new 6th generation of communication systems (xURLLC). Over the past years, to increase throughput, adaptive active antennas were introduced in advanced wireless communications, specifically in the domain of millimeter-wave (mmWave). Consequently, new lower-layer techniques were proposed to cope with practical challenges of high dimensional and electronically-steerable beams. The transition from omni-directional to highly directional antennas presents a new type of wireless systems that deliver high bandwidth, but that are suscep…
Redundancy is abundant in Fog networks (i.e., many computing and storage points) and grows linearly with network size. We demonstrate the transformational role of coding in Fog computing for leveraging such redundancy to substantially reduce the bandwidth consumption and latency of computing. In particular, we discuss two recently proposed coding concepts, namely Minimum Bandwidth Codes and Minimum Latency Codes, and illustrate their impacts in Fog computing. We also review a unified coding framework that includes the above two coding techniques as special cases, and enables a tradeoff between…
Wyner's soft-handoff network is considered where transmitters simultaneously send messages of enhanced mobile broadband (eMBB) and ultra-reliable low-latency communication (URLLC) services. Due to the low-latency requirements, the URLLC messages are transmitted over fewer channel uses compared to the eMBB messages. To improve the reliability of the URLLC transmissions, we propose a coding scheme with finite blocklength codewords that exploits dirty-paper coding (DPC) to precancel the interference from eMBB transmissions. Rigorous bounds are derived for the error probabilities of eMBB and URLLC…
Here we write in a unified fashion (using "R(P, Q, D)") the random coding exponents in channel coding and lossy source coding. We derive their explicit forms and show, that, for a given random codebook distribution Q, the channel decoding error exponent can be viewed as an encoding success exponent in lossy source coding, and the channel correct-decoding exponent can be viewed as an encoding failure exponent in lossy source coding. We then extend the channel exponents to arbitrary D, which corresponds for D > 0 to erasure decoding and for D < 0 to list decoding. For comparison, we also derive …
In order to characterize the fundamental limit of the tradeoff between the amount of cache memory and the delivery transmission rate of multiuser caching systems, various coding schemes have been proposed in the literature. These schemes can largely be categorized into two classes, namely uncoded prefetching schemes and coded prefetching schemes. While uncoded prefetching schemes in general offer order-wise optimal performance, coded prefetching schemes often have better performance at the low cache memory regime. The significant differences in the coding components between the two classes may…
This paper focuses on mitigating the impact of stragglers in distributed learning system. Unlike the existing results designed for a fixed number of stragglers, we developed a new scheme called Adaptive Gradient Coding(AGC) with flexible tolerance of various number of stragglers. Our scheme gives an optimal tradeoff between computation load, straggler tolerance and communication cost. In particular, it allows to minimize the communication cost according to the real-time number of stragglers in the practical environments. Implementations on Amazon EC2 clusters using Python with mpi4py package v…
In this paper, we consider a federated learning problem over wireless channel that takes into account the coding rate and packet transmission errors. Communication channels are modelled as packet erasure channels (PEC), where the erasure probability is determined by the block length, code rate, and signal-to-noise ratio (SNR). To lessen the effect of packet erasure on the FL performance, we propose two schemes in which the central node (CN) reuses either the past local updates or the previous global parameters in case of packet erasure. We investigate the impact of coding rate on the convergen…
