Akisato Kimura, Tomohiko Uyematsu
A coding problem for correlated information sources is investigated. Messages emitted from two correlated sources are jointly encoded, and delivered to two decoders. Each decoder has access to one of the two messages to enable it to reproduce the other message. The rate-distortion function for the coding problem and its interesting properties are clarified.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
In this first part, a computable outer bound is proved for the multiterminal source coding problem, for a setup with two encoders, discrete memoryless sources, and bounded distortion measures.
A conjectural expression of the asymptotic gap between the rate-distortion function of an arbitrary generalized Gaussian multiterminal source coding system and that of its centralized counterpart in the high-resolution regime is proposed. The validity of this expression is verified when the number of sources is no more than 3.
In this paper, we consider a class of multiterminal source coding problems, each subject to distortion constraints computed using a specific, entropy-based, distortion measure. We provide the achievable rate distortion region for two cases and, in so doing, we demonstrate a relationship between the lossy multiterminal source coding problems with our specific distortion measure and (1) the canonical Slepian-Wolf lossless distributed source coding network, and (2) the Ahlswede-Körner-Wyner source coding with side information problem in which only one of the sources is recovered losslessly.
An enhanced covering lemma for a Markov chain is proved in this paper, and then the distributed source coding problem of correlated general sources with one average distortion criterion under fixed-length coding is investigated. Based on the enhanced lemma, a sufficient and necessary condition for determining the achievability of rate-distortion triples is given.
An information-spectrum approach is applied to solve the multiterminal source coding problem for correlated general sources, where sources may be nonstationary and/or nonergodic, and the distortion measure is arbitrary and may be nonadditive. A general formula for the rate-distortion region of the multiterminal source coding problem with the maximum distortion criterion under fixed-length coding is shown in this correspondence.
We prove a new outer bound on the rate-distortion region for the multiterminal source-coding problem. This bound subsumes the best outer bound in the literature and improves upon it strictly in some cases. The improved bound enables us to obtain a new, conclusive result for the binary erasure version of the "CEO problem." The bound recovers many of the converse results that have been established for special cases of the problem, including the recent one for the Gaussian version of the CEO problem.
We consider the classical two-encoder multiterminal source coding problem where distortion is measured under logarithmic loss. We provide a single-letter characterization of the achievable rate distortion region for arbitrarily correlated sources with finite alphabets. In doing so, we also give the rate distortion region for the $m$-encoder CEO problem (also under logarithmic loss). Several applications and examples are given.
In this paper, we present iterative algorithms that numerically compute the rate-distortion regions of two problems: the two-encoder multiterminal source coding problem and the Chief Executive Officer (CEO) problem, both under logarithmic loss distortion measure.With the clear connection of these models with the distributed information bottleneck method, the proposed algorithms may find usefulness in a variety of applications, such as clustering, pattern recognition and learning. We illustrate the efficiency of our algorithms through some numerical examples.
An intuitive outer bound for the multiterminal source coding problem is given. The proposed bound explicitly couples the rate distortion functions for each source and correlation measures which derive from a "strong" data processing inequality. Unlike many standard outer bounds, the proposed bound is not parameterized by a continuous family of auxiliary random variables, but instead only requires maximizing two ratios of divergences which do not depend on the distortion functions under consideration.
This work considers the quadratic Gaussian multiterminal (MT) source coding problem and provides a new sufficient condition for the Berger-Tung sum-rate bound to be tight. The converse proof utilizes a set of virtual remote sources given which the MT sources are block independent with a maximum block size of two. The given MT source coding problem is then related to a set of two-terminal problems with matrix-distortion constraints, for which a new lower bound on the sum-rate is given. Finally, a convex optimization problem is formulated and a sufficient condition derived for the optimal BT sch…
In the first paper of this two part communication, we solved in a unified framework a variety of two terminal source coding problems with noncooperative encoders, thereby consolidating works of Shannon, Slepian-Wolf, Wyner, Ahlswede-Körner, Wyner-Ziv, Berger {\em et al.} and Berger-Yeung. To achieve such unification we made use of a fundamental principle that dissociates bulk of the analysis from the distortion criterion at hand (if any) and extends the typicality arguments of Shannon and Wyner-Ziv. In this second paper, we generalize the fundamental principle for any number of sources and on …
Consider a generalized multiterminal source coding system, where $\ell\choose m$ encoders, each observing a distinct size-$m$ subset of $\ell$ ($\ell\geq 2$) zero-mean unit-variance symmetrically correlated Gaussian sources with correlation coefficient $ρ$, compress their observations in such a way that a joint decoder can reconstruct the sources within a prescribed mean squared error distortion based on the compressed data. The optimal rate-distortion performance of this system was previously known only for the two extreme cases $m=\ell$ (the centralized case) and $m=1$ (the distributed case)…
Consider the following information theoretic setup wherein independent codebooks of N correlated random variables are generated according to their respective marginals. The problem of determining the conditions on the rates of codebooks to ensure the existence of at least one codeword tuple which is jointly typical with respect to a given joint density (called the multivariate covering lemma) has been studied fairly well and the associated rate regions have found applications in several source coding scenarios. However, several multiterminal source coding applications, such as the general mult…
In this work, we investigate the minimum energy of transmitting correlated sources over the Gaussian multiple-access channel (MAC). Compared to other works on joint source-channel coding, we consider the general scenario where the source and channel bandwidths are not naturally matched. In particular, we proposed the use of hybrid digital-analog coding over to improve the transmission energy efficiency. Different models of correlated sources are studied. We first consider lossless transmission of binary sources over the MAC. We then treat lossy transmission of Gaussian sources over the Gaussia…
We investigate distributed source coding of two correlated sources X and Y where messages are passed to a decoder in a cascade fashion. The encoder of X sends a message at rate R_1 to the encoder of Y. The encoder of Y then sends a message to the decoder at rate R_2 based both on Y and on the message it received about X. The decoder's task is to estimate a function of X and Y. For example, we consider the minimum mean squared-error distortion when encoding the sum of jointly Gaussian random variables under these constraints. We also characterize the rates needed to reconstruct a function of X …
This paper investigates the interplay between cooperation and achievable rates in multi-terminal networks. Cooperation refers to the process of nodes working together to relay data toward the destination. There is an inherent tradeoff between achievable information transmission rates and the level of cooperation, which is determined by how many nodes are involved and how the nodes encode/decode the data. We illustrate this trade-off by studying information-theoretic decode-forward based coding strategies for data transmission in multi-terminal networks. Decode-forward strategies are usually di…
This paper deals with a universal coding problem for a certain kind of multiterminal source coding system that we call the complementary delivery coding system. In this system, messages from two correlated sources are jointly encoded, and each decoder has access to one of the two messages to enable it to reproduce the other message. Both fixed-to-fixed length and fixed-to-variable length lossless coding schemes are considered. Explicit constructions of universal codes and bounds of the error probabilities are clarified via type-theoretical and graph-theoretical analyses. [[Keywords]] multiterm…
We consider a multiterminal source coding problem in which a source is estimated at a central processing unit from lossy-compressed remote observations. Each lossy-encoded observation is produced by a remote sensor which obtains a noisy version of the source and compresses this observation minimizing a local distortion measure which depends only on the marginal distribution of its observation. The central node, on the other hand, has knowledge of the joint distribution of the source and all the observations and produces the source estimate which minimizes a different distortion measure between…
We consider the problem of how to determine a fair source coding rate allocation method for the lossless data compression problem in multiterminal networks, e.g, the wireless sensor network where there are a large number of sources to be encoded. We model this problem by a game-theoretic approach and present a decomposition method for obtaining the Shapley value, a fair source coding rate vector in the Slepian-Wolf achievable region. We formulate a coalitional game model where the entropy function quantifies the cost incurred due to the source coding rates in each coalition. In the typical cas…
The problem of secure multiterminal source coding with side information at the eavesdropper is investigated. This scenario consists of a main encoder (referred to as Alice) that wishes to compress a single source but simultaneously satisfying the desired requirements on the distortion level at a legitimate receiver (referred to as Bob) and the equivocation rate --average uncertainty-- at an eavesdropper (referred to as Eve). It is further assumed the presence of a (public) rate-limited link between Alice and Bob. In this setting, Eve perfectly observes the information bits sent by Alice to Bob…
We study the problem of receive beamforming in uplink cascade multiple-input multiple-output (MIMO) systems as an instance of that of cascade multiterminal source coding for lossy function computation. Using this connection, we develop two coding schemes for the second and show that their application leads to beamforming schemes for the first. In the first coding scheme, each terminal in the cascade sends a description of the source that it observes; the decoder reconstructs all sources, lossily, and then computes an estimate of the desired function. This scheme improves upon standard routing …
We prove a new extremal inequality, motivated by the vector Gaussian broadcast channel and the distributed source coding with a single quadratic distortion constraint problems. As a corollary, this inequality yields a generalization of the classical entropy-power inequality (EPI). As another corollary, this inequality sheds insight into maximizing the differential entropy of the sum of two dependent random variables.
We consider the distributed source coding system of $L$ correlated Gaussian sources $Y_i,i=1,2,...,L$ which are noisy observations of correlated Gaussian remote sources $X_k, k=1,2,...,K$. We assume that $Y^{L}={}^{\rm t}(Y_1,Y_2,$ $..., Y_L)$ is an observation of the source vector $X^K={}^{\rm t}(X_1,X_2,..., X_K)$, having the form $Y^L=AX^K+N^L$, where $A$ is a $L\times K$ matrix and $N^L={}^{\rm t}(N_1,N_2,...,N_L)$ is a vector of $L$ independent Gaussian random variables also independent of $X^K$. In this system $L$ correlated Gaussian observations are separately compressed by $L$ encoders…
The three-node multiterminal lossy source coding problem is investigated. We derive an inner bound to the general rate-distortion region of this problem which is a natural extension of the seminal work by Kaspi'85 on the interactive two-terminal source coding problem. It is shown that this (rather involved) inner bound contains several rate-distortion regions of some relevant source coding settings. In this way, besides the non-trivial extension of the interactive two terminal problem, our results can be seen as a generalization and hence unification of several previous works in the field. Spe…
One of the main focus in federated learning (FL) is the communication efficiency since a large number of participating edge devices send their updates to the edge server at each round of the model training. Existing works reconstruct each model update from edge devices and implicitly assume that the local model updates are independent over edge devices. In FL, however, the model update is an indirect multi-terminal source coding problem, also called as the CEO problem where each edge device cannot observe directly the gradient that is to be reconstructed at the decoder, but is rather provided …
A multiterminal lossy coding problem, which includes various problems such as the Wyner-Ziv problem and the complementary delivery problem as special cases, is considered. It is shown that any point in the achievable rate-distortion region can be attained even if the source statistics are not known.
This paper deals with a universal coding problem for a certain kind of multiterminal source coding network called a generalized complementary delivery network. In this network, messages from multiple correlated sources are jointly encoded, and each decoder has access to some of the messages to enable it to reproduce the other messages. Both fixed-to-fixed length and fixed-to-variable length lossless coding schemes are considered. Explicit constructions of universal codes and the bounds of the error probabilities are clarified by using methods of types and graph-theoretical analysis.
Strong typicality and the Markov lemma have been used in the proofs of several multiterminal source coding theorems. Since these two tools can be applied to finite alphabets only, the results proved by them are subject to the same limitation. Recently, a new notion of typicality, namely unified typicality, has been defined. It can be applied to both finite or countably infinite alphabets, and it retains the asymptotic equipartition property and the structural properties of strong typicality. In this paper, unified typicality is used to derive a version of the Markov lemma which works on both f…
This paper deals with a coding problem called complementary delivery, where messages from two correlated sources are jointly encoded and each decoder reproduces one of two messages using the other message as the side information. Both lossless and lossy universal complementary delivery coding schemes are investigated. In the lossless case, it is demonstrated that a universal complementary delivery code can be constructed by only combining two Slepian-Wolf codes. Especially, it is shown that a universal lossless complementary delivery code, for which error probability is exponentially tight, ca…
In this paper, we propose a structured peer-to-peer (P2P) distribution scheme based on Fast Fourier Transform (FFT) graphs. We build a peer-to-peer network that reproduces the FFT graph initially designed for hardware FFT codecs. This topology allows content delivery with a maximum diversity level for a minimum global complexity. The resulting FFTbased network is a structured architecture with an adapted network coding that brings flexibility upon content distribution and robustness upon the dynamic nature of the network. This structure can achieve optimal capacity in terms of content recovery…
The complementary problems of masking and amplifying channel state information in the Gel'fand-Pinsker channel have recently been solved by Merhav and Shamai, and Kim et al., respectively. In this paper, we study a related source coding problem. Specifically, we consider the two-encoder source coding setting where one source is to be amplified, while the other source is to be masked. In general, there is a tension between these two objectives which is characterized by the amplification-masking tradeoff. In this paper, we give a single-letter description of this tradeoff. We apply this result, …
On Finite Blocklength Lossy Source Coding
本专著系统综述有损信源编码的二阶渐近理论，围绕单信源率失真及其五类推广和四种多终端模型，给出有限块长性能界与未来方向。
In this article, we study the problem of secret key generation in the multiterminal source model, where the terminals have access to correlated Gaussian sources. We assume that the sources form a Markov chain on a tree. We give a nested lattice-based key generation scheme whose computational complexity is polynomial in the number, N , of independent and identically distributed samples observed by each source. We also compute the achievable secret key rate and give a class of examples where our scheme is optimal in the fine quantization limit. However, we also give examples that show that our s…
In a single hop broadcast packet erasure network, we demonstrate that it is possible to provide multirate packet delivery outside of what is given by the network min-cut. This is achieved by using a deterministic non-block-based network coding scheme, which allows us to sidestep some of the limitations put in place by the block coding model used to determine the network capacity. Under the network coding scheme we outline, the sender is able to transmit network coded packets above the channel rate of some receivers, while ensuring that they still experience nonzero delivery rates. Interestingl…
In this paper we consider the communication problem that involves transmission of correlated sources over broadcast channels. We consider a graph-based framework for this information transmission problem. The system involves a source coding module and a channel coding module. In the source coding module, the sources are efficiently mapped into a nearly semi-regular bipartite graph, and in the channel coding module, the edges of this graph are reliably transmitted over a broadcast channel. We consider nearly semi-regular bipartite graphs as discrete interface between source coding and channel c…
Cache-aided coded multicast leverages side information at wireless edge caches to efficiently serve multiple unicast demands via common multicast transmissions, leading to load reductions that are proportional to the aggregate cache size. However, the increasingly dynamic, unpredictable, and personalized nature of the content that users consume challenges the efficiency of existing caching-based solutions in which only exact content reuse is explored. This paper generalizes the cache-aided coded multicast problem to specifically account for the correlation among content files, such as, for exa…
A large number of streaming applications use reliable transport protocols such as TCP to deliver content over the Internet. However, head-of-line blocking due to packet loss recovery can often result in unwanted behavior and poor application layer performance. Transport layer coding can help mitigate this issue by helping to recover from lost packets without waiting for retransmissions. We consider the use of an on-line network code that inserts coded packets at strategic locations within the underlying packet stream. If retransmissions are necessary, additional coding packets are transmitted …
Pricing Innovation Under Latency Constraints: A Mean-Field Analysis of Coded Payload Delivery
本文研究了在参与者奖励依赖于重建负载所需时间的低延迟负载交付场景中的定价机制，通过均场方法推导出价格-速率界限，并分析了不同传输模式下的经济价值转化。
In this paper, we consider multiple cache-enabled end-users connected to multiple transmitters through a linear network. We also prevent a totally passive eavesdropper, who sniffs the packets in the delivery phase, from obtaining any information about the original files in cache-aided networks. Three different secure centralized multi-transmitter coded caching scenarios namely, secure multi-transmitter coded caching, secure multi-transmitter coded caching with reduced subpacketization, and secure multi-transmitter coded caching with reduced feedback, are considered and closed-form coding delay…
In this paper, distributed (or multiterminal) source coding with one distortion criterion and correlated messages is considered. This problem can be also called ``Berger-Yeung problem with correlated messages''. It corresponds to the source coding part of the graph-based framework for transmission of a pair of correlated sources over the multiple-access channel (MAC) where one is lossless and the other is lossy. As a result, the achievable rate-distortion region for this problem is provided. It is an information-theoretic characterization of the rate of exponential growth (as a function of the…
In this paper, we deal with the problem of jointly determining the optimal coding strategy and the scheduling decisions when receivers obtain layered data from multiple servers. The layered data is encoded by means of Prioritized Random Linear Coding (PRLC) in order to be resilient to channel loss while respecting the unequal levels of importance in the data, and data blocks are transmitted simultaneously in order to reduce decoding delays and improve the delivery performance. We formulate the optimal coding and scheduling decisions problem in our novel framework with the help of Markov Decisi…
In this paper, we consider the information-theoretic characterization of the set of achievable rates and distortions in a broad class of multiterminal communication scenarios with general continuous-valued sources and channels. A framework is presented which involves fine discretization of the source and channel variables followed by communication over the resulting discretized network. In order to evaluate fundamental performance limits, convergence results for information measures are provided under the proposed discretization process. Using this framework, we consider point-to-point source …
A two-terminal interactive distributed source coding problem with alternating messages for function computation at both locations is studied. For any number of messages, a computable characterization of the rate region is provided in terms of single-letter information measures. While interaction is useless in terms of the minimum sum-rate for lossless source reproduction at one or both locations, the gains can be arbitrarily large for function computation even when the sources are independent. For a class of sources and functions, interaction is shown to be useless, even with infinite messages…
Index coding studies multiterminal source-coding problems where a set of receivers are required to decode multiple (possibly different) messages from a common broadcast, and they each know some messages a priori. In this paper, at the receiver end, we consider a special setting where each receiver knows only one message a priori, and each message is known to only one receiver. At the broadcasting end, we consider a generalized setting where there could be multiple senders, and each sender knows a subset of the messages. The senders collaborate to transmit an index code. This work looks at mini…
In this paper, we consider the use of cross-layer network coding (CLNC), caching, and device-to-device (D2D) communications to jointly optimize the delivery of a set of popular contents to a set of user devices (UDs). In the considered D2D network, a group of near-by UDs cooperate with each other and use NC to combine their cached files, so as the completion time required for delivering all requested contents to all UDs is minimized. Unlike the previous work that considers only one transmitting UD at a time, our work allows multiple UDs to transmit simultaneously given the interference among t…
It is well-known for transform coding of multivariate Gaussian sources, that the Karhunen-Loève transform (KLT) minimizes the mean square error distortion. However, finding the optimal transform for general non-Gaussian sources has been an open problem for decades, despite several important advances that provide some partial answers regarding KLT optimality. In this paper, we present a necessary and sufficient condition for optimality of a transform when high resolution, variable rate quantizers are employed. We hence present not only a complete characterization of when KLT is optimal, but als…
This paper investigates a source coding problem in which two terminals communicating through a relay wish to estimate one another's source within some distortion constraint. The relay has access to side information that is correlated with the sources. Two different schemes based on the order of communication, \emph{distributed source coding/delivery} and \emph{two cascaded rounds}, are proposed and inner and outer bounds for the resulting rate-distortion regions are provided. Examples are provided to show that neither rate-distortion region includes the other one.
Consider a radio access network wherein a base-station is required to deliver a set of order-constrained messages to a set of users over independent erasure channels. This paper studies the delivery time reduction problem using instantly decodable network coding (IDNC). Motivated by time-critical and order-constrained applications, the delivery time is defined, at each transmission, as the number of undelivered messages. The delivery time minimization problem being computationally intractable, most of the existing literature on IDNC propose sub-optimal online solutions. This paper suggests a n…
Non-adaptive joint source network coding of correlated sources is discussed in this paper. By studying the information flow in the network, we propose quantized network coding as an alternative for packet forwarding. This technique has both network coding and distributed source coding advantages, simultaneously. Quantized network coding is a combination of random linear network coding in the (infinite) field of real numbers and quantization to cope with the limited capacity of links. With the aid of the results in the literature of compressed sensing, we discuss theoretical and practical feasi…
Content delivery in a multi-user cache-aided broadcast network is studied, where a server holding a database of correlated contents communicates with the users over a Gaussian broadcast channel (BC). The minimum transmission power required to satisfy all possible demand combinations is studied, when the users are equipped with caches of equal size. A lower bound on the required transmit power is derived, assuming uncoded cache placement, as a function of the cache capacity. A centralized joint cache and channel coding scheme is proposed, which not only utilizes the user's local caches, but als…
In this paper, we study the data gathering problem in the context of power grids by using a network of sensors, where the sensed data have inter-node redundancy. Specifically, we propose a new transmission method, calledquantized network coding, which performs linear net-work coding in the field of real numbers, and quantization to accommodate the finite capacity of edges. By using the concepts in compressed sensing literature, we propose to use l1-minimization to decode the quantized network coded packets, especially when the number of received packets at the decoder is less than the size of …
This paper studies the fundamental limits of content delivery in a cache-aided broadcast network for correlated content generated by a discrete memoryless source with arbitrary joint distribution. Each receiver is equipped with a cache of equal capacity, and the requested files are delivered over a shared error-free broadcast link. A class of achievable correlation-aware schemes based on a two-step source coding approach is proposed. Library files are first compressed, and then cached and delivered using a combination of correlation-unaware multiple-request cache-aided coded multicast schemes.…
Source coding with a side information "vending machine" is a recently proposed framework in which the statistical relationship between the side information and the source, instead of being given and fixed as in the classical Wyner-Ziv problem, can be controlled by the decoder. This control action is selected by the decoder based on the message encoded by the source node. Unlike conventional settings, the message can thus carry not only information about the source to be reproduced at the decoder, but also control information aimed at improving the quality of the side information. In this paper…
Finite-Blocklength Information Theory
本文综述了有限块长信息论的发展，讨论了非渐进行为下的信源编码、信道编码及联合编码的最新进展，旨在为未来无线通信系统的设计提供工程指导。
Side information provides a pivotal role for message delivery in many communication scenarios to accommodate increasingly large data sets, e.g., caching networks. Although index coding provides a fundamental modeling framework to exploit the benefits of side information, the index coding problem itself still remains open and only a few instances have been solved. In this paper, we propose a novel sparse and low- rank optimization modeling framework for the index coding problem to characterize the tradeoff between the amount of side information and the achievable data rate. Specifically, sparsi…
This paper addresses the problem of exponentially increasing sub-packetization with the number of users in a centralized coded caching system by introducing a new coded caching scheme inspired by the symmetric neighboring consecutive side information index coding problem. The scheme has a placement policy where the number of sub-packets required grows only linearly with the number of users, with no restriction on file size, and a delivery policy which is instantaneously decodable. Further, an application of the new delivery scheme in a multi-access coded caching set-up is studied and a few res…
Coded caching is a recently proposed technique that achieves significant performance gains for cache networks compared to uncoded caching schemes. However, this substantial coding gain is attained at the cost of large delivery delay, which is not tolerable in delay-sensitive applications such as video streaming. In this paper, we identify and investigate the tradeoff between the performance gain of coded caching and the delivery delay. We propose a computationally efficient caching algorithm that provides the gains of coding and respects delay constraints. The proposed algorithm achieves the o…
Coding Gain for Age of Information in a Multi-source System with Erasure Channel
本文研究多源系统经 GE 擦除信道传输更新时的 AoI，采用轮询调度与 MDS 编码，刻画长码长平均 AoI，并给出获得最大编码增益的最优编码速率。
We investigate the problem of coded caching for nonuniform demands when the structured clique cover algorithm proposed by Maddah-Ali and Niesen for decentralized caching is used for delivery. We apply this algorithm to all user demands regardless of their request probabilities. This allows for coding among the files that have different request probabilities but makes the allocation of memory to different files challenging during the content placement phase. As our main contribution, we analytically characterize the optimal placement strategy that minimizes the expected delivery rate under a st…
Systems that employ network coding for content distribution convey to the receivers linear combinations of the source packets. If we assume randomized network coding, during this process the network nodes collect random subspaces of the space spanned by the source packets. We establish several fundamental properties of the random subspaces induced in such a system, and show that these subspaces implicitly carry topological information about the network and its state that can be passively collected and inferred. We leverage this information towards a number of applications that are interesting …
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。