The Infinite-message Limit of Two-terminal Interactive Source Coding
Nan Ma, Prakash Ishwar
备注：
40 pages, 5 figures. Submitted to IEEE Transactions on Information Theory in Sep 2010 and revised in Mar 2012. This version (version 3) is the latest revised version. Version 1 was published in Allerton Conference 2009. Version 2 contains extensions to the rate-distortion theory and an achievable proof of the sum-rate R* in the second example
A two-terminal interactive function computation problem with alternating messages is studied within the framework of distributed block source coding theory. For any finite number of messages, a single-letter characterization of the sum-rate-distortion function was established in previous works using standard information-theoretic techniques. This, however, does not provide a satisfactory characterization of the infinite-message limit, which is a new, unexplored dimension for asymptotic-analysis in distributed block source coding involving potentially an infinite number of infinitesimal-rate messages. In this paper, the infinite-message sum-rate-distortion function, viewed as a functional of the joint source pmf and the distortion levels, is characterized as the least element of a partially ordered family of functionals having certain convex-geometric properties. The new characterization does not involve evaluating the infinite-message limit of a finite-message sum-rate-distortion expression. This characterization leads to a family of lower bounds for the infinite-message sum-rate-distortion expression and a simple criterion to test the optimality of any achievable infinite-message sum-rate-distortion expression. For computing the amplewise Boolean AND function, the infinite-message minimum sum-rates are characterized in closed analytic form. These sum-rates are shown to be achievable using infinitely many infinitesimal-rate messages. The new convex-geometric characterization is used to develop an iterative algorithm for evaluating any finite-message sumrate-distortion function. It is also used to construct the first examples which demonstrate that for lossy source reproduction, two messages can strictly improve the one-message Wyner-Ziv rate-distortion function settling an unresolved question from a 1985 paper.
The three-node multiterminal lossy source coding problem is investigated. We derive an inner bound to the general rate-distortion region of this problem which is a natural extension of the seminal work by Kaspi'85 on the interactive two-terminal source coding problem. It is shown that this (rather involved) inner bound contains several rate-distortion regions of some relevant source coding settings. In this way, besides the non-trivial extension of the interactive two terminal problem, our results can be seen as a generalization and hence unification of several previous works in the field. Spe…
An interactive error correcting code ($\mathsf{iECC}$) is an interactive protocol with the guarantee that the receiver can correctly determine the sender's message, even in the presence of noise. This generalizes the concept of an error correcting code ($\mathsf{ECC}$), which is a non-interactive $\mathsf{iECC}$ that is known to have erasure resilience capped at $\frac12$. The work of \cite{GuptaTZ21} constructed the first $\mathsf{iECC}$ resilient to $> \frac12$ adversarial erasures. However, their $\mathsf{iECC}$ has communication complexity quadratic in the message size. In our work, we con…
This paper introduces equal-image-size source partitioning, a new tool for analyzing channel and joint source-channel coding in a multi-terminal discrete memoryless channel environment. Equal-image-size source partitioning divides the source (combination of messages and codewords) into a sub-exponential number of subsets. Over each of these subsets, the exponential orders of the minimum image sizes of most messages are roughly equal to the same entropy term. This property gives us the strength of minimum image sizes and the flexibility of entropy terms. Using the method of equal-image-size sou…
We study the communication rate of coding schemes for interactive communication that transform any two-party interactive protocol into a protocol that is robust to noise. Recently, Haeupler (FOCS '14) showed that if an $ε> 0$ fraction of transmissions are corrupted, adversarially or randomly, then it is possible to achieve a communication rate of $1 - \widetilde{O}(\sqrtε)$. Furthermore, Haeupler conjectured that this rate is optimal for general input protocols. This stands in contrast to the classical setting of one-way communication in which error-correcting codes are known to achieve an opt…
We consider a non-asymptotic (one-shot) version of the multiterminal secret key agreement problem on a finite linear source model. In this model, the observation of each terminal is a linear function of an underlying random vector composed of finitely many i.i.d. uniform random variables. Restricting the public discussion to be a linear function of the terminals' observations, we obtain a characterization of the communication complexity (minimum number of symbols of public discussion) of generating a secret key of maximum length. The minimum discussion is achieved by a non-interactive protocol…
A unified graphical approach to random coding for any memoryless, single-hop, K-user channel with or without common information is defined through two steps. The first step is user virtualization: each user is divided into multiple virtual sub-users according to a chosen rate-splitting strategy. This results in an enhanced channel with a possibly larger number of users for which more coding possibilities are available and for which common messages to any subset of users can be encoded. Following user virtualization, the message of each user in the enhanced model is coded using a chosen combina…
We consider interactive computation of randomized functions between two users with the following privacy requirement: the interaction should not reveal to either user any extra information about the other user's input and output other than what can be inferred from the user's own input and output. We also consider the case where privacy is required against only one of the users. For both cases, we give single-letter expressions for feasibility and optimal rates of communication. Then we discuss the role of common randomness and interaction in both privacy settings. We also study perfectly secu…
We study the rate region of variable-length source-network codes that are used to compute a function of messages observed over a network. The particular network considered here is the simplest instance of a directed acyclic graph (DAG) that is not a tree. Existing work on zero-error function computation in DAG networks provides bounds on the \textit{computation capacity}, which is a measure of the amount of communication required per edge in the worst case. This work focuses on the average case: an achievable rate tuple describes the expected amount of communication required on each edge, wher…
A basic two-terminal secret key generation model is considered, where the interactive communication rate between the terminals may be limited, and in particular may not be enough to achieve the maximum key rate. We first prove a multi-letter characterization of the key-communication rate region (where the number of auxiliary random variables depend on the number of rounds of the communication), and then provide an equivalent but simpler characterization in terms of concave envelopes in the case of unlimited number of rounds. Two extreme cases are given special attention. First, in the regime o…
Lattices possess elegant mathematical properties which have been previously used in the literature to show that structured codes can be efficient in a variety of communication scenarios, including coding for the additive white Gaussian noise (AWGN) channel, dirty-paper channel, Wyner-Ziv coding, coding for relay networks and so forth. We consider the family of single-transmitter multiple-receiver Gaussian channels where the source transmits a set of common messages to all the receivers (multicast scenario), and each receiver has 'coded side information', i.e., prior information in the form of …
We study the degrees of freedom (DoF) regions of the two-user multiple-input multiple-output (MIMO) broadcast channel with a general message set (BC-CM) - that includes private and common messages - under fast fading. Nine different channel state knowledge assumptions -collectively known as hybrid CSIT models - are considered wherein the transmitter has either perfect/instantaneous (P), delayed (D) or no (N) channel state information (CSI) from each of the two receivers. General antenna configurations are addressed wherein the three terminals have arbitrary numbers of antennas. The DoF regions…
This paper aims to characterize the capacity limits of a wireless cellular network with a full-duplex (FD) base-station (BS) and half-duplex user terminals, in which three independent messages are communicated: the uplink message $m_1$ from the uplink user to the BS, the downlink message $m_2$ from the BS to the downlink user, and the device-to-device (D2D) message $m_3$ from the uplink user to the downlink user. From an information theoretical perspective, the overall network can be viewed as a generalization of the FD relay broadcast channel with a side message transmitted from the relay to …
We present experimental data on message transmission in a free-space optical (FSO) link at an eye-safe wavelength, using a testbed consisting of one sender and two receiver terminals, where the latter two are a legitimate receiver and an eavesdropper. The testbed allows us to emulate a typical scenario of physical-layer (PHY) security such as satellite-to-ground laser communications. We estimate information-theoretic metrics including secrecy rate, secrecy outage probability, and expected code lengths for given secrecy criteria based on observed channel statistics. We then discuss operation pr…
A rate splitting based scheme is proposed to operate a broadcast setting with two antennas at the transmit side and two single-antenna receiving terminals. The transmitter knows the magnitude of the channel coefficients, and it is oblivious to the phase information. Each transmit antenna, with a power constraint, sends a private message and a common message to be decoded by both receivers. An achievable rate region is obtained, which enlarges the capacity region of the vector broadcast channel with vector channel magnitude feedback by means of superposition coding.
We construct and analyze the communication cost of protocols (interactive and one-way) for classifying ${\mathbf X}=(X_1,X_2,\ldots,X_n) \in [0,1)^n \subset \mathbb{R}^n$, in a network with $n\geq 2$ nodes, with $X_i$ known only at node $i$. The classifier takes the form $\sum_{i=1}^nh_iX_i \gtrless a$, with weights $h_i \in \{-1,+1\}$. The interactive protocol (a zero-error protocol) exchanges a variable number of messages depending on the input ${\mathbf X}$ and its sum rate is directly proportional to its mean stopping time. An exact analysis, as well as an approximation of the mean stoppin…
For a multi-source multi-terminal noiseless network, the key-dissemination problem involves the task of multicasting a secret key K from the network sources to its terminals. As in secure multicast network-coding, in the key-dissemination problem the source nodes have access to independent randomness and, as the network is noiseless, the resulting key K is a function of the sources' information. However, different from traditional forms of multicast, in key-dissemination the key K need not consist of source messages, but rather may be any function of the information generated at the sources, a…
Given a noiseless protocol $π_0$ computing a function $f(x, y)$ of Alice and Bob's private inputs $x, y$, the goal of interactive coding is to construct an error-resilient protocol $π$ computing $f$ such that even if some fraction of the communication is adversarially corrupted, both parties still learn $f(x, y)$. Ideally, the resulting scheme $π$ should be positive rate, computationally efficient, and achieve optimal error resilience. While interactive coding over large alphabets is well understood, the situation over the binary alphabet has remained evasive. At the present moment, the known …
In two-way channels, each user transmits and receives at the same time. This allows each encoder to interactively adapt the current input to its own message and all previously received signals. Such coding approach can introduce correlation between inputs of different users, since all the users' outputs are correlated by the nature of the channel. However, for some channels, such adaptation in the coding scheme and its induced correlation among users are useless in the sense that they do not help enlarge the capacity region with respect to the standard coding method (where each user encodes on…
We consider the problem of implementing two-party interactive quantum communication over noisy channels, a necessary endeavor if we wish to fully reap quantum advantages for communication. For an arbitrary protocol with $n$ messages, designed for a noiseless qudit channel over a $\mathrm{poly}(n)$ size alphabet, our main result is a simulation method that fails with probability less than $2^{-Θ(nε)}$ and uses a qudit channel over the same alphabet $n\left(1+Θ\left(\sqrtε\right)\right)$ times, of which an $ε$ fraction can be corrupted adversarially. The simulation is thus capacity achieving to …
