We consider a network coding problem where the destination wants to recover the sum of the signals (Gaussian random variables or random finite field elements) at all the source nodes, but the sum must be kept secret from an eavesdropper that can wiretap on a subset of edges. This setting arises naturally in sensor networks and federated learning, where the secrecy of the sum of the signals (e.g. weights, gradients) may be desired. While the case for finite field can be solved, the case for Gaussian random variables is surprisingly difficult. We give a simple conjecture on the necessary and sufficient condition under which such secret computation is possible for the Gaussian case, and prove the conjecture when the number of wiretapped edges is at most 2.
Practical implementations of secret-key generation are often based on sequential strategies, which handle reliability and secrecy in two successive steps, called reconciliation and privacy amplification. In this paper, we propose an alternative approach based on polar codes that jointly deals with reliability and secrecy. Specifically, we propose secret-key capacity-achieving polar coding schemes for the following models: (i) the degraded binary memoryless source (DBMS) model with rate-unlimited public communication, (ii) the DBMS model with one-way rate-limited public communication, (iii) the…
This paper presents a random coding scheme with which two nodes can exchange information with guaranteed integrity over a two-way Byzantine relay. This coding scheme is employed to obtain an inner bound on the capacity region with guaranteed information integrity. No pre-shared secret or secret transmission is needed for the proposed scheme. Hence the inner bound obtained is generally larger than those achieved based on secret transmission schemes. This approach advocates the separation of supporting information integrity and secrecy.
In this article, we study the problem of secret key generation in the multiterminal source model, where the terminals have access to correlated Gaussian sources. We assume that the sources form a Markov chain on a tree. We give a nested lattice-based key generation scheme whose computational complexity is polynomial in the number, N , of independent and identically distributed samples observed by each source. We also compute the achievable secret key rate and give a class of examples where our scheme is optimal in the fine quantization limit. However, we also give examples that show that our s…
We consider secret sharing schemes with a classical secret and quantum shares. One example of such schemes was recently reported whose access structure cannot be realized by any secret sharing schemes with classical shares. In this paper, we report further quantum secret sharing schemes whose access structures cannot be realized by any classical secret sharing schemes.
Evolving secret sharing schemes do not require prior knowledge of the number of parties $n$ and $n$ may be infinitely countable. It is known that the evolving $2$-threshold secret sharing scheme and prefix coding of integers have a one-to-one correspondence. However, it is not known what prefix coding of integers to use to construct the scheme better. In this paper, we propose a new metric $K_Σ$ for evolving $2$-threshold secret sharing schemes $Σ$. We prove that the metric $K_Σ\geq 1.5$ and construct a new prefix coding of integers, termed $λ$ code, to achieve the metric $K_Λ=1.59375$. Thus, …
Distributed arithmetic coding (DAC) has been shown to be effective for Slepian-Wolf coding, especially for short data blocks. In this letter, we propose to use the DAC to compress momery-correlated sources. More specifically, the correlation between sources is modeled as a hidden Markov process. Experimental results show that the performance is close to the theoretical Slepian-Wolf limit.
We introduce explicit schemes based on the polarization phenomenon for the tasks of one-way secret key agreement from common randomness and private channel coding. For the former task, we show how to use common randomness and insecure one-way communication to obtain a strongly secure key such that the key construction has a complexity essentially linear in the blocklength and the rate at which the key is produced is optimal, i.e., equal to the one-way secret-key rate. For the latter task, we present a private channel coding scheme that achieves the secrecy capacity using the condition of stron…
Network coding is a new technique to transmit data through a network by letting the intermediate nodes combine the packets they receive. Given a network, the network coding solvability problem decides whether all the packets requested by the destinations can be transmitted. In this paper, we introduce a new approach to this problem. We define a closure operator on a digraph closely related to the network coding instance and we show that the constraints for network coding can all be expressed according to that closure operator. Thus, a solution for the network coding problem is equivalent to a …
Distributed source coding schemes are typically based on the use of channels codes as source codes. In this paper we propose a new paradigm, termed "distributed arithmetic coding", which exploits the fact that arithmetic codes are good source as well as channel codes. In particular, we propose a distributed binary arithmetic coder for Slepian-Wolf coding with decoder side information, along with a soft joint decoder. The proposed scheme provides several advantages over existing Slepian-Wolf coders, especially its good performance at small block lengths, and the ability to incorporate arbitrary…
In this paper we give a short theoretical description of the general predictive adaptive arithmetic coding technique. The links between this technique and the works of J. Rissanen in the 80's, in particular the BIC information criterion used in parametrical model selection problems, are established. We also design lossless and lossy coding techniques of images. The lossless technique uses a mix between fixed-length coding and arithmetic coding and provides better compression results than those separate methods. That technique is also seen to have an interesting application in the domain of sta…
We show a construction of a quantum ramp secret sharing scheme from a nested pair of linear codes. Necessary and sufficient conditions for qualified sets and forbidden sets are given in terms of combinatorial properties of nested linear codes. An algebraic geometric construction for quantum secret sharing is also given.
A distribution matcher (DM) encodes a binary input data sequence into a sequence of symbols (codeword) with desired target probability distribution. The set of the output codewords constitutes a codebook (or code) of a DM. Constant-composition DM (CCDM) uses arithmetic coding to efficiently encode data into codewords from a constant-composition (CC) codebook. The CC constraint limits the size of the codebook, and hence the coding rate of the CCDM. The performance of CCDM degrades with decreasing output length. To improve the performance for short transmission blocks we present a class of multi…
The modern data compression is mainly based on two approaches to entropy coding: Huffman (HC) and arithmetic/range coding (AC). The former is much faster, but approximates probabilities with powers of 2, usually leading to relatively low compression rates. The latter uses nearly exact probabilities - easily approaching theoretical compression rate limit (Shannon entropy), but at cost of much larger computational cost. Asymmetric numeral systems (ANS) is a new approach to accurate entropy coding, which allows to end this trade-off between speed and rate: the recent implementation [1] provides a…
Secure communication with feedback is studied. An achievability scheme in which the backward channel is used to generate a shared secret key is proposed. The scenario of binary symmetric forward and backward channels is considered, and a combination of the proposed scheme and Maurer's coding scheme is shown to achieve improved secrecy rates. The scenario of a Gaussian channel with perfect output feedback is also analyzed and the Schalkwijk-Kailath coding scheme is shown to achieve the secrecy capacity for this channel.
A distribution matcher (DM) encodes a binary input data sequence into a sequence of symbols with a desired target probability distribution. Several DMs, including shell mapping and constant-composition distribution matcher (CCDM), have been successfully employed for signal shaping, e.g., in optical-fiber or 5G. The CCDM, like many other DMs, is typically implemented by arithmetic coding (AC). In this work we implement AC based DMs using finite-precision arithmetic (FPA). An analysis of the implementation shows that FPA results in a rate-loss that shrinks exponentially with the number of precis…
The constrained linear representability problem (CLRP) for polymatroids determines whether there exists a polymatroid that is linear over a specified field while satisfying a collection of constraints on the rank function. Using a computer to test whether a certain rate vector is achievable with vector linear network codes for a multi-source network coding instance and whether there exists a multi-linear secret sharing scheme achieving a specified information ratio for a given secret sharing instance are shown to be special cases of CLRP. Methods for solving CLRP built from group theoretic tec…
The polarization phenomenon for a single source is extended to a framework with multiple correlated sources. It is shown in addition to extracting the randomness of the source, the polar transforms takes the original arbitrary dependencies to extremal dependencies. Polar coding schemes for the Slepian-Wolf problem and for secret key generations are then proposed based on this phenomenon. In particular, constructions of secret keys achieving the secrecy capacity and compression schemes achieving the Slepian-Wolf capacity region are obtained with a complexity of $O(n \log (n))$.
This introduction to arithmetic coding is divided in two parts. The first explains how and why arithmetic coding works. We start presenting it in very general terms, so that its simplicity is not lost under layers of implementation details. Next, we show some of its basic properties, which are later used in the computational techniques required for a practical implementation. In the second part, we cover the practical implementation aspects, including arithmetic operations with low precision, the subdivision of coding and modeling, and the realization of adaptive encoders. We also analyze the …
This paper considers rateless network error correction codes for reliable multicast in the presence of adversarial errors. Most existing network error correction codes are designed for a given network capacity and maximum number of errors known a priori to the encoder and decoder. However, in certain practical settings it may be necessary to operate without such a priori knowledge. We present rateless coding schemes for two adversarial models, where the source sends more redundancy over time, until decoding succeeds. The first model assumes there is a secret channel between the source and the …
We address the problem of delay in an arithmetic coding system. Due to the nature of the arithmetic coding process, source sequences causing arbitrarily large encoding or decoding delays exist. This phenomena raises the question of just how large is the expected input to output delay in these systems, i.e., once a source sequence has been encoded, what is the expected number of source letters that should be further encoded to allow full decoding of that sequence. In this paper, we derive several new upper bounds on the expected delay for a memoryless source, which improve upon a known bound du…
The aim of this paper is to prove coding theorems for the wiretap channel coding problem and secret key agreement problem based on the the notion of a hash property for an ensemble of functions. These theorems imply that codes using sparse matrices can achieve the optimal rate. Furthermore, fixed-rate universal coding theorems for a wiretap channel and a secret key agreement are also proved.
Differential pulse-code modulation (DPCM) is recently coupled with uniform scalar quantization (SQ) to improve the rate-distortion (RD) performance for the block-based quantized compressive sensing (CS) of images. In this framework, for each block's CS measurements, a prediction is generated based on the reconstructed CS measurements of the previous blocks and subtracted from measurements of the current block in the measurement domain. The resulting residual is then quantized by uniform SQ to generate the quantization index. However, the entropy coding is still required to remove the statistic…
In time entanglement-based quantum key distribution (QKD), Alice and Bob extract the raw key bits from the (identical) arrival times of entangled photon pairs by time-binning. Each of them individually discretizes time into bins and groups them into frames. They retain only the frames with a single occupied bin. Thus, Alice and Bob can use the position of the occupied bin within a frame to generate random key bits, as in PPM modulation. Because of entanglement, their occupied bins and their keys should be identical. However, practical photon detectors suffer from time jitter errors. These erro…
Compression is a technique to reduce the quantity of data without excessively reducing the quality of the multimedia data. The transition and storing of compressed multimedia data is much faster and more efficient than original uncompressed multimedia data. There are various techniques and standards for multimedia data compression, especially for image compression such as the JPEG and JPEG2000 standards. These standards consist of different functions such as color space conversion and entropy coding. Arithmetic and Huffman coding are normally used in the entropy coding phase. In this paper we …
Constrained codes are used to prevent errors from occurring in various data storage and data transmission systems. They can help in increasing the storage density of magnetic storage devices, in managing the lifetime of electronic storage devices, and in increasing the reliability of data transmission over wires. We recently introduced families of lexicographically-ordered constrained (LOCO) codes. These codes achieve capacity with simple encoding and decoding, and they are easy to reconfigure. In this paper, we generalize our work on LOCO codes by presenting a systematic method that guides th…
The secure multiplex coding (SMC) is a technique to remove rate loss in the coding for wire-tap channels and broadcast channels with confidential messages caused by the inclusion of random bits into transmitted signals. SMC replaces the random bits by other meaningful secret messages, and a collection of secret messages serves as the random bits to hide the rest of messages. In the previous researches, multiple secret messages were assumed to have independent and uniform distributions, which is difficult to be ensured in practice. We remove this restrictive assumption by a generalization of th…
We consider the problem of revealing/sharing data in an efficient and secure way via a compact representation. The representation should ensure reliable reconstruction of the desired features/attributes while still preserve privacy of the secret parts of the data. The problem is formulated as a remote lossy source coding with a privacy constraint where the remote source consists of public and secret parts. Inner and outer bounds for the optimal tradeoff region of compression rate, distortion, and privacy leakage rate are given and shown to coincide for some special cases. When specializing the…
We show strongly secret achievable rate regions for two different wiretap multiple-access channel coding problems. In the first problem, each encoder has a private message and both together have a common message to transmit. The encoders have entropy-limited access to common randomness. If no common randomness is available, then the achievable region derived here does not allow for the secret transmission of a common message. The second coding problem assumes that the encoders do not have a common message nor access to common randomness. However, they may have a conferencing link over which th…
Quantized Indexing is a fast and space-efficient form of enumerative (combinatorial) coding, the strongest among asymptotically optimal universal entropy coding algorithms. The present advance in enumerative coding is similar to that made by arithmetic coding with respect to its unlimited precision predecessor, Elias coding. The arithmetic precision, execution time, table sizes and coding delay are all reduced by a factor O(n) at a redundancy below 2*log(e)/2^g bits/symbol (for n input symbols and g-bit QI precision). Due to its tighter enumeration, QI output redundancy is below that of arithm…
Arithmetic Coding (AC) is widely used for the entropy coding of text and video data. It involves recursive partitioning of the range [0,1) in accordance with the relative probabilities of occurrence of the input symbols. A data (image or video) encryption scheme based on arithmetic coding called as Chaotic Arithmetic Coding (CAC) has been presented in previous works. In CAC, a large number of chaotic maps can be used to perform coding, each achieving Shannon optimal compression performance. The exact choice of map is governed by a key. CAC has the effect of scrambling the intervals without mak…
We give an overview of several of the mathematical works of Gilles Lachaud and provide a historical context. This is interspersed with some personal anecdotes highlighting many facets of his personality.
We study secure source-coding with causal disclosure, under the Gaussian distribution. The optimality of Gaussian auxiliary random variables is shown in various scenarios. We explicitly characterize the tradeoff between the rates of communication and secret key. This tradeoff is the result of a mutual information optimization under Markov constraints. As a corollary, we deduce a general formula for Wyner's Common Information in the Gaussian setting.
Recent years have seen renewed attention to arithmetic coding (AC). This is thanks to the use of AC for distribution matching (DM) to control the channel input distribution in probabilistic amplitude shaping. There are two main problems inherent to AC: (1) its required arithmetic precision grows linearly with the input length, and (2) high-precision multiplications and divisions are required. Here, we introduce a multiplication-free AC-based DM technique via three lookup tables (LUTs) which solves both problems above. These LUTs are used to approximate the high-precision multiplications and di…
This paper proposes a novel entropy encoding technique for lossless data compression. Representing a message string by its lexicographic index in the permutations of its symbols results in a compressed version matching Shannon entropy of the message. Commercial data compression standards make use of Huffman or arithmetic coding at some stage of the compression process. In the proposed method, like arithmetic coding entire string is mapped to an integer but is not based on fractional numbers. Unlike both arithmetic and Huffman coding no prior entropy model of the source is required. Simple intu…
A coding scheme based on irregular low-density parity-check (LDPC) codes is proposed to send secret messages from a source over the Gaussian wiretap channel to a destination in the presence of a wiretapper, with the restriction that the source can send only binary phase-shift keyed (BPSK) symbols. The secrecy performance of the proposed coding scheme is measured by the secret message rate through the wiretap channel as well as the equivocation rate about the message at the wiretapper. A code search procedure is suggested to obtain irregular LDPC codes that achieve good secrecy performance in s…
Recently there has been a lot of success in using the deterministic approach to provide approximate characterization of Gaussian network capacity. In this paper, we take a deterministic view and revisit the problem of wiretap channel with side information. A precise characterization of the secrecy capacity is obtained for a linear deterministic model, which naturally suggests a coding scheme which we show to achieve the secrecy capacity of the degraded Gaussian model (dubbed as "secret writing on dirty paper") to within half a bit.
This paper considers the problem of secret communication over a two-receiver multiple-input multiple-output (MIMO) Gaussian broadcast channel. The transmitter has two independent, confidential messages and a common message. Each of the confidential messages is intended for one of the receivers but needs to be kept perfectly secret from the other, and the common message is intended for both receivers. It is shown that a natural scheme that combines secret dirty-paper coding with Gaussian superposition coding achieves the secrecy capacity region. To prove this result, a channel-enhancement appro…
For a multi-source multi-terminal noiseless network, the key-dissemination problem involves the task of multicasting a secret key K from the network sources to its terminals. As in secure multicast network-coding, in the key-dissemination problem the source nodes have access to independent randomness and, as the network is noiseless, the resulting key K is a function of the sources' information. However, different from traditional forms of multicast, in key-dissemination the key K need not consist of source messages, but rather may be any function of the information generated at the sources, a…
Csiszar and Narayan[3] defined the notion of secret key capacity for multiple terminals, characterized it as a linear program with Slepian-Wolf constraints of the related source coding problem of communication for omniscience, and upper bounded it by some information divergence expression from the joint to the product distribution of the private observations. This paper proves that the bound is tight for the important case when all users are active, using the polymatroidal structure[6] underlying the source coding problem. When some users are not active, the bound may not be tight. This paper …
This paper considers secret-key generation between several agents and a base station that observe independent and identically distributed realizations of correlated random variables. Each agent wishes to generate the longest possible individual key with the base station by means of public communication. All keys must be jointly kept secret from all external entities. In this many-to-one secret-key generation setting, it can be shown that the agents can take advantage of a collective protocol to increase the sum-rate of their generated keys. However, when each agent is only interested in maximi…
We present DeepCABAC, a novel context-adaptive binary arithmetic coder for compressing deep neural networks. It quantizes each weight parameter by minimizing a weighted rate-distortion function, which implicitly takes the impact of quantization on to the accuracy of the network into account. Subsequently, it compresses the quantized values into a bitstream representation with minimal redundancies. We show that DeepCABAC is able to reach very high compression ratios across a wide set of different network architectures and datasets. For instance, we are able to compress by x63.6 the VGG16 ImageN…
We investigate the transmission of a secret message from Alice to Bob in the presence of an eavesdropper (Eve) and many of decode-and-forward relay nodes. Each link comprises a set of parallel channels, modeling for example an orthogonal frequency division multiplexing transmission. We consider the impact of discrete constellations and finite-length coding, defining an achievable secrecy rate under a constraint on the equivocation rate at Eve. Then we propose a power and channel allocation algorithm that maximizes the achievable secrecy rate by resorting to two coupled Gale-Shapley algorithms …
We address security and privacy problems for digital devices and biometrics from an information-theoretic optimality perspective, where a secret key is generated for authentication, identification, message encryption/decryption, or secure computations. A physical unclonable function (PUF) is a promising solution for local security in digital devices and this review gives the most relevant summary for information theorists, coding theorists, and signal processing community members who are interested in optimal PUF constructions. Low-complexity signal processing methods such as transform coding …
In the secure network coding for multicasting, there is loss of information rate due to inclusion of random bits at the source node. We show a method to eliminate that loss of information rate by using multiple statistically independent messages to be kept secret from an eavesdropper. The proposed scheme is an adaptation of Yamamoto et al.'s secure multiplex coding to the secure network coding.
In wireless data networks, communication is particularly susceptible to eavesdropping due to its broadcast nature. Security and privacy systems have become critical for wireless providers and enterprise networks. This paper considers the problem of secret communication over the Gaussian broadcast channel, where a multi-antenna transmitter sends independent confidential messages to two users with information-theoretic secrecy. That is, each user would like to obtain its own confidential message in a reliable and safe manner. This communication model is referred to as the multi-antenna Gaussian …
We develop a low-complexity polar coding scheme for the discrete memoryless broadcast channel with confidential messages under strong secrecy and randomness constraints. Our scheme extends previous work by using an optimal rate of uniform randomness in the stochastic encoder, and avoiding assumptions regarding the symmetry or degraded nature of the channels. The price paid for these extensions is that the encoder and decoders are required to share a secret seed of negligible size and to increase the block length through chaining. We also highlight a close conceptual connection between the prop…
Function computation of arbitrarily correlated discrete sources over Gaussian networks with orthogonal components is studied. Two classes of functions are considered: the arithmetic sum function and the type function. The arithmetic sum function in this paper is defined as a set of multiple weighted arithmetic sums, which includes averaging of the sources and estimating each of the sources as special cases. The type or frequency histogram function counts the number of occurrences of each argument, which yields many important statistics such as mean, variance, maximum, minimum, median, and so o…
