The distributed source coding problem is considered when the sensors, or encoders, are under Byzantine attack; that is, an unknown number of sensors have been reprogrammed by a malicious intruder to undermine the reconstruction at the fusion center. Three different forms of the problem are considered. The first is a variable-rate setup, in which the decoder adaptively chooses the rates at which the sensors transmit. An explicit characterization of the variable-rate minimum achievable sum rate is stated, given by the maximum entropy over the set of distributions indistinguishable from the true source distribution by the decoder. In addition, two forms of the fixed-rate problem are considered, one with deterministic coding and one with randomized coding. The achievable rate regions are given for both these problems, with a larger region achievable using randomized coding, though both are suboptimal compared to variable-rate coding.
This paper presents a random coding scheme with which two nodes can exchange information with guaranteed integrity over a two-way Byzantine relay. This coding scheme is employed to obtain an inner bound on the capacity region with guaranteed information integrity. No pre-shared secret or secret transmission is needed for the proposed scheme. Hence the inner bound obtained is generally larger than those achieved based on secret transmission schemes. This approach advocates the separation of supporting information integrity and secrecy.
The problem of cooperative fusion in the presence of Byzantine sensors is considered. An information theoretic formulation is used to characterize the Shannon capacity of sensor fusion. It is shown that when less than half of the sensors are Byzantine, the effect of Byzantine attack can be entirely mitigated, and the fusion capacity is identical to that when all sensors are honest. But when at least half of the sensors are Byzantine, they can completely defeat the sensor fusion so that no information can be transmitted reliably. A capacity achieving transmit-then-verify strategy is proposed fo…
Network coding increases throughput and is robust against failures and erasures. However, since it allows mixing of information within the network, a single corrupted packet generated by a Byzantine attacker can easily contaminate the information to multiple destinations. In this paper, we study the transmission overhead associated with three different schemes for detecting Byzantine adversaries at a node using network coding: end-to-end error correction, packet-based Byzantine detection scheme, and generation-based Byzantine detection scheme. In end-to-end error correction, it is known that w…
The distributed source coding problem is considered when the sensors, or encoders, are under Byzantine attack; that is, an unknown group of sensors have been reprogrammed by a malicious intruder to undermine the reconstruction at the fusion center. Three different forms of the problem are considered. The first is a variable-rate setup, in which the decoder adaptively chooses the rates at which the sensors transmit. An explicit characterization of the variable-rate achievable sum rates is given for any number of sensors and any groups of traitors. The converse is proved constructively by lettin…
We investigate the impact of Byzantine attacks in distributed detection under binary hypothesis testing. It is assumed that a fraction of the transmitted sensor measurements are compromised by the injected data from a Byzantine attacker, whose purpose is to confuse the decision maker at the fusion center. From the perspective of a Byzantine attacker, under the injection energy constraint, an optimization problem is formulated to maximize the asymptotic missed detection error probability, which is based on the Kullback-Leibler divergence. The properties of the optimal attack strategy are analyz…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Recent advances in large-scale distributed learning algorithms have enabled communication-efficient training via SignSGD. Unfortunately, a major issue continues to plague distributed learning: namely, Byzantine failures may incur serious degradation in learning accuracy. This paper proposes Election Coding, a coding-theoretic framework to guarantee Byzantine-robustness for SignSGD with Majority Vote, which uses minimum worker-master communication in both directions. The suggested framework explores new information-theoretic limits of finding the majority opinion when some workers could be mali…
This paper considers rateless network error correction codes for reliable multicast in the presence of adversarial errors. Most existing network error correction codes are designed for a given network capacity and maximum number of errors known a priori to the encoder and decoder. However, in certain practical settings it may be necessary to operate without such a priori knowledge. We present rateless coding schemes for two adversarial models, where the source sends more redundancy over time, until decoding succeeds. The first model assumes there is a secret channel between the source and the …
We consider a Gaussian two-hop network where the source and the destination can communicate only via a relay node who is both an eavesdropper and a Byzantine adversary. Both the source and the destination nodes are allowed to transmit, and the relay receives a superposition of their transmitted signals. We propose a new coding scheme that satisfies two requirements simultaneously: the transmitted message must be kept secret from the relay node, and the destination must be able to detect any Byzantine attack that the relay node might launch reliably and fast. The three main components of the sc…
In the multisensor sequential change detection problem, a disruption occurs in an environment monitored by multiple sensors. This disruption induces a change in the observations of an unknown subset of sensors. In the Byzantine version of this problem, which is the focus of this work, it is further assumed that the postulated change-point model may be misspecified for an unknown subset of sensors. The problem then is to detect the change quickly and reliably, for any possible subset of affected sensors, even if the misspecified sensors are controlled by an adversary. Given a user-specified upp…
The problem of distributed inference with M-ary quantized data at the sensors is investigated in the presence of Byzantine attacks. We assume that the attacker does not have knowledge about either the true state of the phenomenon of interest, or the quantization thresholds used at the sensors. Therefore, the Byzantine nodes attack the inference network by modifying modifying the symbol corresponding to the quantized data to one of the other M symbols in the quantization alphabet-set and transmitting the false symbol to the fusion center (FC). In this paper, we find the optimal Byzantine attack…
This paper studies binary hypothesis testing based on measurements from a set of sensors, a subset of which can be compromised by an attacker. The measurements from a compromised sensor can be manipulated arbitrarily by the adversary. The asymptotic exponential rate, with which the probability of error goes to zero, is adopted to indicate the detection performance of a detector. In practice, we expect the attack on sensors to be sporadic, and therefore the system may operate with all the sensors being benign for extended period of time. This motivates us to consider the trade-off between the d…
In this paper, we consider the problem of distributed Bayesian detection in the presence of Byzantines in the network. It is assumed that a fraction of the nodes in the network are compromised and reprogrammed by an adversary to transmit false information to the fusion center (FC) to degrade detection performance. The problem of distributed detection is formulated as a binary hypothesis test at the FC based on 1-bit data sent by the sensors. The expression for minimum attacking power required by the Byzantines to blind the FC is obtained. More specifically, we show that above a certain fractio…
We investigate the testing-against-independence problem \mw{over a cooperative MAC} with two sensors and a single detector under an average rate constraint on the sensors-detector links. For this setup, we design a variable-length coding scheme that maximizes the achievable type-II error exponent when the type-I error probability is limited to $ε$. Similarly to the single-link result, we show here that the optimal error exponent depends on $ε$ and that variable-length coding allows to \mw{increase the rates over the optimal} fixed-length coding \mw{scheme} by the factor $(1-ε)^{-1}$.
We consider the distributed source coding problem in which correlated data picked up by scattered sensors has to be encoded separately and transmitted to a common receiver, subject to a rate-distortion constraint. Although near-tooptimal solutions based on Turbo and LDPC codes exist for this problem, in most cases the proposed techniques do not scale to networks of hundreds of sensors. We present a scalable solution based on the following key elements: (a) distortion-optimized index assignments for low-complexity distributed quantization, (b) source-optimized hierarchical clustering based on t…
We study the problem of the reconstruction of a Gaussian field defined in [0,1] using N sensors deployed at regular intervals. The goal is to quantify the total data rate required for the reconstruction of the field with a given mean square distortion. We consider a class of two-stage mechanisms which a) send information to allow the reconstruction of the sensor's samples within sufficient accuracy, and then b) use these reconstructions to estimate the entire field. To implement the first stage, the heavy correlation between the sensor samples suggests the use of distributed coding schemes to …
In a recent paper, Jaggi et al. (INFOCOM 2007), presented a distributed polynomial-time rate-optimal network-coding scheme that works in the presence of Byzantine faults. We revisit their adversarial models and augment them with three, arguably realistic, models. In each of the models, we present a distributed scheme that demonstrates the usefulness of the model. In particular, all of the schemes obtain optimal rate $C-z$, where $C$ is the network capacity and $z$ is a bound on the number of links controlled by the adversary.
The Byzantine distributed quickest change detection (BDQCD) is studied, where a fusion center monitors the occurrence of an abrupt event through a bunch of distributed sensors that may be compromised. We first consider the binary hypothesis case where there is only one post-change hypothesis and prove a novel converse to the first-order asymptotic detection delay in the large mean time to a false alarm regime. This converse is tight in that it coincides with the currently best achievability shown by Fellouris et al.; hence, the optimal asymptotic performance of binary BDQCD is characterized. A…
In this paper, we consider the problem of remote vector Gaussian source coding for a wireless acoustic sensor network. Each node receives messages from multiple nodes in the network and decodes these messages using its own measurement of the sound field as side information. The node's measurement and the estimates of the source resulting from decoding the received messages are then jointly encoded and transmitted to a neighboring node in the network. We show that for this distributed source coding scenario, one can encode a so-called conditional sufficient statistic of the sources instead of j…
This paper formulates and studies a general distributed field reconstruction problem using a dense network of noisy one-bit randomized scalar quantizers in the presence of additive observation noise of unknown distribution. A constructive quantization, coding, and field reconstruction scheme is developed and an upper-bound to the associated mean squared error (MSE) at any point and any snapshot is derived in terms of the local spatio-temporal smoothness properties of the underlying field. It is shown that when the noise, sensor placement pattern, and the sensor schedule satisfy certain weak te…
We consider a single-hop data gathering sensor cluster consisting of a set of sensors that need to transmit data periodically to a base-station. We are interested in maximizing the lifetime of this network. Even though the setting of our problem is very simple, it turns out that the solution is far from easy. The complexity arises from several competing system-level opportunities available to reduce the energy consumed in radio transmission. First, sensor data is spatially and temporally correlated. Recent advances in distributed source-coding allow us to take advantage of these correlations t…
Systems that employ network coding for content distribution convey to the receivers linear combinations of the source packets. If we assume randomized network coding, during this process the network nodes collect random subspaces of the space spanned by the source packets. We establish several fundamental properties of the random subspaces induced in such a system, and show that these subspaces implicitly carry topological information about the network and its state that can be passively collected and inferred. We leverage this information towards a number of applications that are interesting …
We consider the situation in which a continuous-time vector Gauss-Markov process is observed through a vector Gaussian channel (sensor) and estimated by the Kalman-Bucy filter. Unlike in standard filtering problems where a sensor model is given a priori, we are concerned with the optimal sensor design by which (i) the mutual information between the source random process and the reproduction (estimation) process is minimized, and (ii) the minimum mean-square estimation error meets a given distortion constraint. We show that such a sensor design problem is tractable by semidefinite programming. …
We consider the problem of single-round private information retrieval (PIR) from $N$ replicated databases. We consider the case when $B$ databases are outdated (unsynchronized), or even worse, adversarial (Byzantine), and therefore, can return incorrect answers. In the PIR problem with Byzantine databases (BPIR), a user wishes to retrieve a specific message from a set of $M$ messages with zero-error, irrespective of the actions performed by the Byzantine databases. We consider the $T$-privacy constraint in this paper, where any $T$ databases can collude, and exchange the queries submitted by t…
Distributed model training is vulnerable to byzantine system failures and adversarial compute nodes, i.e., nodes that use malicious updates to corrupt the global model stored at a parameter server (PS). To guarantee some form of robustness, recent work suggests using variants of the geometric median as an aggregation rule, in place of gradient averaging. Unfortunately, median-based rules can incur a prohibitive computational overhead in large-scale settings, and their convergence guarantees often require strong assumptions. In this work, we present DRACO, a scalable framework for robust distri…
Sensor networks in which energy is a limited resource so that energy consumption must be minimized for the intended application are considered. In this context, an energy-efficient method for the joint estimation of an unknown analog source under a given distortion constraint is proposed. The approach is purely analog, in which each sensor simply amplifies and forwards the noise-corrupted analog bservation to the fusion center for joint estimation. The total transmission power across all the sensor nodes is minimized while satisfying a distortion requirement on the joint estimate. The energy e…
The vision of wireless sensor networks is one of a smart collection of tiny, dumb devices. These motes may be individually cheap, unintelligent, imprecise, and unreliable. Yet they are able to derive strength from numbers, rendering the whole to be strong, reliable and robust. Our approach is to adopt a distributed and randomized mindset and rely on in network processing and network coding. Our general abstraction is that nodes should act only locally and independently, and the desired global behavior should arise as a collective property of the network. We summarize our work and present how t…
Coding and testing schemes for binary hypothesis testing over noisy networks are proposed and their corresponding type-II error exponents are derived. When communication is over a discrete memoryless channel (DMC), our scheme combines Shimokawa-Han-Amari's hypothesis testing scheme with Borade's unequal error protection (UEP) for channel coding. A separate source channel coding architecture is employed. The resulting exponent is optimal for the newly introduced class of \emph{generalized testing against conditional independence}. When communication is over a MAC or a BC, our scheme combines hy…
We consider a wireless sensors network scenario where two nodes detect correlated sources and deliver them to a central collector via a wireless link. Differently from the Slepian-Wolf approach to distributed source coding, in the proposed scenario the sensing nodes do not perform any pre-compression of the sensed data. Original data are instead independently encoded by means of low-complexity convolutional codes. The decoder performs joint decoding with the aim of exploiting the inherent correlation between the transmitted sources. Complexity at the decoder is kept low thanks to the use of an…
We investigate the optimal performance of dense sensor networks by studying the joint source-channel coding problem. The overall goal of the sensor network is to take measurements from an underlying random process, code and transmit those measurement samples to a collector node in a cooperative multiple access channel with potential feedback, and reconstruct the entire random process at the collector node. We provide lower and upper bounds for the minimum achievable expected distortion when the underlying random process is Gaussian. When the Gaussian random process satisfies some general condi…
Consider an Internet-of-Things (IoT) system that monitors a number of multi-valued events through multiple sensors sharing the same bandwidth. Each sensor measures data correlated to one or more events, and communicates to the fusion center at a base station using grant-free random access whenever the corresponding event is active. The base station aims at detecting the active events, and, for each active event, to determine a scalar value describing each active event's state. A conventional solution based on Separate Source-Channel (SSC) coding would use a separate codebook for each sensor an…
A network coding-based scheme is proposed to improve the energy efficiency of distributed storage systems in WSNs (wireless sensor networks), which mainly focuses on two problems: firstly, consideration is given to effective distributed storage technology in WSNs; secondly, we address how to repair the data in failed storage nodes with less resource. For the first problem, we propose a method to obtain a sparse generator matrix to construct network codes, and this sparse generator matrix is proven to be the sparsest. Benefiting from the sparse generator matrix, the energy consumption required …
We study energy management policies for the compression and transmission of source data collected by an energy-harvesting sensor node with a finite energy buffer (e.g., rechargeable battery) and a finite data buffer (memory) between source encoder and channel encoder. The sensor node can adapt the source and channel coding rates depending on the observation and channel states. In such a system, the absence of precise information about the amount of energy available in the future is a key challenge. We provide analytical bounds and scaling laws for the average distortion that depend on the size…
Minimum energy (ME) source coding is an effective technique for efficient communication with energy-constrained devices, such as sensor network nodes. In this paper, the principles of generalized ME source coding is developed that is broadly applicable. Two scenarios - fixed and variable length codewords - are analyzed. The application of this technique to RFID systems where ME source coding is particularly advantageous due to the asymmetric nature of data communications is demonstrated, a first to the best of our knowledge.
This paper presents an in-depth analysis on the energy efficiency of Luby Transform (LT) codes with Frequency Shift Keying (FSK) modulation in a Wireless Sensor Network (WSN) over Rayleigh fading channels with pathloss. We describe a proactive system model according to a flexible duty-cycling mechanism utilized in practical sensor apparatus. The present analysis is based on realistic parameters including the effect of channel bandwidth used in the IEEE 802.15.4 standard, active mode duration and computation energy. A comprehensive analysis, supported by some simulation studies on the probabili…
Source-channel coding for an energy limited wireless sensor node is investigated. The sensor node observes independent Gaussian source samples with variances changing over time slots and transmits to a destination over a flat fading channel. The fading is constant during each time slot. The compressed samples are stored in a finite size data buffer and need to be delivered in at most $d$ time slots. The objective is to design optimal transmission policies, namely, optimal power and distortion allocation, over the time slots such that the average distortion at destination is minimized. In parti…
In cyber-physical systems where sensors measure the temporal evolution of a given phenomenon of interest and radio communication takes place over short distances, the energy spent for source acquisition and compression may be comparable with that used for transmission. Additionally, in order to avoid limited lifetime issues, sensors may be powered via energy harvesting and thus collect all the energy they need from the environment. This work addresses the problem of energy allocation over source acquisition/compression and transmission for energy-harvesting sensors. At first, focusing on a sin…
This paper presents the first in-depth analysis on the energy efficiency of LT codes with Non Coherent M-ary Frequency Shift Keying (NC-MFSK), known as green modulation [1], in a proactive Wireless Sensor Network (WSN) over Rayleigh flat-fading channels with path-loss. We describe the proactive system model according to a pre-determined time-based process utilized in practical sensor nodes. The present analysis is based on realistic parameters including the effect of channel bandwidth used in the IEEE 802.15.4 standard, and the active mode duration. A comprehensive analysis, supported by some …
To enable low-power and low-complexity wireless monitoring, an improved circuit design of Analog Joint Source Channel Coding (AJSCC) is proposed for wireless sensor nodes. This innovative design is based on Analog Divider Blocks (ADB) with tunable spacing between AJSCC levels. The ADB controls the switching between two types of Voltage Controlled Voltage Sources (VCVS). LTSpice simulations were performed to evaluate the performance of the circuit, and the power consumption and circuit complexity of this new ADB-based design were compared with our previous parallel-VCVS design. It is found that…
In this paper, we investigate communication strategies for the multiple access channel with feedback and correlated sources (MACFCS). The MACFCS models a wireless sensor network scenario in which sensors distributed throughout an arbitrary random field collect correlated measurements and transmit them to a common sink. We derive achievable rate regions for the three-node MACFCS. First, we study the strategy when source coding and channel coding are combined, which we term full decoding at sources. Second, we look at several strategies when source coding and channel coding are separated, which …
We consider a monitoring application where sensors periodically report data to a common receiver in a time division multiplex fashion. The sensors are constrained by the limited and unpredictable energy availability provided by Energy Harvesting (EH), and by the channel impairments. To maximize the quality of the reported data, the packets transmitted contain newly generated data blocks together with up to $r - 1$ previously unsuccessfully delivered ones, where $r$ is a design parameter; such blocks are compressed, concatenated and encoded with a channel code. The scheme applies lossy compress…
We consider large-scale wireless sensor networks with $n$ nodes, out of which k are in possession, (e.g., have sensed or collected in some other way) k information packets. In the scenarios in which network nodes are vulnerable because of, for example, limited energy or a hostile environment, it is desirable to disseminate the acquired information throughout the network so that each of the n nodes stores one (possibly coded) packet so that the original k source packets can be recovered, locally and in a computationally simple way from any k(1 + ε) nodes for some small ε> 0. We develop decentra…
A fog-radio access network (F-RAN) architecture is studied for an Internet-of-Things (IoT) system in which wireless sensors monitor a number of multi-valued events and transmit in the uplink using grant-free random access to multiple edge nodes (ENs). Each EN is connected to a central processor (CP) via a finite-capacity fronthaul link. In contrast to conventional information-agnostic protocols based on separate source-channel (SSC) coding, where each device uses a separate codebook, this paper considers an information-centric approach based on joint source-channel (JSC) coding via a non-ortho…
Energy-harvesting wireless sensor networking is an emerging technology with applications to various fields such as environmental and structural health monitoring. A distinguishing feature of wireless sensors is the need to perform both source coding tasks, such as measurement and compression, and transmission tasks. It is known that the overall energy consumption for source coding is generally comparable to that of transmission, and that a joint design of the two classes of tasks can lead to relevant performance gains. Moreover, the efficiency of source coding in a sensor network can be potent…
Traditional distributed source coding rarely considers the possible link between separate encoders. However, the broadcast nature of wireless communication in sensor networks provides a free gossip mechanism which can be used to simplify encoding/decoding and reduce transmission power. Using this broadcast advantage, we present a new two-encoder scheme which imitates the ping-pong game and has a successive approximation structure. For the quadratic Gaussian case, we prove that this scheme is successively refinable on the {sum-rate, distortion pair} surface, which is characterized by the rate-d…
This paper characterizes the optimal type-II error exponent for a distributed hypothesis testing-against-independence problem when the \emph{expected} rate of the sensor-detector link is constrained. Unlike for the well-known Ahlswede-Csiszar result that holds under a \emph{maximum} rate constraint and where a strong converse holds, here the optimal exponent depends on the allowed type-I error exponent. Specifically, if the type-I error probability is limited by $ε$, then the optimal type-II error exponent under an \emph{expected} rate constraint $R$ coincides with the optimal type-II error ex…
We consider a multiterminal source coding problem in which a source is estimated at a central processing unit from lossy-compressed remote observations. Each lossy-encoded observation is produced by a remote sensor which obtains a noisy version of the source and compresses this observation minimizing a local distortion measure which depends only on the marginal distribution of its observation. The central node, on the other hand, has knowledge of the joint distribution of the source and all the observations and produces the source estimate which minimizes a different distortion measure between…
We consider a network coding problem where the destination wants to recover the sum of the signals (Gaussian random variables or random finite field elements) at all the source nodes, but the sum must be kept secret from an eavesdropper that can wiretap on a subset of edges. This setting arises naturally in sensor networks and federated learning, where the secrecy of the sum of the signals (e.g. weights, gradients) may be desired. While the case for finite field can be solved, the case for Gaussian random variables is surprisingly difficult. We give a simple conjecture on the necessary and suf…
We consider the problem of rate and power allocation for a sensor network under the pairwise distributed source coding constraint. For noiseless source-terminal channels, we show that the minimum sum rate assignment can be found by finding a minimum weight arborescence in an appropriately defined directed graph. For orthogonal noisy source-terminal channels, the minimum sum power allocation can be found by finding a minimum weight matching forest in a mixed graph. Numerical results are presented for both cases showing that our solutions always outperform previously proposed solutions. The gain…
The index coding problem is a fundamental transmission problem which occurs in a wide range of multicast networks. Network coding over a large finite field size has been shown to be a theoretically efficient solution to the index coding problem. However the high computational complexity of packet encoding and decoding over a large finite field size, and its subsequent penalty on encoding and decoding throughput and higher energy cost makes it unsuitable for practical implementation in processor and energy constraint devices like mobile phones and wireless sensors. While network coding over GF(…
Cornell University（康奈尔大学）；The University of Texas at Austin（得克萨斯大学奥斯汀分校）
Network coding allows distributed information sources such as sensors to efficiently compress and transmit data to distributed receivers across a bandwidth-limited network. Classical network coding is largely task-agnostic -- the coding schemes mainly aim to faithfully reconstruct data at the receivers, regardless of what ultimate task the received data is used for. In this paper, we analyze a new task-driven network coding problem, where distributed receivers pass transmitted data through machine learning (ML) tasks, which provides an opportunity to improve efficiency by transmitting salient …
