We consider the scenario in which a set of sources generate messages in a network and a receiver node demands an arbitrary linear function of these messages. We formulate an algebraic test to determine whether an arbitrary network can compute linear functions using linear codes. We identify a class of linear functions that can be computed using linear codes in every network that satisfies a natural cut-based condition. Conversely, for another class of linear functions, we show that the cut-based condition does not guarantee the existence of a linear coding solution. For linear functions over the binary field, the two classes are complements of each other.
Redundancy is abundant in Fog networks (i.e., many computing and storage points) and grows linearly with network size. We demonstrate the transformational role of coding in Fog computing for leveraging such redundancy to substantially reduce the bandwidth consumption and latency of computing. In particular, we discuss two recently proposed coding concepts, namely Minimum Bandwidth Codes and Minimum Latency Codes, and illustrate their impacts in Fog computing. We also review a unified coding framework that includes the above two coding techniques as special cases, and enables a tradeoff between…
Coding for distributed computing supports low-latency computation by relieving the burden of straggling workers. While most existing works assume a simple master-worker model, we consider a hierarchical computational structure consisting of groups of workers, motivated by the need to reflect the architectures of real-world distributed computing systems. In this work, we propose a hierarchical coding scheme for this model, as well as analyze its decoding cost and expected computation time. Specifically, we first provide upper and lower bounds on the expected computing time of the proposed schem…
Consider a mobile edge computing system in which users wish to obtain the result of a linear inference operation on locally measured input data. Unlike the offloaded input data, the model weight matrix is distributed across wireless Edge Nodes (ENs). ENs have non-deterministic computing times, and they can transmit any shared computed output back to the users cooperatively. This letter investigates the potential advantages obtained by coding model information prior to ENs' storage. Through an information-theoretic analysis, it is concluded that, while generally limiting cooperation opportuniti…
In this paper we consider point-to-point and distributed source coding problems where the receiver is only interested in a function of the data sent by the source encoder(s), while knowledge of the function remains unknown to the encoder(s). We find the rate region for these problems, and in particular, show that if the destination is interested in computing a non-bijective function then the rate region for the point-to-point source coding problem expands over the entropy, and the rate region over the distributed source coding problem expands over the Slepian-Wolf rate region. A novel proof te…
Coded distributed computing was recently introduced to mitigate the effect of stragglers on distributed computing. This paper combines ideas of approximate computing with coded computing to further accelerate computation. We propose successive approximation coding (SAC) techniques that realize a tradeoff between accuracy and speed, allowing the distributed computing system to produce approximations that increase in accuracy over time. If a sufficient number of compute nodes finish their tasks, SAC exactly recovers the desired computation. We theoretically provide design guidelines for our SAC …
This paper introduces reconfigurable computing (RC) and specifically chooses one of the prototypes in this field, MorphoSys (M1) [1 - 5]. The paper addresses the results obtained when using RC in mapping algorithms pertaining to digital coding in relation to previous research [6 - 10]. The chosen algorithms relate to cyclic coding techniques, namely the CCITT CRC-16 and the CRC-16. A performance analysis study of the M1 RC system is also presented to evaluate the efficiency of the algorithm execution on the M1 system. For comparison purposes, three other systems where used to map the same algo…
The constrained linear representability problem (CLRP) for polymatroids determines whether there exists a polymatroid that is linear over a specified field while satisfying a collection of constraints on the rank function. Using a computer to test whether a certain rate vector is achievable with vector linear network codes for a multi-source network coding instance and whether there exists a multi-linear secret sharing scheme achieving a specified information ratio for a given secret sharing instance are shown to be special cases of CLRP. Methods for solving CLRP built from group theoretic tec…
The idea of computing with DNA was given by Tom Head in 1987, however in 1994 in a seminal paper, the actual successful experiment for DNA computing was performed by Adleman. The heart of the DNA computing is the DNA hybridization, however, it is also the source of errors. Thus the success of the DNA computing depends on the error control techniques. The classical coding theory techniques have provided foundation for the current information and communication technology (ICT). Thus it is natural to expect that coding theory will be the foundational subject for the DNA computing paradigm. For th…
The error exponent of a discrete memoryless channel is expressed in two forms. One is Gallager's expression with a positive slope parameter and the other is Csiszar and Korner's information-theoretic representation expressed using the mutual information and the relative entropy. They differ in appearance, and existing methods to prove their agreement are not elementary, as they require an evaluation of the KKT conditions that the optimal distribution must satisfy. Similarly, there are two types of expressions for the strong converse exponent. They are Arimoto's expression with a negative slope…
An emerging requirement for 5G systems is the ability to provide wireless ultra-reliable communication (URC) services with close-to-full availability for cloud-based applications. Among such applications, a prominent role is expected to be played by mobile cloud computing (MCC), that is, by the offloading of computationally intensive tasks from mobile devices to the cloud. MCC allows battery-limited devices to run sophisticated applications, such as for gaming or for the "tactile" internet. This paper proposes to apply the framework of reliable service composition to the problem of optimal tas…
We consider the distributed computing problem of multiplying a set of vectors with a matrix. For this scenario, Li et al. recently presented a unified coding framework and showed a fundamental tradeoff between computational delay and communication load. This coding framework is based on maximum distance separable (MDS) codes of code length proportional to the number of rows of the matrix, which can be very large. We propose a block-diagonal coding scheme consisting of partitioning the matrix into submatrices and encoding each submatrix using a shorter MDS code. We show that the assignment of c…
The problem of computing the dimension of a left/right ideal in a group algebra F[G] of a finite group G over a field F is considered. The ideal dimension is related to the rank of a matrix originating from a regular left/right representation of G; in particular, when F[G] is semisimple, the dimension of a principal ideal is equal to the rank of the matrix representing a generator. From this observation, a bound and an efficient algorithm to compute the dimension of an ideal in a group ring are established. Since group codes are ideals in finite group rings, the algorithm allows efficient comp…
We study the zero-error source coding problem in which an encoder with Side Information (SI) $g(Y)$ transmits source symbols $X$ to a decoder. The decoder has SI $Y$ and wants to recover $f(X,Y)$ where $f,g$ are deterministic. We exhibit a condition on the source distribution and $g$ that we call "pairwise shared side information", such that the optimal rate has a single-letter expression. This condition is satisfied if every pair of source symbols "share" at least one SI symbol for all output of $g$. It has a practical interpretation, as $Y$ models a request made by the encoder on an image $X…
We consider the problem of distributedly computing a general class of functions, referred to as gradient-type computation, while maintaining the privacy of the input dataset. Gradient-type computation evaluates the sum of some `partial gradients', defined as polynomials of subsets of the input. It underlies many algorithms in machine learning and data analytics. We propose Harmonic Coding, which universally computes any gradient-type function, while requiring the minimum possible number of workers. Harmonic Coding strictly improves computing schemes developed based on prior works, such as Sham…
This work investigates functional source coding problems with maximal distortion, motivated by approximate function computation in many modern applications. The maximal distortion treats imprecise reconstruction of a function value as good as perfect computation if it deviates less than a tolerance level, while treating reconstruction that differs by more than that level as a failure. Using a geometric understanding of the maximal distortion, we propose a hypergraph-based source coding scheme for function computation that is constructive in the sense that it gives an explicit procedure for fin…
Under the paradigm of caching, partial data is delivered before the actual requests of users are known. In this paper, this problem is modeled as a canonical distributed source coding problem with side information, where the side information represents the users' requests. For the single-user case, a single-letter characterization of the optimal rate region is established, and for several important special cases, closed-form solutions are given, including the scenario of uniformly distributed user requests. In this case, it is shown that the optimal caching strategy is closely related to total…
We propose a unified coded framework for distributed computing with straggling servers, by introducing a tradeoff between "latency of computation" and "load of communication" for some linear computation tasks. We show that the coded scheme of [1]-[3] that repeats the intermediate computations to create coded multicasting opportunities to reduce communication load, and the coded scheme of [4], [5] that generates redundant intermediate computations to combat against straggling servers can be viewed as special instances of the proposed framework, by considering two extremes of this tradeoff: mini…
Edge computing is emerging as a new paradigm to allow processing data at the edge of the network, where data is typically generated and collected, by exploiting multiple devices at the edge collectively. However, exploiting the potential of edge computing is challenging mainly due to the heterogeneous and time-varying nature of edge devices. Coded computation, which advocates mixing data in sub-tasks by employing erasure codes and offloading these sub-tasks to other devices for computation, is recently gaining interest, thanks to its higher reliability, smaller delay, and lower communication c…
Discrete polymatroids are the multi-set analogue of matroids. In this paper, we explore the connections among linear network coding, linear index coding and representable discrete polymatroids. We consider vector linear solutions of networks over a field $\mathbb{F}_q,$ with possibly different message and edge vector dimensions, which are referred to as linear fractional solutions. We define a \textit{discrete polymatroidal} network and show that a linear fractional solution over a field $\mathbb{F}_q,$ exists for a network if and only if the network is discrete polymatroidal with respect to a…
Coded distributed computing can alleviate the communication load by leveraging the redundant storage and computation resources with coding techniques in distributed computing. In this paper, we study a MapReduce-type distributed computing framework over star topological network, where all the workers exchange information through a common access point. The optimal tradeoff among the normalized number of the stored files (storage load), computed intermediate values (computation load) and transmitted bits in the uplink and downlink (communication loads) are characterized. A coded computing scheme…
It is common in the study of secure multicast network coding in the presence of an eavesdropper that has access to $z$ network links, to assume that the source node is the only node that generates random keys. In this setting, the secure multicast rate is well understood. Computing the secure multicast rate, or even the secure unicast rate, in the more general setting in which all network nodes may generate (independent) random keys is known to be as difficult as computing the (non-secure) capacity of multiple-unicast network coding instances --- a well known open problem. This work treats an …
We consider the recently proposed Coded Distributed Computing (CDC) framework that leverages carefully designed redundant computations to enable coding opportunities that substantially reduce the communication load of distributed computing. We generalize this framework to heterogeneous systems where different nodes in the computing cluster can have different storage (or processing) capabilities. We provide the information-theoretically optimal data set placement and coded data shuffling scheme that minimizes the communication load in a cluster with 3 nodes. For clusters with $K>3$ nodes, we pr…
Taking into account of both the huge computing power of intruders and untrusted cloud servers, we develop an enhanced secure pseudonym scheme to protect the privacy of mobile cloud data. To face the huge computing power challenge, we develop an unconditionally secure lightweight network coding pseudonym scheme. For the privacy issue of untrusted cloud server, we further design a two tier network coding to decouple the stored mobile cloud data from the owner pseudonyms. Therefore, our proposed network coding based pseudonym scheme can simultaneously defend against attackers from both outside an…
A central issue of distributed computing systems is how to optimally allocate computing and storage resources and design data shuffling strategies such that the total execution time for computing and data shuffling is minimized. This is extremely critical when the computation, storage and communication resources are limited. In this paper, we study the resource allocation and coding scheme for the MapReduce-type framework with limited resources. In particular, we focus on the coded distributed computing (CDC) approach proposed by Li et al.. We first extend the asymmetric CDC (ACDC) scheme prop…
We study the secure decentralized Pliable Index CODing (PICOD) problem with circular side information sets at the users. The security constraint forbids every user to decode more than one message while a decentralized setting means there is no central transmitter in the system. Compared to the secure but centralized version of the problem, a converse bound from one of our previous works showed a factor of three difference in optimal code length under the constraint of linear encoding. In this paper, we first list the linearly infeasible cases, that is, problems where no linear code can simulta…
This paper considers vector network coding based on rank-metric codes and subspace codes. Our main result is that vector network coding can significantly reduce the required field size compared to scalar linear network coding in the same multicast network. The achieved gap between the field size of scalar and vector network coding is in the order of $q^{(\ell-1)t^2/\ell}-q^t$, for any $q \geq 2$, where $t$ denotes the dimension of the vector solution, and the number of messages is $2 \ell$, ${\ell \geq 2}$. Previously, only a gap of constant size had been shown. This implies also the same gap …
It has been known that the insufficiency of linear coding in achieving the optimal rate of the general index coding problem is rooted in its rate's dependency on the field size. However, this dependency has been described only through the two well-known matroid instances, namely the Fano and non-Fano matroids, which, in turn, limits its scope only to the fields with characteristic two. In this paper, we extend this scope to demonstrate the reliance of linear index coding rate on fields with characteristic three. By constructing two index coding instances of size 29, we prove that for the first…
Imperial College London（帝国理工学院）；University of Maryland, College Park（马里兰大学帕克分校）
Gradient descent (GD) methods are commonly employed in machine learning problems to optimize the parameters of the model in an iterative fashion. For problems with massive datasets, computations are distributed to many parallel computing servers (i.e., workers) to speed up GD iterations. While distributed computing can increase the computation speed significantly, the per-iteration completion time is limited by the slowest straggling workers. Coded distributed computing can mitigate straggling workers by introducing redundant computations; however, existing coded computing schemes are mainly d…
It is known that there exists a multiple-unicast network which has a rate $1$ linear network coding solution if and only if the characteristic of the finite field belongs to a given finite or co-finite set of primes. In this paper, we show that for any non-zero positive rational number $\frac{k}{n}$, there exists a multiple-unicast network which has a rate $\frac{k}{n}$ fractional linear network coding solution if and only if the characteristic of the finite field belongs to a given finite or co-finite set of primes.
An Algebraic Method for Full-Rank Characterization in Binary Linear Coding
The Chinese University of Hong Kong（香港中文大学）；Beijing Normal University（北京师范大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Southeast University（东南大学）；KLMM, AMSS, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）
The two-sender unicast index coding problem is the most fundamental multi-sender index coding problem. The two senders collectively cater to the demands of all the receivers, by taking advantage of the knowledge of their side-information. Each receiver demands a unique message and has some side-information. Weakly secure index coding problem is a practical version of the index coding problem in the presence of an eavesdropper. The eavesdropper can not gain any information about the messages he does not have, by listening to the senders' transmissions. We provide constructions of weakly secure …
Insufficiency of linear coding for the network coding problem was first proved by providing an instance which is solvable only by nonlinear network coding (Dougherty et al., 2005).Based on the work of Effros, et al., 2015, this specific network coding instance can be modeled as a groupcast index coding (GIC)instance with 74 messages and 80 users (where a message can be requested by multiple users). This proves the insufficiency of linear coding for the GIC problem. Using the systematic approach proposed by Maleki et al., 2014, the aforementioned GIC instance can be cast into a unicast index co…
Gradient-based distributed learning in Parameter Server (PS) computing architectures is subject to random delays due to straggling worker nodes, as well as to possible communication bottlenecks between PS and workers. Solutions have been recently proposed to separately address these impairments based on the ideas of gradient coding, worker grouping, and adaptive worker selection. This paper provides a unified analysis of these techniques in terms of wall-clock time, communication, and computation complexity measures. Furthermore, in order to combine the benefits of gradient coding and grouping…
