首页/论文/2002.02291

arXiv 2002.022912020-02cs.ITmath.IT

Weighted Gradient Coding with Leverage Score Sampling

Neophytos Charalambides, Mert Pilanci, Alfred O. Hero

备注：: 4 pages, 2 figures, 2 tables, conference

英文摘要：

A major hurdle in machine learning is scalability to massive datasets. Approaches to overcome this hurdle include compression of the data matrix and distributing the computations. \textit{Leverage score sampling} provides a compressed approximation of a data matrix using an importance weighted subset. \textit{Gradient coding} has been recently proposed in distributed optimization to compute the gradient using multiple unreliable worker nodes. By designing coding matrices, gradient coded computations can be made resilient to stragglers, which are nodes in a distributed network that degrade system performance. We present a novel \textit{weighted leverage score} approach, that achieves improved performance for distributed gradient coding by utilizing an importance sampling.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

An Importance Aware Weighted Coding Theorem Using Message Importance Measure

There are numerous scenarios in source coding where not only the code length but the importance of each value should also be taken into account. Different from the traditional coding theorems, by adding the importance weights for the length of the codes, we define the average cost of the weighted codeword length as an importance-aware measure of the codes. This novel information theoretical measure generalizes the average codeword length by assigning importance weights for each symbol according to users' concerns through focusing on user's selections. With such definitions, coding theorems of …

2020-05-21cs.ITmath.IT

A Coding Theoretic Approach for Evaluating Accumulate Distribution on Minimum Cut Capacity of Weighted Random Graphs

The multicast capacity of a directed network is closely related to the $s$-$t$ maximum flow, which is equal to the $s$-$t$ minimum cut capacity due to the max-flow min-cut theorem. If the topology of a network (or link capacities) is dynamically changing or have stochastic nature, it is not so trivial to predict statistical properties on the maximum flow. In this paper, we present a coding theoretic approach for evaluating the accumulate distribution of the minimum cut capacity of weighted random graphs. The main feature of our approach is to utilize the correspondence between the cut space of…

2012-02-04cs.ITmath.IT

Hypersurfaces in weighted projective spaces over finite fields with applications to coding theory

We consider the question of determining the maximum number of $\mathbb{F}_q$-rational points that can lie on a hypersurface of a given degree in a weighted projective space over the finite field $\mathbb{F}_q$, or in other words, the maximum number of zeros that a weighted homogeneous polynomial of a given degree can have in the corresponding weighted projective space over $\mathbb{F}_q$. In the case of classical projective spaces, this question has been answered by J.-P. Serre. In the case of weighted projective spaces, we give some conjectures and partial results. Applications to coding theo…

2017-06-09cs.ITmath.AGmath.IT

加权汉明度量：界与编码

Weighted-Hamming Metric: Bounds and Codes

本文提出加权汉明度量的纠错能力界及基于广义拼接的编码构造，改进了传统最小距离方法，并展示了高效解码方案。

2026-01-19cs.ITmath.IT

通过变换编码对加权图邻接矩阵进行有损压缩

Lossy compression of weighted graph adjacency matrices by transform coding

The University of Osaka（大阪大学）；Kansai University（关西大学）；University of Southern California（南加州大学）

研究加权图邻接矩阵的有损压缩，核心方法是先将无加权图转换为线图，再用图滤波器组变换边权值向量并量化编码，还提出边平滑度衡量压缩难度，实验验证了该方法相较于现有方法的有效性。

2026-07-17cs.ITeess.SPmath.IT

Multi-wise and constrained fully weighted Davenport constants and interactions with coding theory

We consider two families of weighted zero-sum constants for finite abelian groups. For a finite abelian group $( G , + )$, a set of weights $W \subset \mathbb{Z}$, and an integral parameter $m$, the $m$-wise Davenport constant with weights $W$ is the smallest integer $n$ such that each sequence over $G$ of length $n$ has at least $m$ disjoint zero-subsums with weights $W$. And, for an integral parameter $d$, the $d$-constrained Davenport constant with weights $W$ is the smallest $n$ such that each sequence over $G$ of length $n$ has a zero-subsum with weights $W$ of size at most $d$. First, we…

2015-05-21cs.ITmath.COmath.IT

更多相关论文收起更多论文

$C^{3}LES$: Codes for Coded Computation that Leverage Stragglers

In distributed computing systems, it is well recognized that worker nodes that are slow (called stragglers) tend to dominate the overall job execution time. Coded computation utilizes concepts from erasure coding to mitigate the effect of stragglers by running "coded" copies of tasks comprising a job. Stragglers are typically treated as erasures in this process. While this is useful, there are issues with applying, e.g., MDS codes in a straightforward manner. Specifically, several applications such as matrix-vector products deal with sparse matrices. MDS codes typically require dense linear co…

2018-09-17cs.ITmath.IT

A Pliable Index Coding Approach to Data Shuffling

A promising research area that has recently emerged, is on how to use index coding to improve the communication efficiency in distributed computing systems, especially for data shuffling in iterative computations. In this paper, we posit that pliable index coding can offer a more efficient framework for data shuffling, as it can better leverage the many possible shuffling choices to reduce the number of transmissions. We theoretically analyze pliable index coding under data shuffling constraints, and design a hierarchical data-shuffling scheme that uses pliable coding as a component. We find b…

2018-05-03cs.ITmath.IT

On Multi-source Networks: Enumeration, Rate Region Computation, and Hierarchy

Recent algorithmic developments have enabled computers to automatically determine and prove the capacity regions of small hypergraph networks under network coding. A structural theory relating network coding problems of different sizes is developed to make best use of this newfound computational capability. A formal notion of network minimality is developed which removes components of a network coding problem that are inessential to its core complexity. Equivalence between different network coding problems under relabeling is formalized via group actions, an algorithm which can directly list s…

2017-04-06cs.ITmath.IT

Minimizing weighted sum download time for one-to-many file transfer in peer-to-peer networks

This paper considers the problem of transferring a file from one source node to multiple receivers in a peer-to-peer (P2P) network. The objective is to minimize the weighted sum download time (WSDT) for the one-to-many file transfer. Previous work has shown that, given an order at which the receivers finish downloading, the minimum WSD can be solved in polynomial time by convex optimization, and can be achieved by linear network coding, assuming that node uplinks are the only bottleneck in the network. This paper, however, considers heterogeneous peers with both uplink and downlink bandwidth c…

2011-07-06cs.ITcs.NImath.IT

基于迭代块杠杆分数采样的梯度编码

Gradient Coding with Iterative Block Leverage Score Sampling

University of Michigan（密歇根大学）；Stanford University（斯坦福大学）

本文提出一种迭代块杠杆分数采样方法，将随机数值线性代数与近似编码计算统一，用于分布式线性回归，以应对掉队节点故障，并通过均匀采样和加权实现高效近似求解。

2024-06-25cs.DCcs.IRcs.IT

A Generalized Channel Coding Theory for Distributed Communication

This paper presents generalized channel coding theorems for a time-slotted distributed communication system where a transmitter-receiver pair is communicating in parallel with other transmitters. Assume that the channel code of each transmitter is chosen arbitrarily in each time slot. The coding choice of a transmitter is denoted by a code index parameter, which is known neither to other transmitters nor to the receiver. Fundamental performance limitation of the system is characterized using an achievable region defined in the space of the code index vectors. As the codeword length is taken to…

2015-02-01cs.ITmath.IT

Multilevel Illumination Coding for Fourier Transform Interferometry in Fluorescence Spectroscopy

Fourier Transform Interferometry (FTI) is an interferometric procedure for acquiring HyperSpectral (HS) data. Recently, it has been observed that the light source highlighting a (biologic) sample can be coded before the FTI acquisition in a procedure called Coded Illumination-FTI (CI-FTI). This turns HS data reconstruction into a Compressive Sensing (CS) problem regularized by the sparsity of the HS data. CI-FTI combines the high spectral resolution of FTI with the advantages of reduced-light-exposure imaging in biology. In this paper, we leverage multilevel sampling scheme recently developed …

2018-03-08cs.ITmath.IT

Coded sparse matrix computation schemes that leverage partial stragglers

Distributed matrix computations over large clusters can suffer from the problem of slow or failed worker nodes (called stragglers) which can dominate the overall job execution time. Coded computation utilizes concepts from erasure coding to mitigate the effect of stragglers by running 'coded' copies of tasks comprising a job; stragglers are typically treated as erasures. While this is useful, there are issues with applying, e.g., MDS codes in a straightforward manner. Several practical matrix computation scenarios involve sparse matrices. MDS codes typically require dense linear combinations o…

2021-09-24cs.DCcs.ITmath.IT