The dynamic trees problem is to maintain a forest subject to edge insertions and deletions while facilitating queries such as connectivity, path weights, and subtree weights. Dynamic trees are a fundamental building block of a large number of graph algorithms. Although traditionally studied in the single-update setting, dynamic algorithms capable of supporting batches of updates are increasingly relevant today due to the emergence of rapidly evolving dynamic datasets. Since processing updates on a single processor is often unrealistic for large batches of updates, designing parallel batch-dynamic algorithms that achieve provably low span is important for many applications. In this work, we design the first work-efficient parallel batch-dynamic algorithm for dynamic trees that is capable of supporting both path queries and subtree queries, as well as a variety of non-local queries. To achieve this, we propose a framework for algorithmically dynamizing static round-synchronous algorithms that allows us to obtain parallel batch-dynamic algorithms with good bounds on their work and span. In our framework, the algorithm designer can apply the technique to any suitably defined static algorithm. We then obtain theoretical guarantees for algorithms in our framework by defining the notion of a computation distance between two executions of the underlying algorithm.
Our dynamic trees algorithm is obtained by applying our dynamization framework to the parallel tree contraction algorithm of Miller and Reif, and then performing a novel analysis of the computation distance of this algorithm under batch updates. We show that $k$ updates can be performed in $O(k \log(1+n/k))$ work in expectation, which matches an existing algorithm of Tseng et al. while providing support for a substantially larger number of queries and applications.
In this paper, we study new batch-dynamic algorithms for the $k$-clique counting problem, which are dynamic algorithms where the updates are batches of edge insertions and deletions. We study this problem in the parallel setting, where the goal is to obtain algorithms with low (polylogarithmic) depth. Our first result is a new parallel batch-dynamic triangle counting algorithm with $O(Δ\sqrt{Δ+m})$ amortized work and $O(\log^* (Δ+m))$ depth with high probability, and $O(Δ+m)$ space for a batch of $Δ$ edge insertions or deletions. Our second result is an algebraic algorithm based on parallel fa…
Ordered set (and map) is one of the most used data type. In addition to standard set operations, like insert, delete and contains, it can provide set-set operations such as union, intersection, and difference. Each of these set-set operations is equivalent to batched operations: the data structure should process a set of operations insert, delete, and contains. It is obvious that we want these "large" operations to be parallelized. Typically, these sets are implemented with the trees of logarithmic height, such as 2-3 tree, Treap, AVL tree, Red-Black tree, etc. Until now, little attention was …
We study dynamic graph algorithms in the Massively Parallel Computation model, which was inspired by practical data processing systems. Our goal is to provide algorithms that can efficiently handle large batches of edge insertions and deletions. We show algorithms that require fewer rounds to update a solution to problems such as Minimum Spanning Forest, 2-Edge Connected Components, and Maximal Matching than would be required by their static counterparts to compute it from scratch. They work in the most restrictive memory regime, in which local memory per machine is strongly sublinear in the n…
This paper presents a batch-parallel 2-3 tree T in an asynchronous dynamic multithreading model that supports searches, insertions and deletions in sorted batches and has essentially optimal parallelism, even under the restrictive QRMW (queued read-modify-write) memory contention model where concurrent accesses to the same memory location are queued and serviced one by one. Specifically, if T has n items, then performing an item-sorted batch (given as a leaf-based balanced binary tree) of b operations on T takes O( b * log(n/b+1) + b ) work and O( log b + log n ) span (in the worst case as b,n…
Algorithms for dynamically maintaining minimum spanning trees (MSTs) have received much attention in both the parallel and sequential settings. While previous work has given optimal algorithms for dense graphs, all existing parallel batch-dynamic algorithms perform polynomial work per update in the worst case for sparse graphs. In this paper, we present the first work-efficient parallel batch-dynamic algorithm for incremental MST, which can insert $\ell$ edges in $O(\ell \log(1+n/\ell))$ work in expectation and $O(\text{polylog}(n))$ span w.h.p. The key ingredient of our algorithm is an algori…
We consider the problem of scheduling jobs with equal lengths on uniform parallel batch machines with non-identical capacities where each job can only be processed on a specified subset of machines called its processing set. For the case of equal release times, we give efficient exact algorithms for various objective functions. For the case of unequal release times, we give efficient exact algorithms for minimizing makespan.
In this paper, we study batch parallel algorithms for the dynamic connectivity problem, a fundamental problem that has received considerable attention in the sequential setting. The most well known sequential algorithm for dynamic connectivity is the elegant level-set algorithm of Holm, de Lichtenberg and Thorup (HDT), which achieves $O(\log^2 n)$ amortized time per edge insertion or deletion, and $O(\log n / \log\log n)$ time per query. We design a parallel batch-dynamic connectivity algorithm that is work-efficient with respect to the HDT algorithm for small batch sizes, and is asymptoticall…
The dynamic trees problem is to maintain a forest undergoing edge insertions and deletions while supporting queries for information such as connectivity. There are many existing data structures for this problem, but few of them are capable of exploiting parallelism in the batch-setting, in which large batches of edges are inserted or deleted from the forest at once. In this paper, we demonstrate that the Euler tour tree, an existing sequential dynamic trees data structure, can be parallelized in the batch setting. For a batch of $k$ updates over a forest of $n$ vertices, our parallel Euler tou…
In this work, we review Bachelors Theses done under the supervision of Vitaly Aksenov at ITMO University. This overview contains the short description of six theses: "Development of a Streaming Algorithm for the Decomposition of Graph Metrics to Tree Metrics" by Oleg Fafurin, "Development of Memory-friendly Concurrent Data Structures" by Roman Smirnov, "Theoretical Analysis of the Performance of Concurrent Data Structures" by Daniil Bolotov, "Parallel Batched Interpolation Search Tree" by Alena Martsenyuk, "Parallel Batched Self-adjusting Data Structures" by Vitalii Krasnov, and "Parallel Batc…
$k$d-trees are widely used in parallel databases to support efficient neighborhood/similarity queries. Supporting parallel updates to $k$d-trees is therefore an important operation. In this paper, we present BDL-tree, a parallel, batch-dynamic implementation of a $k$d-tree that allows for efficient parallel $k$-NN queries over dynamically changing point sets. BDL-trees consist of a log-structured set of $k$d-trees which can be used to efficiently insert or delete batches of points in parallel with polylogarithmic depth. Specifically, given a BDL-tree with $n$ points, each batch of $B$ updates …
In this paper we study the problem of dynamically maintaining graph properties under batches of edge insertions and deletions in the massively parallel model of computation. In this setting, the graph is stored on a number of machines, each having space strongly sublinear with respect to the number of vertices, that is, $n^ε$ for some constant $0 < ε< 1$. Our goal is to handle batches of updates and queries where the data for each batch fits onto one machine in constant rounds of parallel computation, as well as to reduce the total communication between the machines. This objective corresponds…
Hierarchical agglomerative clustering (HAC) is a popular algorithm for clustering data, but despite its importance, no dynamic algorithms for HAC with good theoretical guarantees exist. In this paper, we study dynamic HAC on edge-weighted graphs. As single-linkage HAC reduces to computing a minimum spanning forest (MSF), our first result is a parallel batch-dynamic algorithm for maintaining MSFs. On a batch of $k$ edge insertions or deletions, our batch-dynamic MSF algorithm runs in $O(k\log^6 n)$ expected amortized work and $O(\log^4 n)$ span with high probability. It is the first fully dynam…
We propose a theoretically-efficient and practical parallel batch-dynamic data structure for the closest pair problem. Our solution is based on a serial dynamic closest pair data structure by Golin et al., and supports batches of insertions and deletions in parallel. For a data set of size $n$, our data structure supports a batch of insertions or deletions of size $m$ in $O(m(1+\log ((n+m)/m)))$ expected work and $O(\log (n+m)\log^*(n+m))$ depth with high probability, and takes linear space. The key techniques for achieving these bounds are a new work-efficient parallel batch-dynamic binary he…
We present a set of parallel algorithms for computing exact k-nearest neighbors in low dimensions. Many k-nearest neighbor algorithms use either a kd-tree or the Morton ordering of the point set; our algorithms combine these approaches using a data structure we call the \textit{zd-tree}. We show that this combination is both theoretically efficient under common assumptions, and fast in practice. For point sets of size $n$ with bounded expansion constant and bounded ratio, the zd-tree can be built in $O(n)$ work with $O(n^ε)$ span for constant $ε<1$, and searching for the $k$-nearest neighbors …
Microsoft Research（微软研究院）；University of Washington（华盛顿大学）；Stanford University（斯坦福大学）
This work characterizes the benefits of averaging schemes widely used in conjunction with stochastic gradient descent (SGD). In particular, this work provides a sharp analysis of: (1) mini-batching, a method of averaging many samples of a stochastic gradient to both reduce the variance of the stochastic gradient estimate and for parallelizing SGD and (2) tail-averaging, a method involving averaging the final few iterates of SGD to decrease the variance in SGD's final iterate. This work presents non-asymptotic excess risk bounds for these schemes for the stochastic approximation problem of leas…
