Lecture notes for the Yale Computer Science course CPSC 4690/5690 Randomized Algorithms. Suitable for use as a supplementary text for an introductory graduate or advanced undergraduate course on randomized algorithms. Discusses tools from probability theory, including random variables and expectations, union bound arguments, concentration bounds, applications of martingales and Markov chains, and the Lovász Local Lemma. Algorithmic topics include analysis of classic randomized algorithms such as Quicksort and Hoare's FIND, randomized tree data structures, hashing, Markov chain Monte Carlo sampling, randomized approximate counting, derandomization, quantum computing, and some examples of randomized distributed algorithms.
These are lecture notes that are based on the lectures from a class I taught on the topic of Randomized Linear Algebra (RLA) at UC Berkeley during the Fall 2013 semester.
We calculate asymptotic expansions for the moments of number of comparisons used by the randomized quick sort algorithm using the singularity analysis of certain generating functions.
In this paper I present a 3SAT algorithm based on the randomized algorithm of Papadimitriou from 1991, and Schoning from 1991. We also present strong arguments that this algorithm finds a solution (if it exists) for a 3SAT problem with high probability in polynomial time.
We present an algorithm for the asymmetric traveling salesman problem on instances which satisfy the triangle inequality. Like several existing algorithms, it achieves approximation ratio O(log n). Unlike previous algorithms, it uses randomized rounding.
We present a randomized parallel algorithm that computes the greatest common divisor of two integers of n bits in length with probability 1-o(1) that takes O(n loglog n / log n) expected time using n^{6+ε} processors on the EREW PRAM parallel model of computation. We believe this to be the first randomized sublinear time algorithm on the EREW PRAM for this problem.
We describe a variant of the Bellman-Ford algorithm for single-source shortest paths in graphs with negative edges but no negative cycles that randomly permutes the vertices and uses this randomized order to process the vertices within each pass of the algorithm. The modification reduces the worst-case expected number of relaxation steps of the algorithm, compared to the previously-best variant by Yen (1970), by a factor of 2/3 with high probability. We also use our high probability bound to add negative cycle detection to the randomized algorithm.
Given a directed graph $G$ and a parameter $k$, the {\sc Long Directed Cycle (LDC)} problem asks whether $G$ contains a simple cycle on at least $k$ vertices, while the {\sc $k$-Path} problems asks whether $G$ contains a simple path on exactly $k$ vertices. Given a deterministic (randomized) algorithm for {\sc $k$-Path} as a black box, which runs in time $t(G,k)$, we prove that {\sc LDC} can be solved in deterministic time $O^*(\max\{t(G,2k),4^{k+o(k)}\})$ (randomized time $O^*(\max\{t(G,2k),4^k\})$). In particular, we get that {\sc LDC} can be solved in randomized time $O^*(4^k)$.
In this paper, we show that there is an O(log k log^2 n)-competitive randomized algorithm for the k-sever problem on any metric space with n points, which improved the previous best competitive ratio O(log^2 k log^3 n log log n) by Nikhil Bansal et al. (FOCS 2011, pages 267-276).
We give a memoryless scale-invariant randomized algorithm for the Buffer Management with Bounded Delay problem that is e/(e-1)-competitive against an adaptive adversary, together with better performance guarantees for many restricted variants, including the s-bounded instances. In particular, our algorithm attains the optimum competitive ratio of 4/3 on 2-bounded instances. Both the algorithm and its analysis are applicable to a more general problem, called Collecting Items, in which only the relative order between packets' deadlines is known. Our algorithm is the optimal randomized memoryless…
The problem of determining if an $r$-CNF boolean formula $F$ over $n$ variables is satisifiable reduces to the problem of determining if $F$ has a satisfying assignment with a Hamming distance of at most $d$ from a fixed assignment $α$. This problem is also a very important subproblem in Schoning's local search algorithm for $r$-SAT. While Schoning described a randomized algorithm solves this subproblem in $O((r-1)^d)$ time, Dantsin et al. presented a deterministic branching algorithm with $O^*(r^d)$ running time. In this paper we present a simple randomized branching algorithm that runs in ti…
In this paper, we describe randomized Shellsort--a simple, randomized, data-oblivious version of the Shellsort algorithm that always runs in O(n log n) time and, as we show, succeeds in sorting any given input permutation with very high probability. Thus, randomized Shellsort is simultaneously simple, time-optimal, and data-oblivious. Taken together, these properties imply applications in the design of new efficient privacy-preserving computations based on the secure multi-party computation (SMC) paradigm. In addition, by a trivial conversion of this Monte Carlo algorithm to its Las Vegas equi…
We comment on two randomized algorithms for constructing low-rank matrix decompositions. Both algorithms employ the Subsampled Randomized Hadamard Transform [14]. The first algorithm appeared recently in [9]; here, we provide a novel analysis that significantly improves the approximation bound obtained in [9]. A preliminary version of the second algorithm appeared in [7]; here, we present a mild modification of this algorithm that achieves the same approximation bound but significantly improves the corresponding running time.
Given a multiset $S$ of $n$ positive integers and a target integer $t$, the Subset Sum problem asks to determine whether there exists a subset of $S$ that sums up to $t$. The current best deterministic algorithm, by Koiliaris and Xu [SODA'17], runs in $\tilde O(\sqrt{n}t)$ time, where $\tilde O$ hides poly-logarithm factors. Bringmann [SODA'17] later gave a randomized $\tilde O(n + t)$ time algorithm using two-stage color-coding. The $\tilde O(n+t)$ running time is believed to be near-optimal. In this paper, we present a simple and elegant randomized algorithm for Subset Sum in $\tilde O(n + t…
In 2005 Li et al. gave a phi-competitive deterministic online algorithm for scheduling of packets with agreeable deadlines with a very interesting analysis. This is known to be optimal due to a lower bound by Hajek. We claim that the algorithm by Li et al. can be slightly simplified, while retaining its competitive ratio. Then we introduce randomness to the modified algorithm and argue that the competitive ratio against oblivious adversary is at most 4/3. Note that this still leaves a gap between the best known lower bound of 5/4 by Chin et al. for randomised algorithms against oblivious adver…
We study the online preemptive scheduling of intervals and jobs (with restarts). Each interval or job has an arrival time, a deadline, a length and a weight. The objective is to maximize the total weight of completed intervals or jobs. While the deterministic case for intervals was settled a long time ago, the randomized case remains open. In this paper we first give a 2-competitive randomized algorithm for the case of equal length intervals. The algorithm is barely random in the sense that it randomly chooses between two deterministic algorithms at the beginning and then sticks with it therea…
In this paper, we settle the randomized $k$-sever conjecture for the following metric spaces: line, circle, Hierarchically well-separated tree (HST). Specially, we show that there are $O(\log k)$-competitive randomized $k$-sever algorithms for above metric spaces. For any general metric space with $n$ points, we show that there is an $O( \log k \log n)$-competitive randomized $k$-sever algorithm.
We revisit the method of Kirschenhofer, Prodinger and Tichy to calculate the moments of comparisons used by the quick sort algorithm. We reemphasize that this approach helps in calculating these quantities with less computation. We also point out that as observed by Knuth this method also gives moments for total path length of a binary search tree built over a random set of n keys.
Randomized algorithms for very large matrix problems have received a great deal of attention in recent years. Much of this work was motivated by problems in large-scale data analysis, and this work was performed by individuals from many different research communities. This monograph will provide a detailed overview of recent work on the theory of randomized matrix algorithms as well as the application of those ideas to the solution of practical problems in large-scale data analysis. An emphasis will be placed on a few simple core ideas that underlie not only recent theoretical advances but als…
Submodularity is one of the most important properties in combinatorial optimization, and $k$-submodularity is a generalization of submodularity. Maximization of a $k$-submodular function requires an exponential number of value oracle queries, and approximation algorithms have been studied. For unconstrained $k$-submodular maximization, Iwata et al. gave randomized $k/(2k-1)$-approximation algorithm for monotone functions, and randomized $1/2$-approximation algorithm for nonmonotone functions. In this paper, we present improved randomized algorithms for nonmonotone functions. Our algorithm give…
We study the question of whether the "termination with probability 1" property of a randomized algorithm is preserved when one replaces the atomic registers that the algorithm uses with linearizable (implementations of) registers. We show that in general this is not so: roughly speaking, every randomized algorithm A has a corresponding algorithm A' that solves the same problem if the registers that it uses are atomic or strongly-linearizable, but does not terminate if these registers are replaced with "merely" linearizable ones. Together with a previous result shown in [15], this implies that …
We study the randomized k-server problem on metric spaces consisting of widely separated subspaces. We give a method which extends existing algorithms to larger spaces with the growth rate of the competitive quotients being at most O(log k). This method yields o(k)-competitive algorithms solving the randomized k-server problem, for some special underlying metric spaces, e.g. HSTs of "small" height (but unbounded degree). HSTs are important tools for probabilistic approximation of metric spaces.
Randomized rounding is a standard method, based on the probabilistic method, for designing combinatorial approximation algorithms. In Raghavan's seminal paper introducing the method (1988), he writes: "The time taken to solve the linear program relaxations of the integer programs dominates the net running time theoretically (and, most likely, in practice as well)." This paper explores how this bottleneck can be avoided for randomized rounding algorithms for packing and covering problems (linear programs, or mixed integer linear programs, having no negative coefficients). The resulting algorith…
A sliding window algorithm receives a stream of symbols and has to output at each time instant a certain value which only depends on the last $n$ symbols. If the algorithm is randomized, then at each time instant it produces an incorrect output with probability at most $ε$, which is a constant error bound. This work proposes a more relaxed definition of correctness which is parameterized by the error bound $ε$ and the failure ratio $ϕ$: A randomized sliding window algorithm is required to err with probability at most $ε$ at a portion of $1-ϕ$ of all time instants of an input stream. This work …
Phase estimation is a quantum algorithm for measuring the eigenvalues of a Hamiltonian. We propose and rigorously analyse a randomized phase estimation algorithm with two distinctive features. First, our algorithm has complexity independent of the number of terms L in the Hamiltonian. Second, unlike previous L-independent approaches, such as those based on qDRIFT, all sources of error in our algorithm can be suppressed by collecting more data samples, without increasing the circuit depth.
Evolutionary algorithms have been frequently used for dynamic optimization problems. With this paper, we contribute to the theoretical understanding of this research area. We present the first computational complexity analysis of evolutionary algorithms for a dynamic variant of a classical combinatorial optimization problem, namely makespan scheduling. We study the model of a strong adversary which is allowed to change one job at regular intervals. Furthermore, we investigate the setting of random changes. Our results show that randomized local search and a simple evolutionary algorithm are ve…
In this work we propose and analyze a simple randomized algorithm to find a satisfiable assignment for a Boolean formula in conjunctive normal form (CNF) having at most 3 literals in every clause. Given a k-CNF formula phi on n variables, and alpha in{0,1}^n that satisfies phi, a clause of phi is critical if exactly one literal of that clause is satisfied under assignment alpha. Paturi et. al. (Chicago Journal of Theoretical Computer Science 1999) proposed a simple randomized algorithm (PPZ) for k-SAT for which success probability increases with the number of critical clauses (with respect to …
The well-known randomized consensus algorithm by Aspnes and Herlihy for asynchronous shared-memory systems was proved to work, even against a strong adversary, under the assumption that the registers that it uses are atomic registers. With atomic registers, every read or write operation is instantaneous (and thus indivisible). As pointed out by Golab et al. (2011), however, a randomized algorithm that works with atomic registers does not necessarily work if we replace the atomic registers that it uses with linearizable implementations of registers. This raises the following question: does the …
In this paper we present randomized algorithms for sorting and convex hull that achieves optimal performance (for speed-up and cache misses) on the multicore model with private cache model. Our algorithms are cache oblivious and generalize the randomized divide and conquer strategy given by Reischuk and Reif and Sen. Although the approach yielded optimal speed-up in the PRAM model, we require additional techniques to optimize cache-misses in an oblivious setting. Under a mild assumption on input and number of processors our algorithm will have optimal time and cache misses with high probabilit…
International Computer Science Institute（国际计算机科学研究所）；University of California Berkeley（加州大学伯克利分校）；Xerox Research Centre India（施乐印度研究中心）；Purdue University（普渡大学）
We present and analyze a simple, two-step algorithm to approximate the optimal solution of the sparse PCA problem. Our approach first solves a L1 penalized version of the NP-hard sparse PCA optimization problem and then uses a randomized rounding strategy to sparsify the resulting dense solution. Our main theoretical result guarantees an additive error approximation and provides a tradeoff between sparsity and accuracy. Our experimental evaluation indicates that our approach is competitive in practice, even compared to state-of-the-art toolboxes such as Spasm.
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Low-rank Tucker and CP tensor decompositions are powerful tools in data analytics. The widely used alternating least squares (ALS) method, which solves a sequence of over-determined least squares subproblems, is costly for large and sparse tensors. We propose a fast and accurate sketched ALS algorithm for Tucker decomposition, which solves a sequence of sketched rank-constrained linear least squares subproblems. Theoretical sketch size upper bounds are provided to achieve $O(ε)$ relative error for each subproblem with two sketching techniques, TensorSketch and leverage score sampling. Experime…
Some randomized algorithms, used to obtain a random $n^2 \times n^2$ Sudoku matrix, where $n$ is a natural number, is reviewed in this study. Below is described the set $Π_n$ of all $(2n) \times n$ matrices, consisting of elements of the set $\mathbb{Z}_n =\{ 1,2,\ldots ,n\}$, such that every row is a permutation. It is proved that such matrices would be particularly useful in developing efficient algorithms in generating Sudoku matrices. An algorithm to obtain random $Π_n$ matrices is presented in this paper. The algorithms are evaluated according to two criteria - probability evaluation, and…
