Randomized matrix algorithms have become workhorse tools in scientific computing and machine learning. To use these algorithms safely in applications, they should be coupled with posterior error estimates to assess the quality of the output. To meet this need, this paper proposes two diagnostics: a leave-one-out error estimator for randomized low-rank approximations and a jackknife resampling method to estimate the variance of the output of a randomized matrix computation. Both of these diagnostics are rapid to compute for randomized low-rank approximation algorithms such as the randomized SVD and randomized Nyström approximation, and they provide useful information that can be used to assess the quality of the computed output and guide algorithmic parameter choices.
In this paper, we investigate phase retrieval algorithm for the single particle X-ray imaging data. We present a variance-reduced randomized Kaczmarz (VR-RK) algorithm for phase retrieval. The VR-RK algorithm is inspired by the randomized Kaczmarz method and the Stochastic Variance Reduce Gradient Descent (SVRG) algorithm. Numerical experiments show that the VR-RK algorithm has a faster convergence rate than randomized Kaczmarz algorithm and the iterative projection phase retrieval methods, such as the hybrid input output (HIO) and the relaxed averaged alternating reflections (RAAR) methods. T…
Randomized quasi-Monte Carlo (RQMC) sampling can bring orders of magnitude reduction in variance compared to plain Monte Carlo (MC) sampling. The extent of the efficiency gain varies from problem to problem and can be hard to predict. This article presents an R function rhalton that produces scrambled versions of Halton sequences. On some problems it brings efficiency gains of several thousand fold. On other problems, the efficiency gain is minor. The code is designed to make it easy to determine whether a given integrand will benefit from RQMC sampling. An RQMC sample of n points in $[0,1]^d$…
We consider estimating an expected infinite-horizon cumulative discounted cost/reward contingent on an underlying stochastic process by Monte Carlo simulation. An unbiased estimator based on truncating the cumulative cost at a random horizon is proposed. Explicit forms for the optimal distributions of the random horizon are given, and explicit expressions for the optimal random truncation level are obtained, leading to a full analysis of the bias-variance tradeoff when comparing this new class of randomized estimators with traditional fixed truncation estimators. Moreover, we characterize when…
Importance sampling (IS) is valuable in reducing the variance of Monte Carlo sampling for many areas, including finance, rare event simulation, and Bayesian inference. It is natural and obvious to combine quasi-Monte Carlo (QMC) methods with IS to achieve a faster rate of convergence. However, a naive replacement of Monte Carlo with QMC may not work well. This paper investigates the convergence rates of randomized QMC-based IS for estimating integrals with respect to a Gaussian measure, in which the IS measure is a Gaussian or $t$ distribution. We prove that if the target function satisfies th…
Big data analysis has become a crucial part of new emerging technologies such as the internet of things, cyber-physical analysis, deep learning, anomaly detection, etc. Among many other techniques, dimensionality reduction plays a key role in such analyses and facilitates feature selection and feature extraction. Randomized algorithms are efficient tools for handling big data tensors. They accelerate decomposing large-scale data tensors by reducing the computational complexity of deterministic algorithms and the communication among different levels of the memory hierarchy, which is the main bo…
Eigensystem Realization Algorithm (ERA) is a data-driven approach for subspace system identification and is widely used in many areas of engineering. However, the computational cost of the ERA is dominated by a step that involves the singular value decomposition (SVD) of a large, dense matrix with block Hankel structure. This paper develops computationally efficient algorithms for reducing the computational cost of the SVD step by using randomized subspace iteration and exploiting the block Hankel structure of the matrix. We provide a detailed analysis of the error in the identified system mat…
The Tensor-Train (TT) format is a highly compact low-rank representation for high-dimensional tensors. TT is particularly useful when representing approximations to the solutions of certain types of parametrized partial differential equations. For many of these problems, computing the solution explicitly would require an infeasible amount of memory and computational time. While the TT format makes these problems tractable, iterative techniques for solving the PDEs must be adapted to perform arithmetic while maintaining the implicit structure. The fundamental operation used to maintain feasible…
We consider randomized computation of continuous data in the sense of Computable Analysis. Our first contribution formally confirms that it is no loss of generality to take as sample space the Cantor space of infinite FAIR coin flips. This extends [Schröder&Simpson'05] and [Hoyrup&Rojas'09] considering sequences of suitably and adaptively BIASED coins. Our second contribution is concerned with 1D Brownian Motion (aka Wiener Process), a probability distribution on the space of continuous functions f:[0,1]->R with f(0)=0 whose computability has been conjectured [Davie&Fouché'13; arXiv:1409.4667,…
