This paper, broadly speaking, covers the use of randomness in two main areas: low-rank approximation and kernel methods.
Low-rank approximation is very important in numerical linear algebra. Many applications depend on matrix decomposition algorithms that provide accurate low-rank representations of data. In modern problems, however, various factors make this hard to accomplish. One solution to these problems is the use of random projections. Instead of directly computing the matrix factorization, we randomly project the matrix onto a lower-dimensional subspace and then compute the factorization. Often, we are able to do this without significant loss of accuracy. We describe how randomization can be used to create more efficient algorithms to perform low-rank matrix approximation, as well as introducing a novel randomized algorithm for matrix decomposition. Compared to standard approaches, random algorithms are often faster and more robust. With these randomized algorithms, analyzing massive data sets becomes tractable.
Kernel methods are almost diametrically opposite from low-rank approximation. The idea is to project low-dimensional data into a higher-dimensional 'feature space,' such that it is linear separable in the feature space. This enables the model to learn a nonlinear separation of the data. As before, with large data matrices, computing the kernel matrix can be expensive, so we use randomized methods to approximate the matrix. In addition, we propose an extension of the random Fourier features kernel in which hyperparameter values are randomly sampled from an interval or Borel set.
The experiments discussed in this paper can be found on our website at https://rishi1999.github.io/random-projections.
We consider convergence of alternating projections between non-convex sets and obtain applications to convergence of the Gerchberg-Saxton error reduction method, of the Gaussian expectation-maximization algorithm, and of Cadzow's algorithm.
We propose a projection-based monolithic model order reduction (MOR) procedure for a class of problems in nonlinear mechanics with internal variables. The work is is motivated by applications to thermo-hydro-mechanical (THM) systems for radioactive waste disposal. THM equations model the behaviour of temperature, pore water pressure and solid displacement in the neighborhood of geological repositories, which contain radioactive waste and are responsible for a significant thermal flux towards the Earth's surface. We develop an adaptive sampling strategy based on the POD-Greedy method, and we de…
The Kolmogorov $n$-width of the solution manifolds of transport-dominated problems can decay slowly. As a result, it can be challenging to design efficient and accurate reduced order models (ROMs) for such problems. To address this issue, we propose a new learning-based projection method to construct nonlinear adaptive ROMs for transport problems. The construction follows the offline-online decomposition. In the offline stage, we train a neural network to construct adaptive reduced basis dependent on time and model parameters. In the online stage, we project the solution to the learned reduced…
Fluid-structure interactions are central to many bio-molecular processes, and they impose a great challenge for computational and modeling methods. In this paper, we consider the immersed boundary method (IBM) for biofluid systems, and to alleviate the computational cost, we apply reduced-order techniques to eliminate the degrees of freedom associated with a large number of fluid variables. We show how reduced models can be derived using Petrov-Galerkin projection and subspaces that maintain the incompressibility condition. More importantly, the reduced-order model is shown to preserve the Lya…
Random projections reduce the dimension of a set of vectors while preserving structural information, such as distances between vectors in the set. This paper proposes a novel use of row-product random matrices in random projection, where we call it Tensor Random Projection (TRP). It requires substantially less memory than existing dimension reduction maps. The TRP map is formed as the Khatri-Rao product of several smaller random projections, and is compatible with any base random projection including sparse maps, which enable dimension reduction with very low query cost and no floating point o…
This work proposes a windowed least-squares (WLS) approach for model-reduction of dynamical systems. The proposed approach sequentially minimizes the time-continuous full-order-model residual within a low-dimensional space-time trial subspace over time windows. The approach comprises a generalization of existing model reduction approaches, as particular instances of the methodology recover Galerkin, least-squares Petrov-Galerkin (LSPG), and space-time LSPG projection. In addition, the approach addresses key deficiencies in existing model-reduction techniques, e.g., the dependence of LSPG and s…
Hierarchical model reduction of nonlinear partial differential equations based on the adaptive empirical projection method and reduced basis techniques
Institute for Applied Mathematics, University of Münster（明斯特大学应用数学研究所）
A fast algorithm for the approximation of a low rank LU decomposition is presented. In order to achieve a low complexity, the algorithm uses sparse random projections combined with FFT-based random projections. The asymptotic approximation error of the algorithm is analyzed and a theoretical error bound is presented. Finally, numerical examples illustrate that for a similar approximation error, the sparse LU algorithm is faster than recent state-of-the-art methods. The algorithm is completely parallelizable that enables to run on a GPU. The performance is tested on a GPU card, showing a signif…
We present a novel projection-based model reduction framework for parametric linear time-invariant systems that allows interpolating the transfer function at a given frequency point along parameter-dependent curves as opposed to the standard approach where transfer function interpolation is achieved for a discrete set of parameter and frequency samples. We accomplish this goal by using parameter-dependent projection spaces. Our main result shows that for holomorphic system matrices, the corresponding interpolatory projection spaces are also holomorphic. The coefficients of the power series rep…
We present an a posteriori error analysis for one-dimensional random hyperbolic systems of conservation laws. For the discretization of the random space we consider the Non-Intrusive Spectral Projection method, the spatio-temporal discretization uses the Runge--Kutta Discontinuous Galerkin method. We derive an a posteriori error estimator using smooth reconstructions of the numerical solution, which combined with the relative entropy stability framework yields computable error bounds for the space-stochastic discretization error. Moreover, we show that the estimator admits a splitting into a s…
In the literature on projection-based nonlinear model order reduction for fluid dynamics problems, it is often claimed that due to modal truncation, a projection-based reduced-order model (PROM) does not resolve the dissipative regime of the turbulent energy cascade and therefore is numerically unstable. Efforts at addressing this claim have ranged from attempting to model the effects of the truncated modes to enriching the classical subspace of approximation in order to account for the truncated phenomena. This paper challenges this claim. Exploring the relationship between projection-based m…
We propose a new model reduction framework for problems that exhibit transport phenomena. As in the moving finite element method (MFEM), our method employs time-dependent transformation operators and, especially, generalizes MFEM to arbitrary basis functions. The new framework is suitable to obtain a low-dimensional approximation with small errors even in situations where classical model order reduction techniques require much higher dimensions for a similar approximation quality. Analogously to the MFEM framework, the reduced model is designed to minimize the residual, which is also the basis…
University of Vienna（维也纳大学）；Medical University of Vienna（维也纳医科大学）；Brno University of Technology（布尔诺理工大学）
The use of orthogonal projections on high-dimensional input and target data in learning frameworks is studied. First, we investigate the relations between two standard objectives in dimension reduction, preservation of variance and of pairwise relative distances. Investigations of their asymptotic correlation as well as numerical experiments show that a projection does usually not satisfy both objectives at once. In a standard classification problem we determine projections on the input data that balance the objectives and compare subsequent results. Next, we extend our application of orthogon…
Model reduction methods aim to describe complex dynamic phenomena using only relevant dynamical variables, decreasing computational cost, and potentially highlighting key dynamical mechanisms. In the absence of special dynamical features such as scale separation or symmetries, the time evolution of these variables typically exhibits memory effects. Recent work has found a variety of data-driven model reduction methods to be effective for representing such non-Markovian dynamics, but their scope and dynamical underpinning remain incompletely understood. Here, we study data-driven model reductio…
Inspired by our previous work on mitigating the Kolmogorov barrier using a quadratic approximation manifold, we propose in this paper a computationally tractable approach for combining a projection-based reduced-order model (PROM) and an artificial neural network (ANN) for mitigating the Kolmogorov barrier to reducibility of convection-dominated flow problems. The main objective the PROM-ANN concept that we propose is to reduce the dimensionality of the online approximation of the solution beyond what is possible using affine and quadratic approximation manifolds. In contrast to previous appro…
We discuss the application of random projections to conic programming: notably linear, second-order and semidefinite programs. We prove general approximation results on feasibility and optimality using the framework of formally real Jordan algebras. We then discuss some computational experiments on randomly generated semidefinite programs in order to illustrate the practical applicability of our ideas
A data-driven analysis method known as dynamic mode decomposition (DMD) approximates the linear Koopman operator on projected space. In the spirit of Johnson-Lindenstrauss Lemma, we will use random projection to estimate the DMD modes in reduced dimensional space. In practical applications, snapshots are in high dimensional observable space and the DMD operator matrix is massive. Hence, computing DMD with the full spectrum is infeasible, so our main computational goal is estimating the eigenvalue and eigenvectors of the DMD operator in a projected domain. We will generalize the current algorit…
Clustering by projection has been proposed as a way to preserve network structure in linear multi-agent systems. Here, we extend this approach to a class of nonlinear network systems. Additionally, we generalize our clustering method which restores the network structure in an arbitrary reduced-order model obtained by projection. We demonstrate this method on a number of examples.
