有保证的函数张量奇异值分解

Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods

Training neural networks is a challenging non-convex optimization problem, and backpropagation or gradient descent can get stuck in spurious local optima. We propose a novel algorithm based on tensor decomposition for guaranteed training of two-layer neural networks. We provide risk bounds for our proposed method, with a polynomial sample complexity in the relevant parameters, such as input dimension and number of neurons. While learning arbitrary target functions is NP-hard, we provide transparent conditions on the function and the input for learnability. Our training method is based on tenso…

Singular Value Decomposition of Operators on Reproducing Kernel Hilbert Spaces

Reproducing kernel Hilbert spaces (RKHSs) play an important role in many statistics and machine learning applications ranging from support vector machines to Gaussian processes and kernel embeddings of distributions. Operators acting on such spaces are, for instance, required to embed conditional probability distributions in order to implement the kernel Bayes rule and build sequential data models. It was recently shown that transfer operators such as the Perron-Frobenius or Koopman operator can also be approximated in a similar fashion using covariance and cross-covariance operators and that …

面向高维高阶数据的最优稀疏奇异值分解

Optimal Sparse Singular Value Decomposition for High-dimensional High-order Data

本文提出STAT-SVD方法，通过双重投影与阈值化方案实现高维高阶稀疏张量的最优降维，并证明其极小极大速率最优性。

The functional singular value decomposition for bivariate stochastic processes

In this article we present some statistical applications of the functional singular value decomposition (FSVD). This tool allows us to decompose the sample mean of a bivariate stochastic process into components that are functions of separate variables. These components are sometimes interpretable functions that summarize salient features of the data. The FSVD can be used to visually detect outliers, to estimate the mean of a stochastic process or to obtain individual smoothers of the sample surfaces. As estimators of the mean, we show by simulation that FSVD estimators are competitive with ten…

Variations of singular spectrum analysis for separability improvement: non-orthogonal decompositions of time series

Singular spectrum analysis (SSA) as a nonparametric tool for decomposition of an observed time series into sum of interpretable components such as trend, oscillations and noise is considered. The separability of these series components by SSA means the possibility of such decomposition. Two variations of SSA, which weaken the separability conditions, are proposed. Both proposed approaches consider inner products corresponding to oblique coordinate systems instead of the conventional Euclidean inner product. One of the approaches performs iterations to obtain separating inner products. The othe…

Generalized eigen, singular value, and partial least squares decompositions: The GSVD package

The generalized singular value decomposition (GSVD, a.k.a. "SVD triplet", "duality diagram" approach) provides a unified strategy and basis to perform nearly all of the most common multivariate analyses (e.g., principal components, correspondence analysis, multidimensional scaling, canonical correlation, partial least squares). Though the GSVD is ubiquitous, powerful, and flexible, it has very few implementations. Here I introduce the GSVD package for R. The general goal of GSVD is to provide a small set of accessible functions to perform the GSVD and two other related decompositions (generali…