Extreme value statistics provides accurate estimates for the small occurrence probabilities of rare events. While theory and statistical tools for univariate extremes are well-developed, methods for high-dimensional and complex data sets are still scarce. Appropriate notions of sparsity and connections to other fields such as machine learning, graphical models and high-dimensional statistics have only recently been established. This article reviews the new domain of research concerned with the detection and modeling of sparse patterns in rare events. We first describe the different forms of extremal dependence that can arise between the largest observations of a multivariate random vector. We then discuss the current research topics including clustering, principal component analysis and graphical modeling for extremes. Identification of groups of variables which can be concomitantly extreme is also addressed. The methods are illustrated with an application to flood risk assessment.
We provide finite sample properties of sparse multivariate ARCH processes, where the linear representation of ARCH models allows for an ordinary least squares estimation. Under the restricted strong convexity of the unpenalized loss function, regularity conditions on the penalty function, strict stationary and beta-mixing process, we prove non-asymptotic error bounds on the regularized ARCH estimator. Based on the primal-dual witness method of Loh and Wainwright (2017), we establish variable selection consistency, including the case when the penalty function is non-convex. These theoretical re…
Principal Component Analysis for Multivariate Extremes
Colorado State University（科罗拉多州立大学）；Université Paris Cité, Université Paris Saclay, ENS Paris Saclay, CNRS, SSA, INSERM, Centre Borelli（巴黎西岱大学、巴黎萨克雷大学、巴黎高等师范学校、法国国家科学研究中心、社会卫生与老龄化研究所、博尔内利中心）
This work aims to give non-asymptotic results for estimating the first principal component of a multivariate random process. We first define the covariance function and the covariance operator in the multivariate case. We then define a projection operator. This operator can be seen as a reconstruction step from the raw data in the functional data analysis context. Next, we show that the eigenelements can be expressed as the solution to an optimization problem, and we introduce the LASSO variant of this optimization problem and the associated plugin estimator. Finally, we assess the estimator's…
In this paper, we propose methods for functional predictor selection and the estimation of smooth functional coefficients simultaneously in a scalar-on-function regression problem under high-dimensional multivariate functional data setting. In particular, we develop two methods for functional group-sparse regression under a generic Hilbert space of infinite dimension. We show the convergence of algorithms and the consistency of the estimation and the selection (oracle property) under infinite-dimensional Hilbert spaces. Simulation studies show the effectiveness of the methods in both the selec…
Covariance estimation is essential yet underdeveloped for analyzing multivariate functional data. We propose a fast covariance estimation method for multivariate sparse functional data using bivariate penalized splines. The tensor-product B-spline formulation of the proposed method enables a simple spectral decomposition of the associated covariance operator and explicit expressions of the resulting eigenfunctions as linear combinations of B-spline bases, thereby dramatically facilitating subsequent principal component analysis. We derive a fast algorithm for selecting the smoothing parameters…
Modeling data with multivariate count responses is a challenging problem due to the discrete nature of the responses. Existing methods for univariate count responses cannot be easily extended to the multivariate case since the dependency among multiple responses needs to be properly accommodated. In this paper, we propose a multivariate Poisson log-normal regression model for multivariate data with count responses. By simultaneously estimating the regression coefficients and inverse covariance matrix over the latent variables with an efficient Monte Carlo EM algorithm, the proposed regression …
Multivariate regression model is a natural generalization of the classical univari- ate regression model for fitting multiple responses. In this paper, we propose a high- dimensional multivariate conditional regression model for constructing sparse estimates of the multivariate regression coefficient matrix that accounts for the dependency struc- ture among the multiple responses. The proposed method decomposes the multivariate regression problem into a series of penalized conditional log-likelihood of each response conditioned on the covariates and other responses. It allows simultaneous esti…
We propose a novel and efficient iterative two-stage variable selection approach for multivariate sparse GLARMA models, which can be used for modelling multivariate discrete-valued time series. Our approach consists in iteratively combining two steps: the estimation of the autoregressive moving average (ARMA) coefficients of multivariate GLARMA models and the variable selection in the coefficients of the Generalized Linear Model (GLM) part of the model performed by regularized methods. We explain how to implement our approach efficiently. Then we assess the performance of our methodology using…
In this paper we propose a computationally efficient algorithm for on-line variable selection in multivariate regression problems involving high dimensional data streams. The algorithm recursively extracts all the latent factors of a partial least squares solution and selects the most important variables for each factor. This is achieved by means of only one sparse singular value decomposition which can be efficiently updated on-line and in an adaptive fashion. Simulation results based on artificial data streams demonstrate that the algorithm is able to select important variables in dynamic se…
Ecole Normale Supérieure de Cachan（卡尚高等师范学院）；CNRS（法国国家科学研究中心）；Université Paris-Saclay（巴黎-萨克雷大学）；Université Paris 13（巴黎第十三大学）
This paper introduces a new multivariate convolutional sparse coding based on tensor algebra with a general model enforcing both element-wise sparsity and low-rankness of the activations tensors. By using the CP decomposition, this model achieves a significantly more efficient encoding of the multivariate signal-particularly in the high order/ dimension setting-resulting in better performance. We prove that our model is closely related to the Kruskal tensor regression problem, offering interesting theoretical guarantees to our setting. Furthermore, we provide an efficient optimization algorith…
Carnegie Mellon University（卡内基梅隆大学）；University of Chicago（芝加哥大学）
We study the problem of multivariate regression where the data are naturally grouped, and a regression matrix is to be estimated for each group. We propose an approach in which a dictionary of low rank parameter matrices is estimated across groups, and a sparse linear combination of the dictionary elements is estimated to form a model within each group. We refer to the method as conditional sparse coding since it is a coding procedure for the response vectors Y conditioned on the covariate vectors X. This approach captures the shared information across the groups while adapting to the structur…
Radiogenomics is an emerging field in cancer research that combines medical imaging data with genomic data to predict patients clinical outcomes. In this paper, we propose a multivariate sparse group lasso joint model to integrate imaging and genomic data for building prediction models. Specifically, we jointly consider two models, one regresses imaging features on genomic features, and the other regresses patients clinical outcomes on genomic features. The regularization penalties through sparse group lasso allow incorporation of intrinsic group information, e.g. biological pathway and imagin…
National University of Singapore（新加坡国立大学）；Arizona State University（亚利桑那州立大学）；Samsung Electronics（三星电子）；Georgia Institute of Technology（佐治亚理工学院）
Multivariate functional data from a complex system are naturally high-dimensional and have complex cross-correlation structure. The complexity of data structure can be observed as that (1) some functions are strongly correlated with similar features, while some others may have almost no cross-correlations with quite diverse features; and (2) the cross-correlation structure may also change over time due to the system evolution. With this regard, this paper presents a dynamic subspace learning method for multivariate functional data modeling. In particular, we consider different functions come f…
LTCI, Télécom ParisTech, Université Paris-Saclay（LTCI，巴黎高等电信学院，巴黎萨克雷大学）；INRIA, Université Paris Saclay（法国国家信息与自动化研究所，巴黎萨克雷大学）
Frequency-specific patterns of neural activity are traditionally interpreted as sustained rhythmic oscillations, and related to cognitive mechanisms such as attention, high level visual processing or motor control. While alpha waves (8-12 Hz) are known to closely resemble short sinusoids, and thus are revealed by Fourier analysis or wavelet transforms, there is an evolving debate that electromagnetic neural signals are composed of more complex waveforms that cannot be analyzed by linear filters and traditional signal representations. In this paper, we propose to learn dedicated representations…
The study of multivariate extremes is dominated by multivariate regular variation, although it is well known that this approach does not provide adequate distinction between random vectors whose components are not always simultaneously large. Various alternative dependence measures and representations have been proposed, with the most well-known being hidden regular variation and the conditional extreme value model. These varying depictions of extremal dependence arise through consideration of different parts of the multivariate domain, and particularly exploring what happens when extremes of …
Block maxima methods constitute a fundamental part of the statistical toolbox in extreme value analysis. However, most of the corresponding theory is derived under the simplifying assumption that block maxima are independent observations from a genuine extreme value distribution. In practice however, block sizes are finite and observations from different blocks are dependent. Theory respecting the latter complications is not well developed, and, in the multivariate case, has only recently been established for disjoint blocks of a single block size. We show that using overlapping blocks instead…
This paper introduces the sparse functional boxplot and the intensity sparse functional boxplot as practical exploratory tools. Besides being available for complete functional data, they can be used in sparse univariate and multivariate functional data. The sparse functional boxplot, based on the functional boxplot, displays sparseness proportions within the 50\% central region. The intensity sparse functional boxplot indicates the relative intensity of fitted sparse point patterns in the central region. The two-stage functional boxplot, which derives from the functional boxplot to detect outl…
Multivariate Poisson processes have many important applications in Insurance, Finance, and many other areas of Applied Probability. In this paper we study the backward simulation approach to modelling multivariate Poisson processes and analyze the connection to the extreme measures describing the joint distribution of the processes at the terminal simulation time.
In this paper we devise a statistical method for tracking and modeling change-points on the dependence structure of multivariate extremes. The methods are motivated by and illustrated on a case study on crypto-assets.
We discuss probabilistic models of random covariance structures defined by distributions over sparse eigenmatrices. The decomposition of orthogonal matrices in terms of Givens rotations defines a natural, interpretable framework for defining distributions on sparsity structure of random eigenmatrices. We explore theoretical aspects and implications for conditional independence structures arising in multivariate Gaussian models, and discuss connections with sparse PCA, factor analysis and Gaussian graphical models. Methodology includes model-based exploratory data analysis and Bayesian analysis…
Columbia University（哥伦比亚大学）；Cornell University（康奈尔大学）；National University of Singapore（新加坡国立大学）
We propose kernel PCA as a method for analyzing the dependence structure of multivariate extremes and demonstrate that it can be a powerful tool for clustering and dimension reduction. Our work provides some theoretical insight into the preimages obtained by kernel PCA, demonstrating that under certain conditions they can effectively identify clusters in the data. We build on these new insights to characterize rigorously the performance of kernel PCA based on an extremal sample, i.e., the angular part of random vectors for which the radius exceeds a large threshold. More specifically, we focus…
A novel elastic time distance for sparse multivariate functional data is proposed and used to develop a robust distance-based two-layer partition clustering method. With this proposed distance, the new approach not only can detect correct clusters for sparse multivariate functional data under outlier settings but also can detect those outliers that do not belong to any clusters. Classical distance-based clustering methods such as density-based spatial clustering of applications with noise (DBSCAN), agglomerative hierarchical clustering, and $K$-medoids are extended to the sparse multivariate f…
Multivariate extreme-value analysis is concerned with the extremes in a multivariate random sample, that is, points of which at least some components have exceptionally large values. Mathematical theory suggests the use of max-stable models for univariate and multivariate extremes. A comprehensive account is given of the various ways in which max-stable models are described. Furthermore, a construction device is proposed for generating parametric families of max-stable distributions. Although the device is not new, its role as a model generator seems not yet to have been fully exploited.
The main approach to inference for multivariate extremes consists in approximating the joint upper tail of the observations by a parametric family arising in the limit for extreme events. The latter may be expressed in terms of componentwise maxima, high threshold exceedances or point processes, yielding different but related asymptotic characterizations and estimators. The present paper clarifies the connections between the main likelihood estimators, and assesses their practical performance. We investigate their ability to estimate the extremal dependence structure and to predict future extr…
We consider regularly varying random vectors. Our goal is to estimate in a non-parametric way some characteristics related to conditioning on an extreme event, like the tail dependence coefficient. We introduce a quasi-spectral decomposition that allow to improve efficiency of estimators. Asymptotic normality of estimators is based on weak convergence of tail empirical processes. Theoretical results are supported by simulation studies.
Statistical modeling of multivariate and spatial extreme events has attracted broad attention in various areas of science. Max-stable distributions and processes are the natural class of models for this purpose, and many parametric families have been developed and successfully applied. Due to complicated likelihoods, the efficient statistical inference is still an active area of research, and usually composite likelihood methods based on bivariate densities only are used. Thibaud et al. (2016, Ann. Appl. Stat., to appear) use a Bayesian approach to fit a Brown--Resnick process to extreme tempe…
University of Southern California（南加州大学）；IBM T.J. Watson Research Center（IBM T.J. Watson研究中心）
In many applications of time series models, such as climate analysis and social media analysis, we are often interested in extreme events, such as heatwave, wind gust, and burst of topics. These time series data usually exhibit a heavy-tailed distribution rather than a Gaussian distribution. This poses great challenges to existing approaches due to the significantly different assumptions on the data distributions and the lack of sufficient past data on extreme events. In this paper, we propose the Sparse-GEV model, a latent state model based on the theory of extreme value modeling to automatic…
Studies of Alzheimer's disease (AD) often collect multiple longitudinal clinical outcomes, which are correlated and predictive of AD progression. It is of great scientific interest to investigate the association between the outcomes and time to AD onset. We model the multiple longitudinal outcomes as multivariate sparse functional data and propose a functional joint model linking multivariate functional data to event time data. In particular, we propose a multivariate functional mixed model (MFMM) to identify the shared progression pattern and outcome-specific progression patterns of the outco…
Let $X_{nr}$ be the $r$th largest of a random sample of size $n$ from a distribution $F (x) = 1 - \sum_{i = 0}^\infty c_i x^{-α- i β}$ for $α> 0$ and $β> 0$. An inversion theorem is proved and used to derive an expansion for the quantile $F^{-1} (u)$ and powers of it. From this an expansion in powers of $(n^{-1}, n^{-β/α})$ is given for the multivariate moments of the extremes $\{X_{n, n - s_i}, 1 \leq i \leq k \}/n^{1/α}$ for fixed ${\bf s} = (s_1, ..., s_k)$, where $k \geq 1$. Examples include the Cauchy, Student $t$, $F$, second extreme distributions and stable laws of index $α< 1$.
We propose a nonparametric method for detecting nonlinear causal relationship within a set of multidimensional discrete time series, by using sparse additive models (SpAMs). We show that, when the input to the SpAM is a $β$-mixing time series, the model can be fitted by first approximating each unknown function with a linear combination of a set of B-spline bases, and then solving a group-lasso-type optimization problem with nonconvex regularization. Theoretically, we characterize the oracle statistical properties of the proposed sparse estimator in function estimation and model selection. Num…
The classical multivariate extreme-value theory concerns the modeling of extremes in a multivariate random sample, suggesting the use of max-stable distributions. In this work, the classical theory is extended to the case where aggregated data, such as maxima of a random number of observations, are considered. We derive a limit theorem concerning the attractors for the distributions of the aggregated data, which boil down to a new family of max-stable distributions. We also connect the extremal dependence structure of classical max-stable distributions and that of our new family of max-stable …
In environmental sciences, it is often of interest to assess whether the dependence between extreme measurements has changed during the observation period. The aim of this work is to propose a statistical test that is particularly sensitive to such changes. The resulting procedure is also extended to allow the detection of changes in the extreme-value dependence under the presence of known breaks in the marginal distributions. Simulations are carried out to study the finite-sample behavior of both versions of the proposed test. Illustrations on hydrological data sets conclude the work.
We develop two models for the temporal evolution of extreme events of multivariate $k$th order Markov processes. The foundation of our methodology lies in the conditional extremes model of Heffernan & Tawn (2004), and it naturally extends the work of Winter & Tawn (2016,2017) and Tendijck et al. (2019) to include multivariate random variables. We use cross-validation-type techniques to develop a model order selection procedure, and we test our models on two-dimensional meteorological-oceanographic data with directional covariates for a location in the northern North Sea. We conclude that the n…
In genetic studies, not only can the number of predictors obtained from microarray measurements be extremely large, there can also be multiple response variables. Motivated by such a situation, we consider semiparametric dimension reduction methods in sparse multivariate regression models. Previous studies on joint variable and rank selection have focused on parametric models while here we consider the more challenging varying-coefficient models which make the investigation on nonlinear interactions of variables possible. Spline approximation, rank constraints and concave group penalties are u…
Numerical climate models are complex and combine a large number of physical processes. They are key tools in quantifying the relative contribution of potential anthropogenic causes (e.g., the current increase in greenhouse gases) on high impact atmospheric variables like heavy rainfall. These so-called climate extreme event attribution problems are particularly challenging in a multivariate context, that is, when the atmospheric variables are measured on a possibly high-dimensional grid. In this paper, we leverage two statistical theories to assess causality in the context of multivariate extr…
