In longitudinal studies, repeated measures are collected over time and hence they tend to be serially correlated. In this paper we consider an extension of skew-normal/independent linear mixed models introduced by Lachos et al. (2010), where the error term has a dependence structure, such as damped exponential correlation or autoregressive correlation of order p. The proposed model provides flexibility in capturing the effects of skewness and heavy tails simultaneously when continuous repeated measures are serially correlated. For this robust model, we present an efficient EM-type algorithm for computation of maximum likelihood estimation of parameters and the observed information matrix is derived analytically to account for standard errors. The methodology is illustrated through an application to schizophrenia data and several simulation studies. The proposed algorithm and methods are implemented in the new R package skewlmm.
In this paper, we propose finite mixtures of multivariate skew Laplace distributions to model both skewness and heavy-tailedness in the heterogeneous data sets. The maximum likelihood estimators for the parameters of interest are obtained by using the EM algorithm. We give a small simulation study and a real data example to illustrate the performance of the proposed mixture model.
In this paper, a new mixture family of multivariate normal distributions, formed by mixing multivariate normal distribution and skewed distribution, is constructed. Some properties of this family, such as characteristic function, moment generating function, and the first four moments are derived. The distributions of affine transformations and canonical forms of the model are also derived. An EM type algorithm is developed for the maximum likelihood estimation of model parameters. We have considered in detail, some special cases of the family, using standard gamma and standard exponential mixt…
In this paper, we present an algorithm for the fitting of a location-scale variant of the canonical fundamental skew t (CFUST) distribution, a superclass of the restricted and unrestricted skew t-distributions. In recent years, a few versions of the multivariate skew $t$ (MST) model have been put forward, together with various EM-type algorithms for parameter estimation. These formulations adopted either a restricted or unrestricted characterization for their MST densities. In this paper, we examine a natural generalization of these developments, employing the CFUST distribution as the paramet…
We propose the finite mixture of skewed sub-Gaussian stable distributions. The maximum likelihood estimator for the parameters of proposed finite mixture model is computed through the expectation-maximization algorithm. The proposed model contains the finite mixture of normal and skewed normal distributions. Since the tails of proposed model is heavier than even the Student's t distribution, it can be used as a powerful model for robust model-based clustering. Performance of the proposed model is demonstrated by clustering simulation data and two sets of real data.
Skew normal mixture models provide a more flexible framework than the popular normal mixtures for modelling heterogeneous data with asymmetric behaviors. Due to the unboundedness of likelihood function and the divergency of shape parameters, the maximum likelihood estimators of the parameters of interest are often not well defined, leading to dissatisfactory inferential process. We put forward a proposal to deal with these issues simultaneously in the context of penalizing the likelihood function. The resulting penalized maximum likelihood estimator is proved to be strongly consistent when the…
The mixture of factor analyzers (MFA) model provides a powerful tool for analyzing high-dimensional data as it can reduce the number of free parameters through its factor-analytic representation of the component covariance matrices. This paper extends the MFA model to incorporate a restricted version of the multivariate skew-normal distribution to model the distribution of the latent component factors, called mixtures of skew-normal factor analyzers (MSNFA). The proposed MSNFA model allows us to relax the need for the normality assumption for the latent factors in order to accommodate skewness…
Nonlinear mixed effects models have received a great deal of attention in the statistical literature in recent years because of their flexibility in handling longitudinal studies, including human immunodeficiency virus viral dynamics, pharmacokinetic analyses, and studies of growth and decay. A standard assumption in nonlinear mixed effects models for continuous responses is that the random effects and the within-subject errors are normally distributed, making the model sensitive to outliers. We present a novel class of asymmetric nonlinear mixed effects models that provides efficient paramete…
The normality assumption on data set is very restrictive approach for modelling. The generalized form of normal distribution, named as an exponential power (EP) distribution, and its scale mixture form have been considered extensively to overcome the problem for modelling non-normal data set since last decades. However, examining the robustness properties of maximum likelihood (ML) estimators of parameters in these distributions, such as the in uence function, gross-error sensitivity, breakdown point and information-standardized sensitivity, has not been considered together. The well-known asy…
We introduce a mixture of generalized hyperbolic distributions as an alternative to the ubiquitous mixture of Gaussian distributions as well as their near relatives of which the mixture of multivariate t and skew-t distributions are predominant. The mathematical development of our mixture of generalized hyperbolic distributions model relies on its relationship with the generalized inverse Gaussian distribution. The latter is reviewed before our mixture models are presented along with details of the aforesaid reliance. Parameter estimation is outlined within the expectation-maximization framewo…
Parametric ROC Analysis and Optimal Cutoff Selection under Scale Mixtures of Skew-Normal Distributions: A Decision-Theoretic Framework with Asymptotic Inference
In this study, we propose a robust mixture regression procedure based on the skew t distribution to model heavy-tailed and/or skewed errors in a mixture regression setting. Using the scale mixture representation of the skew t distribution, we give an Expectation Maximization (EM) algorithm to compute the maximum likelihood (ML) estimates for the paramaters of interest. The performance of proposed estimators is demonstrated by a simulation study and a real data example.
We propose a family of four-parameter distributions that contain the K-distribution as special case. The family is derived as a mixture distribution that uses the three-parameter reflected Gamma distribution as parental and the two-parameter Gamma distribution as prior. Properties of the proposed family are investigated as well; these include probability density function, cumulative distribution function, moments, and cumulants. The family is termed symmetric K-distribution (SKD) based on its resemblance to the K-distribution as well as its symmetric nature. The standard form of the SKD, which…
We develop a class of tests for semiparametric vector autoregressive (VAR) models with unspecified innovation densities, based on the recent measure-transportation-based concepts of multivariate {\it center-outward ranks} and {\it signs}. We show that these concepts, combined with Le Cam's asymptotic theory of statistical experiments, yield novel testing procedures, which (a)~are valid under a broad class of innovation densities (possibly non-elliptical, skewed, and/or with infinite moments), (b)~are optimal (locally asymptotically maximin or most stringent) at selected ones, and (c) are robus…
Recent studies about cryptocurrency returns show that its distribution can be highly-peaked, skewed, and heavy-tailed, with a large excess kurtosis. To accommodate all these peculiarities, we propose the asymmetric Laplace scale mixture (ALSM) family of distributions. Each member of the family is obtained by dividing the scale parameter of the conditional asymmetric Laplace (AL) distribution by a convenient mixing random variable taking values on all or part of the positive real line and whose distribution depends on a parameter vector $\boldsymbolθ$ providing greater flexibility to the result…
Mixtures of skew-t distributions offer a flexible choice for model-based clustering. A mixture model of this sort can be implemented using a variety of formulations of the skew-t distribution. Herein we develop a mixture of skew-t factor analyzers model for clustering of high-dimensional data using a flexible formulation of the skew-t distribution. Methodological details of our approach, which represents an extension of the mixture of factor analyzers model to a flexible skew-t distribution, are outlined and details of parameter estimation are provided. Clustering results are illustrated and c…
We present a proposal to deal with the non-normality issue in the context of regression models with measurement errors when both the response and the explanatory variable are observed with error. We extend the normal model by jointly modeling the unobserved covariate and the random errors by a finite mixture of scale mixture of skew-normal distributions. This approach allows us to model data with great flexibility, accommodating skewness, heavy tails, and multi-modality.
In this paper, we introduce a mixture of skew-t factor analyzers as well as a family of mixture models based thereon. The mixture of skew-t distributions model that we use arises as a limiting case of the mixture of generalized hyperbolic distributions. Like their Gaussian and t-distribution analogues, our mixture of skew-t factor analyzers are very well-suited to the model-based clustering of high-dimensional data. Imposing constraints on components of the decomposed covariance parameter results in the development of eight flexible models. The alternating expectation-conditional maximization …
Mixtures of factor analyzers (MFA) provide a powerful tool for modelling high-dimensional datasets. In recent years, several generalizations of MFA have been developed where the normality assumption of the factors and/or of the errors was relaxed to allow for skewness in the data. However, due to the form of the adopted component densities, the distribution of the factors/errors in most of these models is typically limited to modelling skewness oncentrated in a single direction. Here, we introduce a more flexible finite mixture of factor analyzers based on the class of scale mixtures of canoni…
Clustering time series into similar groups can improve models by combining information across like time series. While there is a well developed body of literature for clustering of time series, these approaches tend to generate clusters independently of model training which can lead to poor model fit. We propose a novel distributed approach that simultaneously clusters and fits autoregression models for groups of similar individuals. We apply a Wishart mixture model so as to cluster individuals while modeling the corresponding autocovariance matrices at the same time. The fitted Wishart scale …
The canonical form of scale mixtures of multivariate skew-normal distribution is defined, emphasizing its role in summarizing some key properties of this class of distributions. It is also shown that the canonical form corresponds to an affine invariant co-ordinate system as defined in Tyler \emph{et} al. (2009), and a method for obtaining the linear transform that converts a scale mixture of multivariate skew-normal distribution into a canonical form is presented. Related results, where the particular case of the multivariate skew $t$ distribution is considered in greater detail, are the gene…
The modeling of personal accident insurance data has been a topic of extreme relevance in the insurance literature. This kind of data often exhibits positive skewness and heavy tails. In this work, we propose a new quantile regression model based on the scale-mixture Birnbaum-Saunders distribution for modeling personal accident insurance data. The maximum likelihood estimates of the model parameters are obtained via the EM algorithm. Two Monte Carlo simulation studies are performed using the R software. The first study aims to analyze the performances of the EM algorithm to obtain the maximum …
Finite mixture of skew distributions have emerged as an effective tool in modelling heterogeneous data with asymmetric features. With various proposals appearing rapidly in the recent years, which are similar but not identical, the connections between them and their relative performance becomes rather unclear. This paper aims to provide a concise overview of these developments by presenting a systematic classification of the existing skew distributions into four types, thereby clarifying their close relationships. This also aids in understanding the link between some of the proposed expectatio…
In recent work, robust mixture modelling approaches using skewed distributions have been explored to accommodate asymmetric data. We introduce parsimony by developing skew-t and skew-normal analogues of the popular GPCM family that employ an eigenvalue decomposition of a positive-semidefinite matrix. The methods developed in this paper are compared to existing models in both an unsupervised and semi-supervised classification framework. Parameter estimation is carried out using the expectation-maximization algorithm and models are selected using the Bayesian information criterion. The efficacy …
A mixture of common skew-t factor analyzers model is introduced for model-based clustering of high-dimensional data. By assuming common component factor loadings, this model allows clustering to be performed in the presence of a large number of mixture components or when the number of dimensions is too large to be well-modelled by the mixtures of factor analyzers model or a variant thereof. Furthermore, assuming that the component densities follow a skew-t distribution allows robust clustering of skewed data. The alternating expectation-conditional maximization algorithm is employed for parame…
Robust clustering from incomplete data is an important topic because, in many practical situations, real data sets are heavy-tailed, asymmetric, and/or have arbitrary patterns of missing observations. Flexible methods and algorithms for model-based clustering are presented via mixture of the generalized hyperbolic distributions and its limiting case, the mixture of multivariate skew-t distributions. An analytically feasible EM algorithm is formulated for parameter estimation and imputation of missing values for mixture models employing missing at random mechanisms. The proposed methodologies a…
Multivariate scale mixtures of skew-normal (SMSN) variables are flexible models that account for non-normality in multivariate data scenarios by tail weight assessment and a shape vector representing the asymmetry of the model in a directional fashion. Its stochastic representation involves a skew-normal (SN) vector and a non negative mixing scalar variable, independent of the SN vector, that injects kurtosis into the SMSN model. We address the problem of finding the maximal skewness projection for vectors that follow a SMSN distribution; when simple conditions on the moments of the mixing var…
First-passage probability estimation of high-dimensional nonlinear stochastic systems is a significant task to be solved in many science and engineering fields, but remains still an open challenge. The present paper develops a novel approach, termed 'fractional moments-based mixture distribution', to address such challenge. This approach is implemented by capturing the extreme value distribution (EVD) of the system response with the concepts of fractional moments and mixture distribution. In our context, the fractional moment itself is by definition a high-dimensional integral with a complicat…
This is an extended version of the paper Lee and McLachlan (2014b) with simulations and applications added. This paper introduces a finite mixture of canonical fundamental skew t (CFUST) distributions for a model-based approach to clustering where the clusters are asymmetric and possibly long-tailed (Lee and McLachlan, 2014b). The family of CFUST distributions includes the restricted multivariate skew t (rMST) and unrestricted multivariate skew t (uMST) distributions as special cases. In recent years, a few versions of the multivariate skew t (MST) model have been put forward, together with va…
BBVA Data & Analytics（BBVA数据分析公司）；Universitat de Barcelona（巴塞罗那大学）
In regression tasks, aleatoric uncertainty is commonly addressed by considering a parametric distribution of the output variable, which is based on strong assumptions such as symmetry, unimodality or by supposing a restricted shape. These assumptions are too limited in scenarios where complex shapes, strong skews or multiple modes are present. In this paper, we propose a generic deep learning framework that learns an Uncountable Mixture of Asymmetric Laplacians (UMAL), which will allow us to estimate heterogeneous distributions of the output variable and shows its connections to quantile regre…
Finite mixture models have been widely used to model and analyze data from a heterogeneous populations. Moreover, data of this kind can be missing or subject to some upper and/or lower detection limits because of the restriction of experimental apparatuses. Another complication arises when measures of each population depart significantly from normality, for instance, asymmetric behavior. For such data structures, we propose a robust model for censored and/or missing data based on finite mixtures of multivariate skew-normal distributions. This approach allows us to model data with great flexibi…
This paper presents an R package EMMIXcskew for the fitting of the canonical fundamental skew t-distribution (CFUST) and finite mixtures of this distribution (FM-CFUST) via maximum likelihood (ML). The CFUST distribution provides a flexible family of models to handle non-normal data, with parameters for capturing skewness and heavy-tails in the data. It formally encompasses the normal, t, and skew-normal distributions as special and/or limiting cases. A few other versions of the skew t-distributions are also nested within the CFUST distribution. In this paper, an Expectation-Maximization (EM) …
We show how the expectation-maximization (EM) algorithm can be applied exactly for the fitting of mixtures of general multivariate skew t (MST) distributions, eliminating the need for computationally expensive Monte Carlo estimation. Finite mixtures of MST distributions have proven to be useful in modelling heterogeneous data with asymmetric and heavy tail behaviour. Recently, they have been exploited as an effective tool for modelling flow cytometric data. However, without restrictions on the the characterizations of the component skew t-distributions, Monte Carlo methods have been used to fi…
Mixtures of multivariate normal inverse Gaussian (MNIG) distributions can be used to cluster data that exhibit features such as skewness and heavy tails. However, for cluster analysis, using a traditional finite mixture model framework, either the number of components needs to be known $a$-$priori$ or needs to be estimated $a$-$posteriori$ using some model selection criterion after deriving results for a range of possible number of components. However, different model selection criteria can sometimes result in different number of components yielding uncertainty. Here, an infinite mixture model…
In the past few years, there have been a number of proposals for generalizing the factor analysis (FA) model and its mixture version (known as mixtures of factor analyzers (MFA)) using non-normal and asymmetric distributions. These models adopt various types of skew densities for either the factors or the errors. While the relationships between various choices of skew distributions have been discussed in the literature, the differences between placing the assumption of skewness on the factors or on the errors have not been closely studied. This paper examines these formulations and discusses t…
Université de Caen-Normandie（卡昂-诺曼底大学）；Normandie Université（诺曼底大学）；CNRS（法国国家科学研究中心）
Mixture of Experts (MoE) is a popular framework in the fields of statistics and machine learning for modeling heterogeneity in data for regression, classification and clustering. MoE for continuous data are usually based on the normal distribution. However, it is known that for data with asymmetric behavior, heavy tails and atypical observations, the use of the normal distribution is unsuitable. We introduce a new robust non-normal mixture of experts modeling using the skew $t$ distribution. The proposed skew $t$ mixture of experts, named STMoE, handles these issues of the normal mixtures expe…
This paper describes an algorithm for fitting finite mixtures of unrestricted Multivariate Skew t (FM-uMST) distributions. The package EMMIX-uskew implements a closed-form expectation-maximization (EM) algorithm for computing the maximum likelihood (ML) estimates of the parameters for the (unrestricted) FM-MST model in R. EMMIX-uskew also supports visualization of fitted contours in two and three dimensions, and random sample generation from a specified FM-uMST distribution. Finite mixtures of skew t-distributions have proven to be useful in modelling heterogeneous data with asymmetric and hea…
Normal mean-variance mixture distributions are widely applied to simplify a model's implementation and improve their computational efficiency under the Maximum Likelihood (ML) approach. Especially for distributions with normal mean-variance mixtures representation such as the multivariate skewed variance gamma (MSVG) distribution, it utilises the expectation-conditional-maximisation (ECM) algorithm to iteratively obtain the ML estimates. To facilitate application to financial time series, the mean is further extended to include autoregressive terms. Techniques are proposed to deal with the unb…
Mixture of Experts (MoE) is a popular framework for modeling heterogeneity in data for regression, classification and clustering. For continuous data which we consider here in the context of regression and cluster analysis, MoE usually use normal experts, that is, expert components following the Gaussian distribution. However, for a set of data containing a group or groups of observations with asymmetric behavior, heavy tails or atypical observations, the use of normal experts may be unsuitable and can unduly affect the fit of the MoE model. In this paper, we introduce new non-normal mixture o…
An expanded family of mixtures of multivariate power exponential distributions is introduced. While fitting heavy-tails and skewness has received much attention in the model-based clustering literature recently, we investigate the use of a distribution that can deal with both varying tail-weight and peakedness of data. A family of parsimonious models is proposed using an eigen-decomposition of the scale matrix. A generalized expectation-maximization algorithm is presented that combines convex optimization via a minorization-maximization approach and optimization based on accelerated line searc…
Mixture modeling, which considers the potential heterogeneity in data, is widely adopted for classification and clustering problems. Mixture models can be estimated using the Expectation-Maximization algorithm, which works with the complete estimating equations conditioned by the latent membership variables of the cluster assignment based on the hierarchical expression of mixture models. However, when the mixture components have light tails such as a normal distribution, the mixture model can be sensitive to outliers. This study proposes a method of weighted complete estimating equations (WCE)…
Given a random sample of observations, mixtures of normal densities are often used to estimate the unknown continuous distribution from which the data come. Here we propose the use of this semiparametric framework for testing symmetry about an unknown value. More precisely, we show how the null hypothesis of symmetry may be formulated in terms of normal mixture model, with weights about the centre of symmetry constrained to be equal one another. The resulting model is nested in a more general unconstrained one, with same number of mixture components and free weights. Therefore, after having ma…
KTH Royal Institute of Technology（瑞典皇家理工学院）；RaySearch Laboratories（瑞镭实验室）；Silo AI（Silo AI（西洛人工智能公司））
We present a new nonparametric mixture-of-experts model for multivariate regression problems, inspired by the probabilistic k-nearest neighbors algorithm. Using a conditionally specified model, predictions for out-of-sample inputs are based on similarities to each observed data point, yielding predictive distributions represented by Gaussian mixtures. Posterior inference is performed on the parameters of the mixture components as well as the distance metric using a mean-field variational Bayes algorithm accompanied with a stochastic gradient-based optimization procedure. The proposed method is…
Appropriately designing the proposal kernel of particle filters is an issue of significant importance, since a bad choice may lead to deterioration of the particle sample and, consequently, waste of computational power. In this paper we introduce a novel algorithm adaptively approximating the so-called optimal proposal kernel by a mixture of integrated curved exponential distributions with logistic weights. This family of distributions, referred to as mixtures of experts, is broad enough to be used in the presence of multi-modality or strongly skewed distributions. The mixtures are fitted, via…
