Mixtures of multivariate contaminated shifted asymmetric Laplace distributions are developed for handling asymmetric clusters in the presence of outliers (also referred to as bad points herein). In addition to the parameters of the related non-contaminated mixture, for each (asymmetric) cluster, our model has one parameter controlling the proportion of outliers and one specifying the degree of contamination. Crucially, these parameters do not have to be specified a priori, adding a flexibility to our approach that is absent from other approaches such as trimming. Moreover, each observation is given a posterior probability of belonging to a particular cluster, and of being an outlier or not; advantageously, this allows for the automatic detection of outliers. An expectation-conditional maximization algorithm is outlined for parameter estimation and various implementation issues are discussed. The behaviour of the proposed model is investigated, and compared with well-established finite mixtures, on artificial and real data.
Robust Conformal Outlier Detection under Contaminated Reference Data
Department of Electrical and Computer Engineering, Technion IIT, Haifa, Israel(技术学院电子与计算机工程系);Department of Computer Science, Technion IIT, Haifa, Israel(技术学院计算机科学系);Department of Data Sciences and Operations, University of Southern California, Los Angeles, California, USA(南加州大学数据科学与运营系);Department of Computer Science, University of Southern California, Los Angeles, California, USA(南加州大学计算机科学系)
Handling mild outliers and unobserved values in compositional datasets using finite mixtures of mean-parametrised Dirichlet models
University of Pretoria(比勒陀利亚大学);National Institute for Theoretical and Computational Sciences (NITheCS)(理论与计算科学国家研究所);San José State University(圣何塞州立大学);University of Catania(卡塔尼亚大学)
积矩协方差是多元数据分析的基石,从中可以得到相关性,主成分,马氏距离和许多其他结果。不幸的是,积矩协方差和相应的皮尔逊相关性非常容易受到数据中离群值(异常)的影响。虽然已有一些稳健的协方差度量方法,但很少有适合于目前越来越普遍的高维数据。为此,人们需要这样的方法,其计算与维数成比例,保证产生半正定的协方差阵,并且对离群值足够鲁棒,并且在低变异性的统计意义上足够准确。我们使用数据转换来构建这样的方法。该方法简单、快速、适用范围广。我们通过推导影响函数和故障值,以及计算污染数据的均方误差来研究其鲁棒性。利用这些结果,我们选择了一种整体表现良好的方法。这也使我们能够构建一个更快版本的DetectDeviatingCells方法(Risseeuw和Van den Bossche,2018)来检测细胞离群值,可以处理更高的维度。该方法在具有12,000个变量的基因组数据和具有920,000个维度的彩色视频数据上进行了说明。