Detecting Sparse Heterogeneous Mixtures in a Two-Sample Problem

Modelling heterogeneous distributions with an Uncountable Mixture of Asymmetric Laplacians

In regression tasks, aleatoric uncertainty is commonly addressed by considering a parametric distribution of the output variable, which is based on strong assumptions such as symmetry, unimodality or by supposing a restricted shape. These assumptions are too limited in scenarios where complex shapes, strong skews or multiple modes are present. In this paper, we propose a generic deep learning framework that learns an Uncountable Mixture of Asymmetric Laplacians (UMAL), which will allow us to estimate heterogeneous distributions of the output variable and shows its connections to quantile regre…

Grouped Heterogeneous Mixture Modeling for Clustered Data

Clustered data is ubiquitous in a variety of scientific fields. In this paper, we propose a flexible and interpretable modeling approach, called grouped heterogenous mixture modeling, for clustered data, which models cluster-wise conditional distributions by mixtures of latent conditional distributions common to all the clusters. In the model, we assume that clusters are divided into a finite number of groups and mixing proportions are the same within the same group. We provide a simple generalized EM algorithm for computing the maximum likelihood estimator, and an information criterion to sel…

Anonymous Heterogeneous Distributed Detection: Optimal Decision Rules, Error Exponents, and the Price of Anonymity

We explore the fundamental limits of heterogeneous distributed detection in an anonymous sensor network with n sensors and a single fusion center. The fusion center collects the single observation from each of the n sensors to detect a binary parameter. The sensors are clustered into multiple groups, and different groups follow different distributions under a given hypothesis. The key challenge for the fusion center is the anonymity of sensors - although it knows the exact number of sensors and the distribution of observations in each group, it does not know which group each sensor belongs to.…

Regularized Maximum Likelihood Estimation and Feature Selection in Mixtures-of-Experts Models

Mixture of Experts (MoE) are successful models for modeling heterogeneous data in many statistical learning problems including regression, clustering and classification. Generally fitted by maximum likelihood estimation via the well-known EM algorithm, their application to high-dimensional problems is still therefore challenging. We consider the problem of fitting and feature selection in MoE models, and propose a regularized maximum likelihood estimation approach that encourages sparse solutions for heterogeneous regression data models with potentially high-dimensional predictors. Unlike stat…

用于动态异质群体的变系数专家混合模型：在小鼠皮质发育中的应用

Varying-coefficient mixture of experts model for dynamic heterogeneous populations: application to mouse cortical development

研究细胞分化中基因关联变化难题，提出VCMoE模型，其系数随索引变量平滑变化，开发标签一致EM算法及相关理论，经模拟验证，应用于小鼠皮质数据发现新动态，有实现该方法的软件包。

基于有限混合失败率的异质步应力加速寿命测试（h-SSALT）模型

A Finite Mixture Failure-rate based Heterogeneous Step-stress Accelerated Life Testing (h-SSALT) Model

本文提出一种基于有限混合失败率的异质步应力加速寿命测试模型，通过Weibull分布的II型截尾失效时间，允许在第二应力水平上通过有限混合的m个潜在子组产生异质性，利用期望最大化算法进行最大似然估计，并通过模拟研究验证了忽略异质性对寿命预测的系统性偏差。

Distribution-Free Tests for Sparse Heterogeneous Mixtures

We consider the problem of detecting sparse heterogeneous mixtures from a nonparametric perspective, and develop distribution-free tests when all effects have the same sign. Specifically, we assume that the null distribution is symmetric about zero, while the true effects have positive median. We evaluate the precise performance of classical tests for the median (t-test, sign test) and classical tests for symmetry (signed-rank, Smirnov, total number of runs, longest run tests) showing that none of them is asymptotically optimal for the normal mixture model in all sparsity regimes. We then sugg…

The Average Likelihood Ratio for Large-scale Multiple Testing and Detecting Sparse Mixtures

Large-scale multiple testing problems require the simultaneous assessment of many p-values. This paper compares several methods to assess the evidence in multiple binomial counts of p-values: the maximum of the binomial counts after standardization (the `higher-criticism statistic'), the maximum of the binomial counts after a log-likelihood ratio transformation (the `Berk-Jones statistic'), and a newly introduced average of the binomial counts after a likelihood ratio transformation. Simulations show that the higher criticism statistic has a superior performance to the Berk-Jones statistic in …

Finite Mixtures of ERGMs for Modeling Ensembles of Networks

Ensembles of networks arise in many scientific fields, but there are few statistical tools for inferring their generative processes, particularly in the presence of both dyadic dependence and cross-graph heterogeneity. To fill in this gap, we propose characterizing network ensembles via finite mixtures of exponential family random graph models, a framework for parametric statistical modeling of graphs that has been successful in explicitly modeling the complex stochastic processes that govern the structure of edges in a network. Our proposed modeling framework can also be used for applications…

Copula Density Estimation by Finite Mixture of Parametric Copula Densities

A Copula density estimation method that is based on a finite mixture of heterogeneous parametric copula densities is proposed here. More specifically, the mixture components are Clayton, Frank, Gumbel, T, and normal copula densities, which are capable of capturing lower tail,strong central, upper tail, heavy tail, and symmetrical elliptical dependence, respectively. The model parameters are estimated by an interior-point algorithm for the constrained maximum likelihood problem. The interior-point algorithm is compared with the commonly used EM algorithm. Simulation and real data application sh…

利用脑磁图扫描数据检测轻度创伤性脑损伤：一对多样本检验

Detecting Mild Traumatic Brain Injury with MEG Scan Data: One-vs-K-Sample Tests

本文针对MEG数据偏态、多峰和异质性导致传统病例-对照分析失效的问题，提出一种基于源成像、区域对比检验、交叉验证临界值和相似性校正的一对多样本检验方法，并在神经创伤数据上验证其优于常用非参数方法。

Penalized Maximum Likelihood Estimator for Skew Normal Mixtures

Skew normal mixture models provide a more flexible framework than the popular normal mixtures for modelling heterogeneous data with asymmetric behaviors. Due to the unboundedness of likelihood function and the divergency of shape parameters, the maximum likelihood estimators of the parameters of interest are often not well defined, leading to dissatisfactory inferential process. We put forward a proposal to deal with these issues simultaneously in the context of penalizing the likelihood function. The resulting penalized maximum likelihood estimator is proved to be strongly consistent when the…

通过子空间跟踪的动态异质网络变点检测

Change point detection in dynamic heterogeneous networks via subspace tracking

针对动态网络变点检测，本文提出允许网络概率矩阵连续变化而潜在子空间结构仅在特定时间点改变的新模型，设计两个统计量联合检测子空间变点，并证明渐近一致性及不可行区域，实验验证了方法有效性。

Asymptotic Optimality of Mixture Rules for Detecting Changes in General Stochastic Models

The paper addresses a sequential changepoint detection problem for a general stochastic model, assuming that the observed data may be non-i.i.d. (i.e., dependent and non-identically distributed) and the prior distribution of the change point is arbitrary. Tartakovsky and Veeravalli (2005), Baron and Tartakovsky (2006), and, more recently, Tartakovsky (2017) developed a general asymptotic theory of changepoint detection for non-i.i.d.\ stochastic models, assuming the certain stability of the log-likelihood ratio process, in the case of simple hypotheses when both pre-change and post-change mode…

Mixture of Conditional Gaussian Graphical Models for unlabelled heterogeneous populations in the presence of co-factors

Conditional correlation networks, within Gaussian Graphical Models (GGM), are widely used to describe the direct interactions between the components of a random vector. In the case of an unlabelled Heterogeneous population, Expectation Maximisation (EM) algorithms for Mixtures of GGM have been proposed to estimate both each sub-population's graph and the class labels. However, we argue that, with most real data, class affiliation cannot be described with a Mixture of Gaussian, which mostly groups data points according to their geometrical proximity. In particular, there often exists external c…

Detection of Sparse Mixtures: Higher Criticism and Scan Statistic

We consider the problem of detecting a sparse mixture as studied by Ingster (1997) and Donoho and Jin (2004). We consider a wide array of base distributions. In particular, we study the situation when the base distribution has polynomial tails, a situation that has not received much attention in the literature. Perhaps surprisingly, we find that in the context of such a power-law distribution, the higher criticism does not achieve the detection boundary. However, the scan statistic does.

异质响应回归中的强可辨识性与参数学习

Strong identifiability and parameter learning in regression with heterogeneous response

本文研究有限回归混合模型在精确拟合、过度拟合及成分数未知时的强可辨识性、估计收敛速率与贝叶斯后验收缩，并通过模拟和实例说明参数学习行为。

Statistical Limits of Sparse Mixture Detection

We consider the problem of detecting a general sparse mixture and obtain an explicit characterization of the phase transition under some conditions, generalizing the univariate results of Cai and Wu. Additionally, we provide a sufficient condition for the adaptive optimality of a Higher Criticism type testing statistic formulated by Gao and Ma. In the course of establishing these results, we offer a unified perspective through the large deviations theory. The phase transition and adaptive optimality we establish are direct consequences of the large deviation principle of the normalized log-lik…

Curse of Heterogeneity: Computational Barriers in Sparse Mixture Models and Phase Retrieval

We study the fundamental tradeoffs between statistical accuracy and computational tractability in the analysis of high dimensional heterogeneous data. As examples, we study sparse Gaussian mixture model, mixture of sparse linear regressions, and sparse phase retrieval model. For these models, we exploit an oracle-based computational model to establish conjecture-free computationally feasible minimax lower bounds, which quantify the minimum signal strength required for the existence of any algorithm that is both computationally tractable and statistically accurate. Our analysis shows that there…