By integrating two powerful methods of density reduction and intrinsic dimensionality estimation, a new data-driven method, referred to as OLPP-MLE (orthogonal locality preserving projection-maximum likelihood estimation), is introduced for process monitoring. OLPP is utilized for dimensionality reduction, which provides better locality preserving power than locality preserving projection. Then, the MLE is adopted to estimate intrinsic dimensionality of OLPP. Within the proposed OLPP-MLE, two new static measures for fault detection $T_{\scriptscriptstyle {OLPP}}^2$ and ${\rm SPE}_{\scriptscriptstyle {OLPP}}$ are defined. In order to reduce algorithm complexity and ignore data distribution, kernel density estimation is employed to compute thresholds for fault diagnosis. The effectiveness of the proposed method is demonstrated by three case studies.
We show that the mean-model parameter is always orthogonal to the error distribution in generalized linear models. Thus, the maximum likelihood estimator of the mean-model parameter will be asymptotically efficient regardless of whether the error distribution is known completely, known up to a finite vector of parameters, or left completely unspecified, in which case the likelihood is taken to be an appropriate semiparametric likelihood. Moreover, the maximum likelihood estimator of the mean-model parameter will be asymptotically independent of the maximum likelihood estimator of the error dis…
Many statistical problems involve the estimation of a $\left(d\times d\right)$ orthogonal matrix $\textbf{Q}$. Such an estimation is often challenging due to the orthonormality constraints on $\textbf{Q}$. To cope with this problem, we propose a very simple decomposition for orthogonal matrices which we abbreviate as PLR decomposition. It produces a one-to-one correspondence between $\textbf{Q}$ and a $\left(d\times d\right)$ unit lower triangular matrix $\textbf{L}$ whose $d\left(d-1\right)/2$ entries below the diagonal are unconstrained real values. Once the decomposition is applied, regardl…
A Local Orthogonal Polynomial Expansion (LOrPE) of the empirical density function is proposed as a novel method to estimate the underlying density. The estimate is constructed by matching localized expectation values of orthogonal polynomials to the values observed in the sample. LOrPE is related to several existing methods, and generalizes straightforwardly to multivariate settings. By manner of construction, it is similar to Local Likelihood Density Estimation (LLDE). In the limit of small bandwidths, LOrPE functions as Kernel Density Estimation (KDE) with high-order (effective) kernels inhe…
Optimal and suboptimal decentralized estimators in wireless sensor networks (WSNs) over orthogonal multiple-access fading channels are studied in this paper. Considering multiple-bit quantization before digital transmission, we develop maximum likelihood estimators (MLEs) with both known and unknown channel state information (CSI). When training symbols are available, we derive a MLE that is a special case of the MLE with unknown CSI. It implicitly uses the training symbols to estimate the channel coefficients and exploits the estimated CSI in an optimal way. To reduce the computational comple…
Hard thresholding, LASSO , adaptive LASSO and SCAD point estimators have been suggested for use in the linear regression context when most of the components of the regression parameter vector are believed to be zero, a sparsity type of assumption. Potscher and Schneider, 2010, Electronic Journal of Statistics, have considered the properties of fixed-width confidence intervals that include one of these point estimators (for all possible data values). They consider a normal linear regression model with orthogonal regressors and show that these confidence intervals are longer than the standard co…
A Neyman-Orthogonalization Approach to the Incidental Parameter Problem in Likelihood Models
University of Chicago（芝加哥大学）；Toulouse School of Economics, Université Toulouse Capitole（图卢兹经济学院，图卢兹第三大学）；Department of Economics and Nuffield College, University of Oxford（牛津大学经济系与纽菲尔德学院）
This article presents maximum likelihood estimators (MLEs) and log-likelihood ratio (LLR) tests for the eigenvalues and eigenvectors of Gaussian random symmetric matrices of arbitrary dimension, where the observations are independent repeated samples from one or two populations. These inference problems are relevant in the analysis of diffusion tensor imaging data and polarized cosmic background radiation data, where the observations are, respectively, $3\times3$ and $2\times2$ symmetric positive definite matrices. The parameter sets involved in the inference problems for eigenvalues and eigen…
University of Tübingen（蒂宾根大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Cambridge（剑桥大学）；Amazon Web Services（亚马逊云服务）
Variational autoencoders (VAEs) are a popular framework for modeling complex data distributions; they can be efficiently trained via variational inference by maximizing the evidence lower bound (ELBO), at the expense of a gap to the exact (log-)marginal likelihood. While VAEs are commonly used for representation learning, it is unclear why ELBO maximization would yield useful representations, since unregularized maximum likelihood estimation cannot invert the data-generating process. Yet, VAEs often succeed at this task. We seek to elucidate this apparent paradox by studying nonlinear VAEs in …
We consider the problem of sparse estimation via a lasso-type penalized likelihood procedure in a factor analysis model. Typically, the model estimation is done under the assumption that the common factors are orthogonal (uncorrelated). However, the lasso-type penalization method based on the orthogonal model can often estimate a completely different model from that with the true factor structure when the common factors are correlated. In order to overcome this problem, we propose to incorporate a factor correlation into the model, and estimate the factor correlation along with parameters incl…
We introduce Gaussian orthogonal latent factor processes for modeling and predicting large correlated data. To handle the computational challenge, we first decompose the likelihood function of the Gaussian random field with a multi-dimensional input domain into a product of densities at the orthogonal components with lower-dimensional inputs. The continuous-time Kalman filter is implemented to compute the likelihood function efficiently without making approximations. We also show that the posterior distribution of the factor processes is independent, as a consequence of prior independence of f…
Occupancy models involve both the probability a site is occupied and the probability occupancy is detected. The homogeneous occupancy model, where the occupancy and detection probabilities are the same at each site, admits an orthogonal parameter transformation that yields a two-stage process to calculate the maximum likelihood estimates so that it is not necessary to simultaneously estimate the occupancy and detection probabilities. The two-stage approach is examined here for the heterogeneous occupancy model where the occupancy and detection probabilities now depend on covariates that may va…
Random effect models are popular statistical models for detecting and correcting spurious sample correlations due to hidden confounders in genome-wide gene expression data. In applications where some confounding factors are known, estimating simultaneously the contribution of known and latent variance components in random effect models is a challenge that has so far relied on numerical gradient-based optimizers to maximize the likelihood function. This is unsatisfactory because the resulting solution is poorly characterized and the efficiency of the method may be suboptimal. Here we prove anal…
