Generalized linear mixed models (GLMMs) are used to model responses from exponential families with a combination of fixed and random effects. For variance components in GLMMs, we propose an approximate restricted likelihood ratio test that conducts testing on the working responses used in penalized quasi-likelihood estimation. This presents the hypothesis test in terms of normalized responses, allowing for application of existing testing methods for linear mixed models. Our test is flexible, computationally efficient, and outperforms several competitors. We illustrate the utility of the proposed method with an extensive simulation study and two data applications. An R package is provided.
In order to overcome multicollinearity, we propose a stochastic restricted Liu-type max- imum likelihood estimator by incorporating Liu-type maximum likelihood estimator (Inan and Erdo- gan, 2013) to the logistic regression model when the linear restrictions are stochastic. We also discuss the properties of the new estimator. Moreover, we give a method to choose the biasing parameter in the new estimator. Finally, a simulation study is given to show the performance of the new estimator.
Linear mixed-effects models are widely used in analyzing repeated measures data, including clustered and longitudinal data, where inferences of both fixed effects and variance components are of importance. Unlike the fixed effect inference that has been well studied, inference on the variance components is more challenging due to null value being on the boundary and the nuisance parameters of the fixed effects. Existing methods often require strong distributional assumptions on the random effects and random errors. In this paper, we develop empirical likelihood-based methods for the inference …
We propose a new methodology for denoising, variance-stabilizing and normalizing signals whose both mean and variance are parameterized by a single unknown varying parameter, such as Poisson or scaled chi-squared. Key to our methodology is the observation that the signed and square-rooted generalized log-likelihood ratio test for the equality of the local means is approximately and asymptotically distributed as standard normal under the null. We use these test statistics within the Haar wavelet transform at each scale and location, referring to them as the likelihood ratio Haar (LRH) coefficie…
It is an important task in the literature to check whether a fitted autoregressive moving average (ARMA) model is adequate, while the currently used tests may suffer from the size distortion problem when the underlying autoregressive models have low persistence. To fill this gap, this paper proposes two empirical likelihood-based portmanteau tests. The first one is naive but can serve as a benchmark, and the second is for the case with infinite variance innovations. The asymptotic distributions under the null hypothesis are derived under mild moment conditions, and their usefulness is demonstr…
In this paper, we present an algorithm for the fitting of a location-scale variant of the canonical fundamental skew t (CFUST) distribution, a superclass of the restricted and unrestricted skew t-distributions. In recent years, a few versions of the multivariate skew $t$ (MST) model have been put forward, together with various EM-type algorithms for parameter estimation. These formulations adopted either a restricted or unrestricted characterization for their MST densities. In this paper, we examine a natural generalization of these developments, employing the CFUST distribution as the paramet…
Mixed effects models are widely used to describe heterogeneity in a population. A crucial issue when adjusting such a model to data consists in identifying fixed and random effects. From a statistical point of view, it remains to test the nullity of the variances of a given subset of random effects. Some authors have proposed to use the likelihood ratio test and have established its asymptotic distribution in some particular cases. Nevertheless, to the best of our knowledge, no general variance components testing procedure has been fully investigated yet. In this paper, we study the likelihood…
We propose a procedure for testing the linearity of a scalar-on-function regression relationship. To do so, we use the functional generalized additive model (FGAM), a recently developed extension of the functional linear model. For a functional covariate X(t), the FGAM models the mean response as the integral with respect to t of F{X(t),t} where F is an unknown bivariate function. The FGAM can be viewed as the natural functional extension of generalized additive models. We show how the functional linear model can be represented as a simple mixed model nested within the FGAM. Using this represe…
This manuscript develops a general purpose inner-product norm for the Kendall \(τ\) and Spearman's \(ρ\), which operates as an unbiased MLE even in the presence of ties. We derive and prove the strict sub-Gaussianity of the Kemeny norm-space, thereby disproving conclusions developed by both \textcite{kendall1948} and \textcite{diaconis1977} as to the nature of the appropriate, finite sample, probability distribution and test statistics. A non-parametric MLE framework for all bivariate pairs is developed, thereby resolving an hypothesis of \textcite{olkin1994} concerning an exponential multivar…
Accurate knowledge of the null distribution of hypothesis tests is important for valid application of the tests. In previous papers and software, the asymptotic null distribution of likelihood ratio tests for detecting genetic linkage in multivariate variance components models has been stated to be a mixture of chi-square distributions with binomial mixing probabilities. Here we show, by simulation and by theoretical arguments based on the geometry of the parameter space, that all aspects of the previously stated asymptotic null distribution are incorrect--both the binomial mixing probabilitie…
We propose simple formulas of confidence intervals for the Wald statistic, likelihood ratio statistic, and score statistic for a network meta-analysis. In addition, we consider resolutions for concerns that network meta-analyses with a small number of studies cannot hold a nominal confidence level. For a bias adjustment in analyses with a small number of studies, a Bartlett-type adjustment is a well-known method. Many Bartlett-type adjustment-type methods are based on maximum likelihood estimators. However, the network meta-analysis often uses the restricted maximum likelihood estimators that …
In this article we discuss estimation of the common variance of several normal populations with tree order restricted means. We discuss the asymptotic properties of the maximum likelihood estimator of the variance as the number of populations tends to infinity. We consider several cases of various orders of the sample sizes and show that the maximum likelihood estimator of the variance may or may not be consistent or be asymptotically normal.
Maximum likelihood estimation in logistic regression with mixed effects is known to often result in estimates on the boundary of the parameter space. Such estimates, which include infinite values for fixed effects and singular or infinite variance components, can cause havoc to numerical estimation procedures and inference. We introduce an appropriately scaled additive penalty to the log-likelihood function, or an approximation thereof, which penalizes the fixed effects by the Jeffreys' invariant prior for the model with no random effects and the variance components by a composition of negativ…
Asymptotic Theory for Restricted Maximum Likelihood Estimators in Generalized Linear Mixed Models
School of Systems and Computing, University of New South Wales（新南威尔士大学系统计算学院）；UNSW Data Science Hub（新南威尔士大学数据科学中心）；School of Mathematics and Statistics, University of New South Wales（新南威尔士大学数学与统计学院）
The present manuscript studies signal detection by likelihood ratio tests in a number of spiked random matrix models, including but not limited to Gaussian mixtures and spiked Wishart covariance matrices. We work directly with multi-spiked cases in these models and with flexible priors on the signal component that allow dependence across spikes. We derive asymptotic normality for the log-likelihood ratios when the signal-to- noise ratios are below certain thresholds. In addition, we show that the variances of the log-likelihood ratios can be asymptotically decomposed as the sums of those of a …
Most statistical software packages implement numerical strategies for computation of maximum likelihood estimates in random effects models. Little is known, however, about the algebraic complexity of this problem. For the one-way layout with random effects and unbalanced group sizes, we give formulas for the algebraic degree of the likelihood equations as well as the equations for restricted maximum likelihood estimation. In particular, the latter approach is shown to be algebraically less complex. The formulas are obtained by studying a univariate rational equation whose solutions correspond …
It is well-known that in some situations it is not easy to compute the likelihood function as the datasets might be large or the model is too complex. In that contexts composite likelihood, derived by multiplying the likelihoods of subjects of the variables, may be useful. The extension of the classical likelihood ratio test statistics to the framework of composite likelihoods is used as a procedure to solve the problem of testing in the context of composite likelihood. In this paper we introduce and study a new family of test statistics for composite likelihood: Composite ϕ-divergence test st…
Likelihood ratio tests are widely used to test statistical hypotheses about parametric families of probability distributions. If interest is restricted to a subfamily of distributions, then it is natural to inquire if the restricted LRT is superior to the unrestricted LRT. Marden's general LRT conjecture posits that any restriction placed on the alternative hypothesis will increase power. The only published counterexample to this conjecture is rather technical and involves a restriction that maintains the dimension of the alternative. We formulate the dimension-restricted LRT conjecture, which…
Recently, the school of Takemura and Takayama have developed a quite interesting minimization method called holonomic gradient descent method (HGD). It works by a mixed use of Pfaffian differential equation satisfied by an objective holonomic function and an iterative optimization method. They successfully applied the method to several maximum likelihood estimation (MLE) problems, which have been intractable in the past. On the other hand, in statistical models, it is not rare that parameters are constrained and therefore the MLE with constraints has been surely one of fundamental topics in st…
In this paper we address the problem of sensor fusion. We formulate the joint detection problem using a general linear observation model and inter-modality independence assumption for noises. We derive the fusion architecture based on the generalized likelihood ratio principle and calculate the expressions for the distributions of the test statistic under the signal present and the null hypotheses. To obtain these results we develop a methodology for the joint detection algorithm analysis based on the theory of the Meijer G-function.
This paper develops a bias correction scheme for a multivariate heteroskedastic errors-in-variables model. The applicability of this model is justified in areas such as astrophysics, epidemiology and analytical chemistry, where the variables are subject to measurement errors and the variances vary with the observations. We conduct Monte Carlo simulations to investigate the performance of the corrected estimators. The numerical results show that the bias correction scheme yields nearly unbiased estimates. We also give an application to a real data set.
Random-effects models are frequently used to synthesise information from different studies in meta-analysis. While likelihood-based inference is attractive both in terms of limiting properties and of implementation, its application in random-effects meta-analysis may result in misleading conclusions, especially when the number of studies is small to moderate. The current paper shows how methodology that reduces the asymptotic bias of the maximum likelihood estimator of the variance component can also substantially improve inference about the mean effect size. The results are derived for the mo…
We propose a new method for the Maximum Likelihood Estimator (MLE) of nonlinear mixed effects models when the variance matrix of Gaussian random effects has a prescribed pattern of zeros (PPZ). The method consists in coupling the recently developed Iterative Conditional Fitting (ICF) algorithm with the Expectation Maximization (EM) algorithm. It provides positive definite estimates for any sample size, and does not rely on any structural assumption on the PPZ. It can be easily adapted to many versions of EM.
Parametric inference posits a statistical model that is a specified family of probability distributions. Restricted inference, e.g., restricted likelihood ratio testing, attempts to exploit the structure of a statistical submodel that is a subset of the specified family. We consider the problem of testing a simple hypothesis against alternatives from such a submodel. In the case of an unknown submodel, it is not clear how to realize the benefits of restricted inference. To do so, we first construct information tests that are locally asymptotically equivalent to likelihood ratio tests. Informat…
Variance components estimation and mixed model analysis are central themes in statistics with applications in numerous scientific disciplines. Despite the best efforts of generations of statisticians and numerical analysts, maximum likelihood estimation and restricted maximum likelihood estimation of variance component models remain numerically challenging. Building on the minorization-maximization (MM) principle, this paper presents a novel iterative algorithm for variance components estimation. MM algorithm is trivial to implement and competitive on large data problems. The algorithm readily…
Recently the shape-restricted inference has gained popularity in statistical and econometric literature in order to relax the linear or quadratic covariate effect in regression analyses. The typical shape-restricted covariate effect includes monotonic increasing, decreasing, convexity or concavity. In this paper, we introduce the shape-restricted inference to the celebrated Cox regression model (SR-Cox), in which the covariate response is modeled as shape-restricted additive functions. The SR-Cox regression approximates the shape-restricted functions using a spline basis expansion with data dr…
In this paper we consider two closely related problems : estimation of eigenvalues and eigenfunctions of the covariance kernel of functional data based on (possibly) irregular measurements, and the problem of estimating the eigenvalues and eigenvectors of the covariance matrix for high-dimensional Gaussian vectors. In Peng and Paul (2007), a restricted maximum likelihood (REML) approach has been developed to deal with the first problem. In this paper, we establish consistency and derive rate of convergence of the REML estimator for the functional data case, under appropriate smoothness conditi…
We expect that some observers in perceptual signal detection experiments, such as radiologists, will make rational decisions, and therefore ratings from those observers are expected to form a convex ROC curve. However, measured and published curves are often not convex. This article examines the convexity-constrained nonparametric maximum likelihood estimator of the ROC curve given by Lloyd (2002). Like Lloyd we use the Pool Adjacent Violator Algorithm (PAVA) to construct the estimate of the convex curve. We present a direct proof that this estimate is a convex hull of the empirical ROC curve.…
Confidence intervals based on penalized maximum likelihood estimators such as the LASSO, adaptive LASSO, and hard-thresholding are analyzed. In the known-variance case, the finite-sample coverage properties of such intervals are determined and it is shown that symmetric intervals are the shortest. The length of the shortest intervals based on the hard-thresholding estimator is larger than the length of the shortest interval based on the adaptive LASSO, which is larger than the length of the shortest interval based on the LASSO, which in turn is larger than the standard interval based on the ma…
New York University（纽约大学）；Princeton University（普林斯顿大学）；University of Helsinki（赫尔辛基大学）；University of Geneva（日内瓦大学）
The fate of scientific hypotheses often relies on the ability of a computational model to explain the data, quantified in modern statistical approaches by the likelihood function. The log-likelihood is the key element for parameter estimation and model evaluation. However, the log-likelihood of complex models in fields such as computational biology and neuroscience is often intractable to compute analytically or numerically. In those cases, researchers can often only estimate the log-likelihood by comparing observed data with synthetic observations generated by model simulations. Standard tech…
In this paper we propose a new approach for sequential monitoring of a parameter of a $d$-dimensional time series, which can be estimated by approximately linear functionals of the empirical distribution function. We consider a closed-end-method, which is motivated by the likelihood ratio test principle and compare the new method with two alternative procedures. We also incorporate self-normalization such that estimation of the long-run variance is not necessary. We prove that for a large class of testing problems the new detection scheme has asymptotic level $α$ and is consistent. The asympto…
A linear Gaussian state-space smoothing algorithm is presented for estimation of derivatives from a sequence of noisy measurements. The algorithm uses numerically stable square-root formulas, can handle simultaneous independent measurements and non-equally spaced abscissas, and can compute state estimates at points between the data abscissas. The state space model's parameters, including driving noise intensity, measurement variance, and initial state, are determined from the given data sequence using maximum likelihood estimation computed using a expectation maximisation iteration. In tests w…
In finite mixtures of location-scale distributions, if there is no constraint on the parameters then the maximum likelihood estimate does not exist. But when the ratios of the scale parameters are restricted appropriately, the maximum likelihood estimate exists. We prove that the maximum likelihood estimator (MLE) is strongly consistent, if the ratios of the scale parameters are restricted from below by $\exp(-n^{d}), 0 < d < 1 $, where $n$ is the sample size.
Statistical multispecies models of multiarea marine ecosystems use a variety of data sources to estimate parameters using composite or weighted likelihood functions with associated weighting issues and questions on how to obtain variance estimates. Regardless of the method used to obtain point estimates, a method is needed for variance estimation. A bootstrap technique is introduced for the evaluation of uncertainty in such models, taking into account inherent spatial and temporal correlations in the data sets thus avoiding many model--specification issues, which are commonly transferred as as…
This paper deals with the problems of consistence and strong consistence of the maximum likelihood estimators of the mean and variance of the drift fractional Brownian motions observed at discrete time instants. A central limit theorem for these estimators is also obtained by using the Malliavin calculus.
We employ a parameter-free distribution estimation framework where estimators are random distributions and utilize the Kullback-Leibler (KL) divergence as a loss function. Wu and Vos [J. Statist. Plann. Inference 142 (2012) 1525-1536] show that when an estimator obtained from an i.i.d. sample is viewed as a random distribution, the KL risk of the estimator decomposes in a fashion parallel to the mean squared error decomposition when the estimator is a real-valued random variable. In this paper, we explore how conditional versions of distribution expectation ($E^{\dagger}$) can be defined so th…
This work studies the statistical properties of the maximum penalized likelihood approach in a semi-parametric framework. We recall the penalized likelihood approach for estimating a function and review some asymptotic results. We investigate the properties of two estimators of the variance of maximum penalized likelihood estimators: sandwich estimator and a Bayesian estimator. The coverage rates of confidence intervals based on these estimators are studied through a simulation study of survival data. In a first simulation the coverage rates for the survival function and the hazard function ar…
In a finite mixture of location-scale distributions maximum likelihood estimator does not exist because of the unboundedness of the likelihood function when the scale parameter of some mixture component approaches zero. In order to study the strong consistency of maximum likelihood estimator, we consider the case that the scale parameters of the component distributions are restricted from below by $c_n$, where $c_n$ is a sequence of positive real numbers which tend to zero as the sample size $n$ increases. We prove that under mild regularity conditions maximum likelihood estimator is strongly …
Zhang (2019) presented a general estimation approach based on the Gaussian distribution for general parametric models where the likelihood of the data is difficult to obtain or unknown, but the mean and variance-covariance matrix are known. Castilla and Zografos (2021) extended the method to density power divergence-based estimators, which are more robust than the likelihood-based Gaussian estimator against data contamination. In this paper we introduce the restricted minimum density power divergence Gaussian estimator (MDPDGE) and study its main asymptotic properties. Also, we examine it robu…
Indirect inference estimators (i.e., simulation-based minimum distance estimators) in a parametric model that are based on auxiliary non-parametric maximum likelihood density estimators are shown to be asymptotically normal. If the parametric model is correctly specified, it is furthermore shown that the asymptotic variance-covariance matrix equals the inverse of the Fisher-information matrix. These results are based on uniform-in-parameters convergence rates and a uniform-in-parameters Donsker-type theorem for non-parametric maximum likelihood density estimators.
There is a difficulty in finding an estimate of variance of the profile likelihood estimator in the joint model of longitudinal and survival data. We solve the difficulty by introducing the ``statistical generalized derivative''. The derivative is used to show the asymptotic normality of the estimator without assuming the second derivative of the density function in the model exists.
The Maximum Likelihood (ML) and Cross Validation (CV) methods for estimating covariance hyper-parameters are compared, in the context of Kriging with a misspecified covariance structure. A two-step approach is used. First, the case of the estimation of a single variance hyper-parameter is addressed, for which the fixed correlation function is misspecified. A predictive variance based quality criterion is introduced and a closed-form expression of this criterion is derived. It is shown that when the correlation function is misspecified, the CV does better compared to ML, while ML is optimal whe…
In this paper, we present a maximum likelihood method for estimating the parameters of a univariate Hawkes process with self-excitation or inhibition. Our work generalizes techniques and results that were restricted to the self-exciting scenario. The proposed estimator is implemented for the classical exponential kernel and we show that, in the inhibition context, our procedure provides more accurate estimations than current alternative approaches.
Maximum likelihood estimation of linear functionals in the inverse problem of deconvolution is considered. Given observations of a random sample from a distribution $P_0\equiv P_{F_0}$ indexed by a (potentially infinite-dimensional) parameter $F_0$, which is the distribution of the latent variable in a standard additive Laplace measurement error model, one wants to estimate a linear functional of $F_0$. Asymptotically efficient maximum likelihood estimation (MLE) of integral linear functionals of the mixing distribution $F_0$ in a convolution model with the Laplace kernel density is investigat…
Generative adversarial networks are a novel method for statistical inference that have achieved much empirical success; however, the factors contributing to this success remain ill-understood. In this work, we attempt to analyze generative adversarial learning -- that is, statistical inference as the result of a game between a generator and a discriminator -- with the view of understanding how it differs from classical statistical inference solutions such as maximum likelihood inference and the method of moments. Specifically, we provide a theoretical characterization of the distribution infer…
We consider the problem of recovering the unknown noise variance in the linear regression model. To estimate the nuisance (a vector of regression coefficients) we use a family of spectral regularisers of the maximum likelihood estimator. The noise estimation is based on the adaptive normalisation of the squared error. We derive the upper bound for the concentration of the proposed method around the ideal estimator (the case of zero nuisance).
