In a mixed generalized linear model, the goal is to learn multiple signals from unlabeled observations: each sample comes from exactly one signal, but it is not known which one. We consider the prototypical problem of estimating two statistically independent signals in a mixed generalized linear model with Gaussian covariates. Spectral methods are a popular class of estimators which output the top two eigenvectors of a suitable data-dependent matrix. However, despite the wide applicability, their design is still obtained via heuristic considerations, and the number of samples $n$ needed to guarantee recovery is super-linear in the signal dimension $d$. In this paper, we develop exact asymptotics on spectral methods in the challenging proportional regime in which $n, d$ grow large and their ratio converges to a finite constant. This allows us optimize the design of the spectral method, and combine it with a simple linear estimator, to minimize the estimation error. Our characterization exploits a mix of tools from random matrices, free probability and the theory of approximate message passing algorithms. Numerical simulations for mixed linear regression and phase retrieval demonstrate the advantage enabled by our analysis over existing designs of spectral methods.
We extend a recently established asymptotic normality theorem for generalized linear mixed models to include the dispersion parameter. The new results show that the maximum likelihood estimators of all model parameters have asymptotically normal distributions with asymptotic mutual independence between fixed effects, random effects covariance and dispersion parameters. The dispersion parameter maximum likelihood estimator has a particularly simple asymptotic distribution which enables straightforward valid likelihood-based inference.
We study sufficient conditions for local asymptotic mixed normality. We weaken the sufficient conditions in Theorem 1 of Jeganathan (Sankhya Ser. A 1982) so that they can be applied to a wider class of statistical models including a jump-diffusion model. Moreover, we show that local asymptotic mixed normality of a statistical model generated by approximated transition density functions is implied for the original model. Together with density approximation by means of thresholding techniques, we show local asymptotic normality for a statistical model of discretely observed jump-diffusion proces…
In $M$-estimation under standard asymptotics, the weak convergence combined with the polynomial type large deviation estimate of the associated statistical random field Yoshida (2011) provides us with not only the asymptotic distribution of the associated $M$-estimator but also the convergence of its moments, the latter playing an important role in theoretical statistics. In this paper, we study the above program for statistical random fields of multiple and also possibly mixed-rates type in the sense of Radchenko (2008) where the associated statistical random fields may be non-differentiable …
Asymptotic Theory for Restricted Maximum Likelihood Estimators in Generalized Linear Mixed Models
School of Systems and Computing, University of New South Wales（新南威尔士大学系统计算学院）；UNSW Data Science Hub（新南威尔士大学数据科学中心）；School of Mathematics and Statistics, University of New South Wales（新南威尔士大学数学与统计学院）
We consider a linear mixed-effects model with a clustered structure, where the parameters are estimated using maximum likelihood (ML) based on possibly unbalanced data. Inference with this model is typically done based on asymptotic theory, assuming that the number of clusters tends to infinity with the sample size. However, when the number of clusters is fixed, classical asymptotic theory developed under a divergent number of clusters is no longer valid and can lead to erroneous conclusions. In this paper, we establish the asymptotic properties of the ML estimators of random-effects parameter…
We consider three problems in high-dimensional Gaussian linear mixed models. Without any assumptions on the design for the fixed effects, we construct an asymptotic $F$-statistic for testing whether a collection of random effects is zero, derive an asymptotic confidence interval for a single random effect at the parametric rate $\sqrt{n}$, and propose an empirical Bayes estimator for a part of the mean vector in ANOVA type models that performs asymptotically as well as the oracle Bayes estimator. We support our results with numerical simulations and provide comparisons with oracle estimators. …
We consider a re-sampling scheme for estimation of the population parameters in the mixed effects nonlinear regression models of the type use for example in clinical pharmacokinetics, say. We provide an estimation procedure which {\it recycles}, via random weighting, the relevant two-stage parameters estimates to construct consistent estimates of the sampling distribution of the various estimates. We establish the asymptotic consistency and asymptotic normality of the resampled estimates and demonstrate the applicability of the {\it recycling} approach in a small simulation study and via examp…
Motivated by better modeling of intra-individual variability in longitudinal data, we propose a class of location-scale mixed effects models, in which the data of each individual is modeled by a parameter-varying generalized hyperbolic distribution. We first study the local maximum-likelihood asymptotics and reveal the instability in the numerical optimization of the log-likelihood. Then, we construct an asymptotically efficient estimator based on the Newton-Raphson method based on the original log-likelihood function with the initial estimator being naive least-squares-type. Numerical experim…
This paper is devoted to parameter estimation of the mixed fractional Ornstein-Uhlenbeck process with a drift. Large sample asymptotical properties of the Maximum Likelihood Estimator is deduced using the Laplace transform computations or the Cameron-Martin formula with extra part from \cite{CK19}
We obtain central limit theorems for stationary random fields employing a novel measure of dependence called $θ$-lex weak dependence. We show that this dependence notion is more general than strong mixing, i.e., it applies to a broader class of models. Moreover, we discuss hereditary properties for $θ$-lex and $η$-weak dependence and illustrate the possible applications of the weak dependence notions to the study of the asymptotic properties of stationary random fields. Our general results apply to mixed moving average fields (MMAF in short) and ambit fields. We show general conditions such th…
We study minimax density estimation on the product space $\mathbb{R}^{d_1}\times\mathbb{R}^{d_2}$. We consider $L^p$-risk for probability density functions defined over regularity spaces that allow for different level of smoothness in each of the variables. Precisely, we study probabilities on Sobolev spaces with dominating mixed-smoothness. We provide the rate of convergence that is optimal even for the classical Sobolev spaces.
This paper introduces a general framework for estimating variance components in the linear mixed models via general unbiased estimating equations, which include some well-used estimators such as the restricted maximum likelihood estimator. We derive the asymptotic covariance matrices and second-order biases under general estimating equations without assuming the normality of the underlying distributions and identify a class of second-order unbiased estimators of variance components. It is also shown that the asymptotic covariance matrices and second-order biases do not depend on whether the re…
Based on the negative binomial model for the duration of wet periods measured in days, an asymptotic approximation is proposed for the distribution of the maximum daily precipitation volume within a wet period. This approximation has the form of a scale mixture of the Frechet distribution with the gamma mixing distribution and coincides with the distribution of a positive power of a random variable having the Snedecor-Fisher distribution. The proof of this result is based on the representation of the negative binomial distribution as a mixed geometric (and hence, mixed Poisson) distribution an…
Properties of mixed fractional Brownian motion has been discussed by Cheridito (2001) and Zili (2006). We have proposed an estimator of volatility parameter for a model driven by MFBM. In our article we have shown that the estimator has some desirable asymptotic properties.
We study sufficient conditions for a local asymptotic mixed normality property of statistical models. We develop a scheme with the $L^2$ regularity condition proposed by Jeganathan [\textit{Sankhya Ser. A} \textbf{44} (1982) 173--212] so that it is applicable to high-frequency observations of stochastic processes. Moreover, by combining with Malliavin calculus techniques by Gobet [\textit{Bernoulli} \textbf{7} (2001) 899--912, 2001], we introduce tractable sufficient conditions for smooth observations in the Malliavin sense, which do not require Aronson-type estimates of the transition density…
This paper is a survey of recent contributions on estimation in stochastic differential equations with mixed-effects. These models involve N stochastic differential equations with common drift and diffusion functions but random parameters that allow for differences between processes. The main objective is to estimate the distribution of the random effects and possibly other fixed parameters that are common to the N processes. While many algorithms have been proposed, the theoretical aspects related to estimation have been little studied. This review article focuses only on theoretical inferenc…
We consider a stochastic process model with time trend and measurement error. We establish consistency and derive the limiting distributions of the maximum likelihood (ML) estimators of the covariance function parameters under a general asymptotic framework, including both the fixed domain and the increasing domain frameworks, even when the time trend model is misspecified or its complexity increases with the sample size. In particular, the convergence rates of the ML estimators are thoroughly characterized in terms of the growing rate of the domain and the degree of model misspecification/com…
We study the principal components of covariance estimators in multivariate mixed-effects linear models. We show that, in high dimensions, the principal eigenvalues and eigenvectors may exhibit bias and aliasing effects that are not present in low-dimensional settings. We derive the first-order limits of the principal eigenvalue locations and eigenvector projections in a high-dimensional asymptotic framework, allowing for general population spectral distributions for the random effects and extending previous results from a more restrictive spiked model. Our analysis uses free probability techni…
In this paper, we study the asymptotic behaviors of the extreme of mixed skew-t distribution. We considered limits on distribution and density of maximum of mixed skew-t distribution under linear and power normalization, and further derived their higher-order expansions, respectively. Examples are given to support our findings.
