The Behrens-Fisher problem concerns testing the equality of the means of two normal populations with possibly different variances. The null hypothesis in this problem induces a statistical model for which the likelihood function may have more than one local maximum. We show that such multimodality contradicts the null hypothesis in the sense that if this hypothesis is true then the probability of multimodality converges to zero when both sample sizes tend to infinity. Additional results include a finite-sample bound on the probability of multimodality under the null and asymptotics for the probability of multimodality under the alternative.
In this paper, the $k$ sample Behrens-Fisher problem is investigated in high dimensional setting. We propose a new test statistic and demonstrate that the proposed test is expected to have more powers than some existing test especially when sample sizes are unbalanced. We provide theoretical investigation as well as numerical studies on both sizes and powers of the proposed tests and existing test. Both theoretical comparison of the asymptotic power functions and numerical studies show that the proposed test tends to have more powers than existing test in many cases of unbalanced sample sizes.
The Behrens-Fisher problem is a well-known hypothesis testing problem in statistics concerning two-sample mean comparison. In this article, we confirm one conjecture in Eaton and Olshen (1972), which provides stochastic bounds for the multivariate Behrens-Fisher test statistic under the null hypothesis. We also extend their results on the stochastic ordering of random quotients to the arbitrary finite dimensional case. This work can also be seen as a generalization of Hsu (1938) that provided the bounds for the univariate Behrens-Fisher problem. The results obtained in this article can be used…
The Behrens-Fisher Problem is a classical statistical problem. It is to test the equality of the means of two normal populations using two independent samples, when the equality of the population variances is unknown. Linnik (1968) has shown that this problem has no exact fixed-level tests based on the complete sufficient statistics. However, exact conventional solutions based on other statistics and approximate solutions based the complete sufficient statistics do exist. Existing methods are mainly asymptotic tests, and usually don't perform well when the variances or sample sizes differ a lo…
In this paper we provide a provably convergent algorithm for the multivariate Gaussian Maximum Likelihood version of the Behrens--Fisher Problem. Our work builds upon a formulation of the log-likelihood function proposed by Buot and Richards \citeBR. Instead of focusing on the first order optimality conditions, the algorithm aims directly for the maximization of the log-likelihood function itself to achieve a global solution. Convergence proof and complexity estimates are provided for the algorithm. Computational experiments illustrate the applicability of such methods to high-dimensional data…
Let $μ$ be a $p$-dimensional vector, and let $Σ_1$ and $Σ_2$ be $p \times p$ positive definite covariance matrices. On being given random samples of sizes $N_1$ and $N_2$ from independent multivariate normal populations $N_p(μ,Σ_1)$ and $N_p(μ,Σ_2)$, respectively, the Behrens-Fisher problem is to solve the likelihood equations for estimating the unknown parameters $μ$, $Σ_1$, and $Σ_2$. We shall prove that for $N_1, N_2 > p$ there are, almost surely, exactly $2p+1$ complex solutions of the likelihood equations. For the case in which $p = 2$, we utilize Monte Carlo simulation to estimate the re…
This paper is concerned with the problem of comparing the population means of two groups of independent observations. An approximate randomization test procedure based on the test statistic of Chen and Qin (2010) is proposed. The asymptotic behavior of the test statistic as well as the randomized statistic is studied under weak conditions. In our theoretical framework, observations are not assumed to be identically distributed even within groups. No condition on the eigenstructure of the covariance matrices is imposed. And the sample sizes of the two groups are allowed to be unbalanced. Under …
For in vivo research experiments with small sample sizes and available historical data, we propose a sequential Bayesian method for the Behrens-Fisher problem. We consider it as a model choice question with two models in competition: one for which the two expectations are equal and one for which they are different. The choice between the two models is performed through a Bayesian analysis, based on a robust choice of combined objective and subjective priors, set on the parameters space and on the models space. Three steps are necessary to evaluate the posterior probability of each model using …
The problem of testing the equality of mean vectors for high-dimensional data has been intensively investigated in the literature. However, most of the existing tests impose strong assumptions on the underlying group covariance matrices which may not be satisfied or hardly be checked in practice. In this article, an F-type test for two-sample Behrens--Fisher problems for high-dimensional data is proposed and studied. When the two samples are normally distributed and when the null hypothesis is valid, the proposed F-type test statistic is shown to be an F-type mixture, a ratio of two independen…
Fisher's Method of Maximum Likelihood is shown to be a procedure for the construction of likelihood intervals or regions, instead of a procedure of point estimation. Based on Fisher's articles and books it is justified that by estimation Fisher meant the construction of likelihood intervals or regions from appropriate likelihood function and that an estimate is a statistic, that is, a function from a sample space to a parameter space such that the likelihood function obtained from the sampling distribution of the statistic at the observed value of the statistic is used to construct likelihood …
Maximum likelihood estimates and corresponding confidence regions of the estimates are commonly used in statistical inference. In practice, people often construct approximate confidence regions with the Fisher information at given sample data based on the asymptotic normal distribution of the MLE (maximum likelihood estimate). Two common Fisher information matrices (FIMs, for multivariate parameters) are the observed FIM (the Hessian matrix of negative log-likelihood function) and the expected FIM (the expectation of the observed FIM). In this article, we prove that under certain conditions an…
Maximum likelihood estimation is a popular method in statistical inference. As a way of assessing the accuracy of the maximum likelihood estimate (MLE), the calculation of the covariance matrix of the MLE is of great interest in practice. Standard statistical theory shows that the normalized MLE is asymptotically normally distributed with covariance matrix being the inverse of the Fisher information matrix (FIM) at the unknown parameter. Two commonly used estimates for the covariance of the MLE are the inverse of the observed FIM (the same as the inverse Hessian of the negative log-likelihood)…
Suppose that $k$ series, all having the same autocorrelation function, are observed in parallel at $n$ points in time or space. From a single series of moderate length, the autocorrelation parameter $β$ can be estimated with limited accuracy, so we aim to increase the information by formulating a suitable model for the joint distribution of all series. Three Gaussian models of increasing complexity are considered, two of which assume that the series are independent. This paper studies the rate at which the information for $β$ accumulates as $k$ increases, possibly even beyond $n$. The profile …
A number of discrete time, finite population size models in genetics describing the dynamics of allele frequencies are known to converge (subject to suitable scaling) to a diffusion process in the infinite population limit, termed the Wright-Fisher diffusion. In this article we show that the diffusion is ergodic uniformly in the selection and mutation parameters, and that the measures induced by the solution to the stochastic differential equation are uniformly locally asymptotically normal. Subsequently these two results are used to analyse the statistical properties of the Maximum Likelihood…
We study asymptotic behavior of maximum likelihood estimator for a time inhomogeneous diffusion process given by a SDE $dX_t=αb(t)X_t dt + σ(t) dB_t$, $t\in[0,T)$, with a parameter $α\in R$, where $T\in(0,\infty]$ and $(B_t)_{t\in[0,T)}$ is a standard Wiener process. We formulate sufficient conditions under which the MLE of $α$ normalized by Fisher information converges to the limit distribution of Dickey-Fuller statistics. Next we study a SDE $dY_t=αb(t)a(Y_t) dt + σ(t) dB_t$, $t\in[0,T)$, with a perturbed drift satisfying $a(x)=x+O(1+|x|^γ)$ with some $γ\in[0,1)$. We give again sufficient co…
If the log likelihood is approximately quadratic with constant Hessian, then the maximum likelihood estimator (MLE) is approximately normally distributed. No other assumptions are required. We do not need independent and identically distributed data. We do not need the law of large numbers (LLN) or the central limit theorem (CLT). We do not need sample size going to infinity or anything going to infinity. Presented here is a combination of Le Cam style theory involving local asymptotic normality (LAN) and local asymptotic mixed normality (LAMN) and Cramér style theory involving derivatives and…
Universite Paris Dauphine-PSL（巴黎第九大学-巴黎文理研究大学）；Universite Paris 1 Pantheon-Sorbonne（巴黎第一大学-先贤祠-索邦大学）
Mixtures of von Mises-Fisher distributions can be used to cluster data on the unit hypersphere. This is particularly adapted for high-dimensional directional data such as texts. We propose in this article to estimate a von Mises mixture using a l 1 penalized likelihood. This leads to sparse prototypes that improve clustering interpretability. We introduce an expectation-maximisation (EM) algorithm for this estimation and explore the trade-off between the sparsity term and the likelihood one with a path following algorithm. The model's behaviour is studied on simulated data and, we show the adv…
Motivated by studying asymptotic properties of the maximum likelihood estimator (MLE) in stochastic volatility (SV) models, in this paper we investigate likelihood estimation in state space models. We first prove, under some regularity conditions, there is a consistent sequence of roots of the likelihood equation that is asymptotically normal with the inverse of the Fisher information as its variance. With an extra assumption that the likelihood equation has a unique root for each $n$, then there is a consistent sequence of estimators of the unknown parameters. If, in addition, the supremum of…
The aim of this note is to state a couple of general results about the properties of the penalized maximum likelihood estimators (pMLE) and of the posterior distribution for parametric models in a non-asymptotic setup and for possibly large or even infinite parameter dimension. We consider a special class of stochastically linear smooth (SLS) models satisfying two major conditions: the stochastic component of the log-likelihood is linear in the model parameter, while the expected log-likelihood is a smooth function. The main results simplify a lot if the expected log-likelihood is concave. For…
We introduce some new classes of unimodal rotational invariant directional distributions, which generalize von Mises-Fisher distribution. We propose three types of distributions, one of which represents axial data. For each new type we provide formulae and short computational study of parameter estimators by the method of moments and the method of maximum likelihood. The main goal of the paper is to develop the goodness of fit test to detect that sample entries follow one of the introduced generalized von Mises--Fisher distribution based on the maximum entropy principle. We use $k$th nearest n…
Gaussian mixture models form a flexible and expressive parametric family of distributions that has found applications in a wide variety of applications. Unfortunately, fitting these models to data is a notoriously hard problem from a computational perspective. Currently, only moment-based methods enjoy theoretical guarantees while likelihood-based methods are dominated by heuristics such as Expectation-Maximization that are known to fail in simple examples. In this work, we propose a new algorithm to compute the nonparametric maximum likelihood estimator (NPMLE) in a Gaussian mixture model. Ou…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；Purdue University（普渡大学）
We consider 1-dimensional location estimation, where we estimate a parameter $λ$ from $n$ samples $λ+ η_i$, with each $η_i$ drawn i.i.d. from a known distribution $f$. For fixed $f$ the maximum-likelihood estimate (MLE) is well-known to be optimal in the limit as $n \to \infty$: it is asymptotically normal with variance matching the Cramér-Rao lower bound of $\frac{1}{n\mathcal{I}}$, where $\mathcal{I}$ is the Fisher information of $f$. However, this bound does not hold for finite $n$, or when $f$ varies with $n$. We show for arbitrary $f$ and $n$ that one can recover a similar theory based on…
Motivated by applications to single-particle cryo-electron microscopy (cryo-EM), we study several problems of function estimation in a high noise regime, where samples are observed after random rotation and possible linear projection of the function domain. We describe a stratification of the Fisher information eigenvalues according to transcendence degrees of graded pieces of the algebra of group invariants, and we relate critical points of the log-likelihood landscape to a sequence of moment optimization problems, extending previous results for a discrete rotation group without projections. …
