The maximum ${\log}_q$ likelihood estimation method is a generalization of the known maximum $\log$ likelihood method to overcome the problem for modeling non-identical observations (inliers and outliers). The parameter $q$ is a tuning constant to manage the modeling capability. Weibull is a flexible and popular distribution for problems in engineering. In this study, this method is used to estimate the parameters of Weibull distribution when non-identical observations exist. Since the main idea is based on modeling capability of objective function $ρ(x;\boldsymbolθ)=\log_q\big[f(x;\boldsymbolθ)\big]$, we observe that the finiteness of score functions cannot play a role in the robust estimation for inliers. The properties of Weibull distribution are examined. In the numerical experiment, the parameters of Weibull distribution are estimated by $\log_q$ and its special form, $\log$, likelihood methods if the different designs of contamination into underlying Weibull distribution are applied. The optimization is performed via genetic algorithm. The modeling competence of $ρ(x;\boldsymbolθ)$ and insensitiveness to non-identical observations are observed by Monte Carlo simulation. The value of $q$ can be chosen by use of the mean squared error in simulation and the $p$-value of Kolmogorov-Smirnov test statistic used for evaluation of fitting competence. Thus, we can overcome the problem about determining of the value of $q$ for real data sets.
Handling missing values plays an important role in the analysis of survival data, especially, the ones marked by cure fraction. In this paper, we discuss the properties and implementation of stochastic approximations to the expectation-maximization (EM) algorithm to obtain maximum likelihood (ML) type estimates in situations where missing data arise naturally due to right censoring and a proportion of individuals are immune to the event of interest. A flexible family of three parameter exponentiated-Weibull (EW) distributions is assumed to characterize lifetimes of the non-immune individuals a…
The peculiar properties of the Inverse Weibull (IW) distribution are shown. It is proven that the IW distribution is one of the few models having upside- down bathtub (UBT) shaped hazard function. Three real and typical de generative mechanisms, which lead exactly to the IW random variable, are formulated. So a new approach to proper application of this relatively unknown survival model is supported. However, we consider also the case in which any knowledge about generative mechanism is unavailable. In this hypothesis, we study a procedure based on the Anderson-Darling statistic and log-likeli…
Recently, Lee and Cha (2015, `On two generalized classes of discrete bivariate distributions', {\it American Statistician}, 221 - 230) proposed two general classes of discrete bivariate distributions. They have discussed some general properties and some specific cases of their proposed distributions. In this paper we have considered one model, namely bivariate discrete Weibull distribution, which has not been considered in the literature yet. The proposed bivariate discrete Weibull distribution is a discrete analogue of the Marshall-Olkin bivariate Weibull distribution. We study various proper…
For the basic maximum likelihood estimating function of the two parameters Weibull distribution, a simple proof on its global monotonicity is given to ensure the existence and uniqueness of its solution. The boundary of the function's first-order derivative is defined based on its scale-free property. With a bounded derivative, the possible range of the root of this function can be determined. A novel root-finding algorithm employing these established results is proposed accordingly, its convergence is proved analytically as well. Compared with other typical algorithms for this problem, the ef…
We develop two new classes of tests for the Weibull distribution based on Stein's method. The proposed tests are applied in the full sample case as well as in the framework of random right censoring. We investigate the finite sample performance of the new tests using a comprehensive Monte Carlo study. In both the absence and presence of censoring, it is found that the newly proposed classes of tests outperform competing tests against the majority of the distributions considered. In the cases where censoring is present we consider various censoring distributions. Some remarks on the asymptotic …
We describe a Monte Carlo method to approximate the maximum likelihood estimate (MLE), when there are missing data and the observed data likelihood is not available in closed form. This method uses simulated missing data that are independent and identically distributed and independent of the observed data. Our Monte Carlo approximation to the MLE is a consistent and asymptotically normal estimate of the minimizer $θ^*$ of the Kullback--Leibler information, as both Monte Carlo and observed data sample sizes go to infinity simultaneously. Plug-in estimates of the asymptotic variance are provided…
The beta Weibull distribution was introduced by Famoye et al. (2005) and studied by these authors. However, they do not give explicit expressions for the moments. We now derive explicit closed form expressions for the cumulative distribution function and for the moments of this distribution. We also give an asymptotic expansion for the moment generating function. Further, we discuss maximum likelihood estimation and provide formulae for the elements of the Fisher information matrix. We also demonstrate the usefulness of this distribution on a real data set.
When an unbiased estimator of the likelihood is used within a Metropolis--Hastings chain, it is necessary to trade off the number of Monte Carlo samples used to construct this estimator against the asymptotic variances of averages computed under this chain. Many Monte Carlo samples will typically result in Metropolis--Hastings averages with lower asymptotic variances than the corresponding Metropolis--Hastings averages using fewer samples. However, the computing time required to construct the likelihood estimator increases with the number of Monte Carlo samples. Under the assumption that the d…
In this review, we address the use of Monte Carlo methods for approximating definite integrals of the form $Z = \int L(x) d P(x)$, where $L$ is a target function (often a likelihood) and $P$ a finite measure. We present vertical-likelihood Monte Carlo, which is an approach for designing the importance function $g(x)$ used in importance sampling. Our approach exploits a duality between two random variables: the random draw $X \sim g$, and the corresponding random likelihood ordinate $Y\equiv L(X)$ of the draw. It is natural to specify $g(x)$ and ask: what is the the implied distribution of $Y$?…
We consider Monte Carlo approximations to the maximum likelihood estimator in models with intractable norming constants. This paper deals with adaptive Monte Carlo algorithms, which adjust control parameters in the course of simulation. We examine asymptotics of adaptive importance sampling and a new algorithm, which uses resampling and MCMC. This algorithm is designed to reduce problems with degeneracy of importance weights. Our analysis is based on martingale limit theorems. We also describe how adaptive maximization algorithms of Newton-Raphson type can be combined with the resampling techn…
This paper considers the three-parameter exponentiated Weibull family under type II censoring. It first graphically illustrates the shape property of the hazard function. Then, it proposes a simple algorithm for computing the maximum likelihood estimator and derives the Fisher information matrix. The latter one is represented through a single integral in terms of hazard function, hence it solves the problem of computation difficulty in constructing inference for the maximum likelihood estimator. Real data analysis is conducted to illustrate the effect of censoring rate on the maximum likelihoo…
In this paper we study a broad class of distribution functions which is defined by means of reflected generalized beta distribution. This class includes that of Beta-generated distribution as a special case. In particular, we use this class to extend the Inverse Weibull distribution in order to obtain the Reflected Generalized Beta Inverse Weibull Distribution.For this new distribution, moments, entropy, order statistics and a reliability measure are derived. The link between the Inverse Weibull and the Dagum distribution is generalized. Then the maximum likelihood estimators of the parameters…
We propose sequential Monte Carlo based algorithms for maximum likelihood estimation of the static parameters in hidden Markov models with an intractable likelihood using ideas from approximate Bayesian computation. The static parameter estimation algorithms are gradient based and cover both offline and online estimation. We demonstrate their performance by estimating the parameters of three intractable models, namely the alpha-stable distribution, g-and-k distribution, and the stochastic volatility model with alpha-stable returns, using both real and synthetic data.
Statistical inference for discretely observed jump-diffusion processes is a complex problem which motivates new methodological challenges. Thus existing approaches invariably resort to time-discretisations which inevitably lead to approximations in inference. In this paper, we give the first general collection of methodologies for exact (in this context meaning discretisation-free) likelihood-based inference for discretely observed finite activity jump-diffusions. The only sources of error involved are Monte Carlo error and convergence of EM or MCMC algorithms. We shall introduce both frequent…
The Weibull distribution, with shape parameter $k>0$ and scale parameter $λ>0$, is one of the most popular parametric distributions in survival analysis with complete or censored data. Although inference of the parameters of the Weibull distribution is commonly done through maximum likelihood, it is well established that the maximum likelihood estimate of the shape parameter is inadequate due to the associated large bias when the sample size is small or the proportion of censored data is large. This manuscript demonstrates how the Bayesian information-theoretic minimum message length principle…
Likelihood-Based Regression for Weibull Accelerated Life Testing Model Under Censored Data
Indian Statistical Institute（印度统计研究所）；Moody’s Ratings（穆迪评级）；Department of Mathematics, Indian Institute of Technology Delhi（德里印度理工学院数学系）；Department of Management Sciences, Indian Institute of Technology Kanpur（坎普尔印度理工学院管理科学系）
Some new survival distributions are introduced based on a generalised exponential function. This class of distributions includes heavy-tailed generalisations of exponential, Weibull and gamma distributions. Properties of the distributions are described, and R code is available for computation of pdf, quantiles, inverse quantiles, random numbers, etc. A use of these distributions for robust inference is suggested, and this is exemplified with a Monte-Carlo study.
This paper focuses on variational inference with intractable likelihood functions that can be unbiasedly estimated. A flexible variational approximation based on Gaussian mixtures is developed, by adopting the mixture population Monte Carlo (MPMC) algorithm in \cite{cappe2008adaptive}. MPMC updates iteratively the parameters of mixture distributions with importance sampling computations, instead of the complicated gradient estimation of the optimization objective in usual variational Bayes. Noticing that MPMC uses a fixed number of mixture components, which is difficult to predict for real app…
In this short note we provide an unbiased multilevel Monte Carlo estimator of the log marginal likelihood and discuss its application to variational Bayes.
The main focus of the analysts who deal with clustered data is usually not on the clustering variables, and hence the group-specific parameters are treated as nuisance. If a fixed effects formulation is preferred and the total number of clusters is large relative to the single-group sizes, classical frequentist techniques relying on the profile likelihood are often misleading. The use of alternative tools, such as modifications to the profile likelihood or integrated likelihoods, for making accurate inference on a parameter of interest can be complicated by the presence of nonstandard modellin…
We present a sequential Monte Carlo sampler variant of the partial rejection control algorithm, and show that this variant can be considered as a sequential Monte Carlo sampler with a modified mutation kernel. We prove that the new sampler can reduce the variance of the incremental importance weights when compared with standard sequential Monte Carlo samplers. We provide a study of theoretical properties of the new algorithm, and make connections with some existing algorithms. Finally, the sampler is adapted for application under the challenging "likelihood free," approximate Bayesian computat…
The likelihood-informed subspace (LIS) method offers a viable route to reducing the dimensionality of high-dimensional probability distributions arising in Bayesian inference. LIS identifies an intrinsic low-dimensional linear subspace where the target distribution differs the most from some tractable reference distribution. Such a subspace can be identified using the leading eigenvectors of a Gram matrix of the gradient of the log-likelihood function. Then, the original high-dimensional target distribution is approximated through various forms of marginalization of the likelihood function, in…
Modeling is a challenging topic and using parametric models is an important stage to reach flexible function for modeling. Weibull distribution has two parameters which are shape $α$ and scale $β$. In this study, bimodality parameter is added and so bimodal Weibull distribution is proposed by using a quadratic transformation technique used to generate bimodal functions produced due to using the quadratic expression. The analytical simplicity of Weibull and quadratic form give an advantage to derive a bimodal Weibull via constructing normalizing constant. The characteristics and properties of t…
In this article, we describe a {\tt R} package for sampling from an empirical likelihood-based posterior using a Hamiltonian Monte Carlo method. Empirical likelihood-based methodologies have been used in Bayesian modeling of many problems of interest in recent times. This semiparametric procedure can easily combine the flexibility of a non-parametric distribution estimator together with the interpretability of a parametric model. The model is specified by estimating equations-based constraints. Drawing an inference from a Bayesian empirical likelihood (BayesEL) posterior is challenging. The li…
Stochastic approximation methods play a central role in maximum likelihood estimation problems involving intractable likelihood functions, such as marginal likelihoods arising in problems with missing or incomplete data, and in parametric empirical Bayesian estimation. Combined with Markov chain Monte Carlo algorithms, these stochastic optimisation methods have been successfully applied to a wide range of problems in science and industry. However, this strategy scales poorly to large problems because of methodological and theoretical difficulties related to using high-dimensional Markov chain …
A group of approaches for calculating forensic likelihood ratios first calculates scores which quantify the degree of difference or the degree of similarity between pairs of samples, then converts those scores to likelihood ratios. In order for a score-based approach to produce a forensically interpretable likelihood ratio, however, in addition to accounting for the similarity of the questioned sample with respect to the known sample, it must also account for the typicality of the questioned sample with respect to the relevant population. The present paper explores a number of score-based appr…
A six parameter distribution so-called the McDonald modified Weibull distribution is defined and studied. The new distribution contains, as special submodels, several important distributions discussed in the literature, such as the beta modified Weibull, Kumaraswamy modified Weibull, McDonald Weibull and modified Weibull distribution,among others. The new distribution can be used effectively in the analysis of survival data since it accommodates monotone, unimodal and bathtub-shaped hazard functions. We derive the moments.We propose the method of maximum likelihood for estimating the model par…
In engineering systems, it is usually assumed that lifetimes of components are independent and identically distributed (iid). But, the failure of a component results in a higher load on the remaining components and hence causes the distribution of the surviving components change. For modeling this kind of systems, the theory of sequential order statistics (SOS) can be used. Assuming Weibull distribution for lifetimes of components and conditionally proportional hazard rates model as a special case of the SOS theory, the maximum likelihood estimates of the unknown parameters are obtained in dif…
Synthetic likelihood is an attractive approach to likelihood-free inference when an approximately Gaussian summary statistic for the data, informative for inference about the parameters, is available. The synthetic likelihood method derives an approximate likelihood function from a plug-in normal density estimate for the summary statistic, with plug-in mean and covariance matrix obtained by Monte Carlo simulation from the model. In this article, we develop alternatives to Markov chain Monte Carlo implementations of Bayesian synthetic likelihoods with reduced computational overheads. Our approa…
Markov Chain Monte Carlo (MCMC) requires to evaluate the full data likelihood at different parameter values iteratively and is often computationally infeasible for large data sets. In this paper, we propose to approximate the log-likelihood with subsamples taken according to nonuniform subsampling probabilities, and derive the most likely optimal (MLO) subsampling probabilities for better approximation. Compared with existing subsampled MCMC algorithm with equal subsampling probabilities, our MLO subsampled MCMC has a higher estimation efficiency with the same subsampling ratio. We also derive…
University of Toronto（多伦多大学）；University of Cambridge（剑桥大学）；Twitter（推特）；Harvard University（哈佛大学）
Computing the marginal likelihood (ML) of a model requires marginalizing out all of the parameters and latent variables, a difficult high-dimensional summation or integration problem. To make matters worse, it is often hard to measure the accuracy of one's ML estimates. We present bidirectional Monte Carlo, a technique for obtaining accurate log-ML estimates on data simulated from a model. This method obtains stochastic lower bounds on the log-ML using annealed importance sampling or sequential Monte Carlo, and obtains stochastic upper bounds by running these same algorithms in reverse startin…
We present an original simulation-based method to estimate likelihood ratios efficiently for general state-space models. Our method relies on a novel use of the conditional Sequential Monte Carlo (cSMC) algorithm introduced in \citet{Andrieu_et_al_2010} and presents several practical advantages over standard approaches. The ratio is estimated using a unique source of randomness instead of estimating separately the two likelihood terms involved. Beyond the benefits in terms of variance reduction one may expect in general from this type of approach, an important point here is that the variance o…
Article describes the results of the development and using of Rare-Event Monte-Carlo Simulation Algorithms for Dynamic Fault Trees Estimation. For Fault Trees estimation usually analytical methods are used (Minimal Cut sets, Markov Chains, etc.), but for complex models with Dynamic Gates it is necessary to use Monte-Carlo simulation with combination of Importance Sampling method. Proposed article describes approach for this problem solution according for specific features of Dynamic Fault Trees. There are assumed, that failures are non-repairable with general distribution functions of times to…
Expectation maximization (EM) is a technique for estimating maximum-likelihood parameters of a latent variable model given observed data by alternating between taking expectations of sufficient statistics, and maximizing the expected log likelihood. For situations where sufficient statistics are intractable, stochastic approximation EM (SAEM) is often used, which uses Monte Carlo techniques to approximate the expected log likelihood. Two common implementations of SAEM, Batch EM (BEM) and online EM (OEM), are parameterized by a "learning rate", and their efficiency depend strongly on this param…
Hamiltonian Monte Carlo has emerged as a standard tool for posterior computation. In this article, we present an extension that can efficiently explore target distributions with discontinuous densities. Our extension in particular enables efficient sampling from ordinal parameters though embedding of probability mass functions into continuous spaces. We motivate our approach through a theory of discontinuous Hamiltonian dynamics and develop a corresponding numerical solver. The proposed solver is the first of its kind, with a remarkable ability to exactly preserve the Hamiltonian. We apply our…
In this article we develop a new sequential Monte Carlo (SMC) method for multilevel (ML) Monte Carlo estimation. In particular, the method can be used to estimate expectations with respect to a target probability distribution over an infinite-dimensional and non-compact space as given, for example, by a Bayesian inverse problem with Gaussian random field prior. Under suitable assumptions the MLSMC method has the optimal $O(ε^{-2})$ bound on the cost to obtain a mean-square error of $O(ε^2)$. The algorithm is accelerated by dimension-independent likelihood-informed (DILI) proposals designed for…
Informatics Institute（信息学研究所）；University of Amsterdam（阿姆斯特丹大学）；Donald Bren School of Information and Computer Sciences（唐纳德·布伦信息与计算机科学学院）；University of California, Irvine（加州大学欧文分校）；Canadian Institute for Advanced Research（加拿大高等研究院）
We describe an embarrassingly parallel, anytime Monte Carlo method for likelihood-free models. The algorithm starts with the view that the stochasticity of the pseudo-samples generated by the simulator can be controlled externally by a vector of random numbers u, in such a way that the outcome, knowing u, is deterministic. For each instantiation of u we run an optimization procedure to minimize the distance between summary statistics of the simulator and the data. After reweighing these samples using the prior and the Jacobian (accounting for the change of volume in transforming from the space…
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Mount Holyoke College（曼荷莲学院）
Recent work in variational inference (VI) uses ideas from Monte Carlo estimation to tighten the lower bounds on the log-likelihood that are used as objectives. However, there is no systematic understanding of how optimizing different objectives relates to approximating the posterior distribution. Developing such a connection is important if the ideas are to be applied to inference-i.e., applications that require an approximate posterior and not just an approximation of the log-likelihood. Given a VI objective defined by a Monte Carlo estimator of the likelihood, we use a "divide and couple" pr…
In this paper, we present a method for computing the marginal likelihood, also known as the model likelihood or Bayesian evidence, from Markov Chain Monte Carlo (MCMC), or other sampled posterior distributions. In order to do this, one needs to be able to estimate the density of points in parameter space, and this can be challenging in high numbers of dimensions. Here we present a Bayesian analysis, where we obtain the posterior for the marginal likelihood, using $k$th nearest-neighbour distances in parameter space, using the Mahalanobis distance metric, under the assumption that the points in…
Counting experiments often rely on Monte Carlo simulations for predictions of Poisson expectations. The accompanying uncertainty from the finite Monte Carlo sample size can be incorporated into parameter estimation by modifying the Poisson likelihood. We first review previous Frequentist methods of this type by Barlow et al, Bohm et al, and Chirkin, as well as recently proposed probabilistic methods by the author and Argüelles et al. We show that all these approaches can be understood in a unified way: they all approximate the underlying probability distribution of the sum of weights in a give…
A generalization of the generalized inverse Weibull distribution so-called transmuted generalized inverse Weibull dis- tribution is proposed and studied. We will use the quadratic rank transmutation map (QRTM) in order to generate a flexible family of probability distributions taking generalized inverse Weibull distribution as the base value distribution by introducing a new parameter that would offer more distributional flexibility. Various structural properties including explicit expressions for the mo- ments, quantiles, and moment generating function of the new dis- tribution are derived.We…
We obtain new mathematical properties of the exponentiated odd log-logistic family of distributions, and of its special case named the exponentiated odd log-logistic Weibull, and its log transformed. A new location and scale regression model is constructed, and some simulations are carried out to verify the behavior of the maximum likelihood estimators, and of the modified deviance-based residuals. The methodology is applied to the Japanese-Brazilian emigration data.
ABC (approximate Bayesian computation) is a general approach for dealing with models with an intractable likelihood. In this work, we derive ABC algorithms based on QMC (quasi- Monte Carlo) sequences. We show that the resulting ABC estimates have a lower variance than their Monte Carlo counter-parts. We also develop QMC variants of sequential ABC algorithms, which progressively adapt the proposal distribution and the acceptance threshold. We illustrate our QMC approach through several examples taken from the ABC literature. Keywords: Approximate Bayesian computation, Likelihood-free inference,…
