arXiv 0803.27112008-03physics.data-anhep-ex

A pitfall in the use of extended likelihood for fitting fractions of pure samples in mixed samples

A. Nappi

备注：: 10 pages with 1 figure, to appear in Computer Physics Communications. New version including major changes and additions, resulting from discussion with journal reviewers
发表信息：: Comput.Phys.Commun.180:269-275,2009
DOI：: 10.1016/j.cpc.2008.10.008

英文摘要：

The paper elucidates, with an analytic example, a subtle mistake in the application of the extended likelihood method to the problem of determining the fractions of pure samples in a mixed sample from the shape of the distribution of a random variable. This mistake, which affects two widely used software packages, leads to a misestimate of the errors.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Pitfalls of Goodness-of-Fit from Likelihood

The value of the likelihood is occasionally used by high energy physicists as a statistic to measure goodness-of-fit in unbinned maximum likelihood fits. Simple examples are presented that illustrate why this (seemingly intuitive) method fails in practice to achieve the desired goal.

2003-10-31physics.data-an

Constraints on Yield Parameters in Extended Maximum Likelihood Fits

The method of extended maximum likelihood is a well known concept of parameter estimation. One can implement external knowledge on the unknown parameters by multiplying the likelihood by constraint terms. In this note, we emphasize that this is also true for yield parameters in an extended maximum likelihood fit, which is widely used in the particle physics community. We recommend a way to generate pseudo-experiments in presence of constraint terms on yield parameters, and point to pitfalls inside the RooFit framework.

2012-10-26hep-exphysics.data-anstat.AP

Chi-square Fitting When Overall Normalization is a Fit Parameter

The problem of fitting an event distribution when the total expected number of events is not fixed, keeps appearing in experimental studies. In a chi-square fit, if overall normalization is one of the parameters parameters to be fit, the fitted curve may be seriously low with respect to the data points, sometimes below all of them. This problem and the solution for it are well known within the statistics community, but, apparently, not well known among some of the physics community. The purpose of this note is didactic, to explain the cause of the problem and the easy and elegant solution. The…

2015-06-30hep-exphysics.data-an

Empirical Survival Jensen-Shannon Divergence as a Goodness-of-Fit Measure for Maximum Likelihood Estimation and Curve Fitting

The coefficient of determination, known as $R^2$, is commonly used as a goodness-of-fit criterion for fitting linear models. $R^2$ is somewhat controversial when fitting nonlinear models, although it may be generalised on a case-by-case basis to deal with specific models such as the logistic model. Assume we are fitting a parametric distribution to a data set using, say, the maximum likelihood estimation method. A general approach to measure the goodness-of-fit of the fitted parameters, which is advocated herein, is to use a nonparametric measure for comparison between the empirical distributi…

2019-06-04physics.data-anq-fin.STstat.ME

The distinct flavors of Zipf's law in the rank-size and in the size-distribution representations, and its maximum-likelihood fitting

In the last years, researchers have realized the difficulties of fitting power-law distributions properly. These difficulties are higher in Zipf's systems, due to the discreteness of the variables and to the existence of two representations for these systems, i.e., two versions about which is the random variable to fit. The discreteness implies that a power law in one of the representations is not a power law in the other, and vice versa. We generate synthetic power laws in both representations and apply a state-of-the-art fitting method (based on maximum-likelihood plus a goodness-of-fit test…

2019-08-04nlin.AOphysics.data-an

A Thermodynamical Approach for Probability Estimation

Sony Computer Science Laboratories, Inc.（索尼计算机科学实验室公司）

The issue of discrete probability estimation for samples of small size is addressed in this study. The maximum likelihood method often suffers over-fitting when insufficient data is available. Although the Bayesian approach can avoid over-fitting by using prior distributions, it still has problems with objective analysis. In response to these drawbacks, a new theoretical framework based on thermodynamics, where energy and temperature are introduced, was developed. Entropy and likelihood are placed at the center of this method. The key principle of inference for probability mass functions is th…

2012-12-12cs.LGphysics.data-anstat.ME

更多相关论文收起更多论文

Fitting Power-laws in empirical data with estimators that work for all exponents

It has been repeatedly stated that maximum likelihood (ML) estimates of exponents of power-law distributions can only be reliably obtained for exponents smaller than minus one. The main argument that power laws are otherwise not normalizable, depends on the underlying sample space the data is drawn from, and is true only for sample spaces that are unbounded from above. Here we show that power-laws obtained from bounded sample spaces (as is the case for practically all data related problems) are always free of such limitations and maximum likelihood estimates can be obtained for arbitrary power…

2016-09-17physics.data-anphysics.soc-ph