PierGianLuca Porta Mana
It is shown that the log-likelihood of a hypothesis or model given some data is equivalent to an average of all leave-one-out cross-validation log-scores that can be calculated from all subsets of the data. This relation can be generalized to any $k$-fold cross-validation log-scores.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
In Bayesian statistics, the marginal likelihood, also known as the evidence, is used to evaluate model fit as it quantifies the joint probability of the data under the prior. In contrast, non-Bayesian models are typically compared using cross-validation on held-out data, either through $k$-fold partitioning or leave-$p$-out subsampling. We show that the marginal likelihood is formally equivalent to exhaustive leave-$p$-out cross-validation averaged over all values of $p$ and all held-out test sets when using the log posterior predictive probability as the scoring rule. Moreover, the log poster…
Complex phenomena in engineering and the sciences are often modeled with computationally intensive feed-forward simulations for which a tractable analytic likelihood does not exist. In these cases, it is sometimes necessary to estimate an approximate likelihood or fit a fast emulator model for efficient statistical inference; such surrogate models include Gaussian synthetic likelihoods and more recently neural density estimators such as autoregressive models and normalizing flows. To date, however, there is no consistent way of quantifying the quality of such a fit. Here we propose a statistic…
The Influence of Validation Data on Logical and Scientific Interpretations of Forensic Expert Opinions
本文研究司法鉴定专家意见接收者如何运用贝叶斯推理评估自身似然比，并通过多类示例说明验证数据对其逻辑与科学解释的关键影响。
Generalized linear models play an essential role in a wide variety of statistical applications. This paper discusses an approximation of the likelihood in these models that can greatly facilitate computation. The basic idea is to replace a sum that appears in the exact log-likelihood by an expectation over the model covariates; the resulting "expected log-likelihood" can in many cases be computed significantly faster than the exact log-likelihood. In many neuroscience experiments the distribution over model covariates is controlled by the experimenter and the expected log-likelihood approximat…
Fast and robust cross-validation-based scoring rule inference for spatial statistics
提出基于留一交叉验证分数的LOOS参数估计方法，通过优化预测实现快速计算与鲁棒性定制，在空间数据实验中展现出远超最大似然估计的计算速度及更好的预测性能。
The goodness of the long-term prediction in the state-space model was evaluated using the squared long-term prediction error. In order to estimate the model parameters suitable for long-term prediction, we devised a modified log-likelihood corresponding to the long-term prediction error variance. Trend models and seasonally adjusted models with and without AR component are examined as examples.
The maximum likelihood estimation of the left-truncated log-logistic distribution with a given truncation point is analyzed in detail from both mathematical and numerical perspectives. These maximum likelihood equations often do not possess a solution, even for small truncations. A simple criterion is provided for the existence of a regular maximum likelihood solution. In this case a profile likelihood function can be constructed and the optimisation problem is reduced to one dimension. When the maximum likelihood equations do not admit a solution for certain data samples, it is shown that the…
Bayesian Optimization of Noisy Log-Likelihoods Evaluated by Particle Filters -- One Parameter Case --
本文提出利用贝叶斯优化方法在粒子滤波评估的噪声对数似然函数中寻找最大值，通过高斯过程代理和平衡探索与利用的获取函数，实现稳健的估计。
A logconcave likelihood is as important to proper statistical inference as a convex cost function is important to variational optimization. Quantization is often disregarded when writing likelihood models, ignoring the limitations of the physical detectors used to collect the data. These two facts call for the question: would including quantization in likelihood models preclude logconcavity? are the true data likelihoods logconcave? We provide a general proof that the same simple assumption that leads to logconcave continuous-data likelihoods also leads to logconcave quantized-data likelihoods…
Although the log-likelihood is widely used in model selection, the log-likelihood ratio has had few applications in this area. We develop a log-likelihood ratio based method for selecting regression models by focusing on the set of models deemed plausible by the likelihood ratio test. We show that when the sample size is large and the significance level of the test is small, there is a high probability that the smallest model in the set is the true model; thus, we select this smallest model. The significance level of the test serves as a parameter for this method. We consider three levels of t…
In this paper, we use the method of modified signed log-likelihood ratio test for the problem of testing the equality of correlation coefficients in two independent bivariate normal distributions. We compare this method with two other %competing approaches, Fisher's Z-transform and generalized test variable, using a Monte Carlo simulation. It indicates that the proposed method is better than the other approaches, in terms of the actual sizes and powers especially when the sample sizes are unequal. We illustrate performance of the proposed approach, using a real data set.
We study semiparametric time series models with innovations following a log-concave distribution. We propose a general maximum likelihood framework which allows us to estimate simultaneously the parameters of the model and the density of the innovations. This framework can be easily adapted to many well-known models, including ARMA, GARCH and ARMA-GARCH. Furthermore, we show that the estimator under our new framework is consistent in both ARMA and ARMA-GARCH settings. We demonstrate its finite sample performance via a thorough simulation study and apply it to model the daily log-return of FTSE…
Asymptotics of the discrete log-concave maximum likelihood estimator and related applications
本文研究离散对数凹概率质量函数的最大似然估计，证明其强相合性并建立正确与错误设定下的逐点渐近理论，进而构造置信区间，并通过R包logcondiscr在H1N1疫情数据上验证。
We consider the use of randomised forward models and log-likelihoods within the Bayesian approach to inverse problems. Such random approximations to the exact forward model or log-likelihood arise naturally when a computationally expensive model is approximated using a cheaper stochastic surrogate, as in Gaussian process emulation (kriging), or in the field of probabilistic numerical methods. We show that the Hellinger distance between the exact and approximate Bayesian posteriors is bounded by moments of the difference between the true and approximate log-likelihoods. Example applications of …
General log-linear models specified by non-negative integer design matrices have a potentially wide range of applications, although using models without the genuine overall effect, that is, ones which cannot be reparameterized to include a normalizing constant, is still rare. The log-linear models without the overall effect arise naturally in practice, and can be handled in a similar manner to models with the overall effect. A novel iterative scaling procedure for the MLE computation under such models is proposed, and its convergence is proved. The results are illustrated using data from a rec…
When some treatments are ordered according to the categories of an ordinal categorical variable (e.g., extent of side effects) in a monotone order, one might be interested in knowing wether the treatments are equally effective or not. One way to do that is to test if the likelihood ratio order is strictly verified. A method based on log-linear models is derived to make statistical inference and phi-divergence test-statistics are proposed for the test of interest. Focussed on loglinear modeling, the theory associated with the asymptotic distribution of the phi-divergence test-statistics is deve…
We consider nonparametric maximum-likelihood estimation of a log-concave density in case of interval-censored, right-censored and binned data. We allow for the possibility of a subprobability density with an additional mass at $+\infty$, which is estimated simultaneously. The existence of the estimator is proved under mild conditions and various theoretical aspects are given, such as certain shape and consistency properties. An EM algorithm is proposed for the approximate computation of the estimator and its performance is illustrated in two examples.
We study the smoothed log-concave maximum likelihood estimator of a probability distribution on $\mathbb{R}^d$. This is a fully automatic nonparametric density estimator, obtained as a canonical smoothing of the log-concave maximum likelihood estimator. We demonstrate its attractive features both through an analysis of its theoretical properties and a simulation study. Moreover, we use our methodology to develop a new test of log-concavity, and show how the estimator can be used as an intermediate stage of more involved procedures, such as constructing a classifier or estimating a functional o…
We propose and implement a nonlinear Verification and Validation (V&V) methodology to test two fitting procedures for the log-periodic power law model (LPPL), a model that has diverse applications across data analysis, but known estimation issues. Prior studies have focused on ex-post analyses of rare events: Earthquakes, glacial break-off events, and financial crashes. Or, on non-dynamical simulations such as additive noise or resampling. Our results reject an estimation scheme that pre-conditions observed data by fitting and removing an exponential trend. We validate a subordinated algorithm…
The fate of scientific hypotheses often relies on the ability of a computational model to explain the data, quantified in modern statistical approaches by the likelihood function. The log-likelihood is the key element for parameter estimation and model evaluation. However, the log-likelihood of complex models in fields such as computational biology and neuroscience is often intractable to compute analytically or numerically. In those cases, researchers can often only estimate the log-likelihood by comparing observed data with synthetic observations generated by model simulations. Standard tech…
Scalable simulation-based inference for implicitly defined models using a metamodel for Monte Carlo log-likelihood estimator
本文针对隐式定义模型提出一种基于蒙特卡洛对数似然估计器元模型的可扩展仿真推断方法，利用局部渐近正态性量化参数不确定性，并通过多个示例验证其准确性与可扩展性。
Consider a set of categorical variables $\mathcal{P}$ where at least one, denoted by $Y$, is binary. The log-linear model that describes the counts in the resulting contingency table implies a specific logistic regression model, with the binary variable as the outcome. Extending results in Christensen (1997), by also considering the case where factors present in the contingency table disappear from the logistic regression model, we prove that the Maximum Likelihood Estimate (MLE) for the parameters of the logistic regression equals the MLE for the corresponding parameters of the log-linear mod…
The state-space model and the Kalman filter provide us with unified and computationaly efficient procedure for computing the log-likelihood of the diverse type of time series models. This paper presents an algorithm for computing the gradient and the Hessian matrix of the log-likelihood by extending the Kalman filter without resorting to the numerical difference. Different from the previous paper(Kitagawa 2020), it is assumed that the observation noise variance R=1. It is known that for univariate time series, by maximizing the log-likelihood of this restricted model, we can obtain the same es…
Jackknife empirical likelihood ratio test for log symmetric distribution using probability weighted moments
针对对数对称分布的拟合检验需求，基于概率加权矩提出对数对称性拟合优度检验及更易实施的刀切经验似然比检验，推导了统计量渐近分布，经模拟和数据集验证了方法有效性。
Phylogenetic trees are key data objects in biology, and the method of phylogenetic reconstruction has been highly developed. The space of phylogenetic trees is a nonpositively curved metric space. Recently, statistical methods to analyze the set of trees on this space are being developed utilizing this property. Meanwhile, in Euclidean space, the log-concave maximum likelihood method has emerged as a new nonparametric method for probability density estimation. In this paper, we derive a sufficient condition for the existence and uniqueness of the log-concave maximum likelihood estimator on tre…
Let X_1, ..., X_n be independent and identically distributed random vectors with a log-concave (Lebesgue) density f. We first prove that, with probability one, there exists a unique maximum likelihood estimator of f. The use of this estimator is attractive because, unlike kernel density estimation, the method is fully automatic, with no smoothing parameters to choose. Although the existence proof is non-constructive, we are able to reformulate the issue of computation in terms of a non-differentiable convex optimisation problem, and thus combine techniques of computational geometry with Shor's…
Posterior distributions often feature intractable normalizing constants, called marginal likelihoods or evidence, that are useful for model comparison via Bayes factors. This has motivated a number of methods for estimating ratios of normalizing constants in statistics. In computational physics the logarithm of these ratios correspond to free energy differences. Combining unbiased Markov chain Monte Carlo estimators with path sampling, also called thermodynamic integration, we propose new unbiased estimators of the logarithm of ratios of normalizing constants. As a by-product, we propose unbia…
Log-linear models are typically fitted to contingency table data to describe and identify the relationship between different categorical variables. However, the data may include observed zero cell entries. The presence of zero cell entries can have an adverse effect on the estimability of parameters, due to parameter redundancy. We describe a general approach for determining whether a given log-linear model is parameter redundant for a pattern of observed zeros in the table, prior to fitting the model to the data. We derive the estimable parameters or functions of parameters and also explain h…
Fast calculation of Gaussian Process multiple-fold cross-validation residuals and their covariances
本文推广高斯过程留一法公式，提出多重交叉验证残差及其协方差的快速计算方法，验证其准确性与加速效果，并说明其在模型诊断、参数估计和折设计中的价值。
We study nonparametric maximum likelihood estimation of a log-concave probability density and its distribution and hazard function. Some general properties of these estimators are derived from two characterizations. It is shown that the rate of convergence with respect to supremum norm on a compact interval for the density and hazard rate estimator is at least $(\log(n)/n)^{1/3}$ and typically $(\log(n)/n)^{2/5}$, whereas the difference between the empirical and estimated distribution function vanishes with rate $o_{\mathrm{p}}(n^{-1/2})$ under certain regularity assumptions.
The maximum ${\log}_q$ likelihood estimation method is a generalization of the known maximum $\log$ likelihood method to overcome the problem for modeling non-identical observations (inliers and outliers). The parameter $q$ is a tuning constant to manage the modeling capability. Weibull is a flexible and popular distribution for problems in engineering. In this study, this method is used to estimate the parameters of Weibull distribution when non-identical observations exist. Since the main idea is based on modeling capability of objective function $ρ(x;\boldsymbolθ)=\log_q\big[f(x;\boldsymbol…
Motivated by studies in biological sciences to detect differentially expressed genes, a semiparametric two-component mixture model with one known component is being studied in this paper. Assuming the density of the unknown component to be log-concave, which contains a very broad family of densities, we develop a semiparametric maximum likelihood estimator and propose an EM algorithm to compute it. Our new estimation method finds the mixing proportions and the distribution of the unknown component simultaneously. We establish the identifiability of the proposed semiparametric mixture model and…
Optimal and computationally tractable lower bounds for logistic log-likelihoods
针对logit变换相关目标函数优化缺乏解析解的问题，本文设计了一种新颖的分段二次下界，统一改进切线二次最小化子，可提升MM方案收敛速度及VB近似准确性。
Cross validation approaches for penalized Cox regression
本文提出两种新的Cox回归交叉验证方法，通过模拟数据和肺癌生存研究数据验证其在性能和数值稳定性上的优势。
Bits per Spike as a Betting Game: An Interpretable Unit for Held-Out Log-Likelihood in Neural Data Analysis
该研究借鉴博弈论统计学，将神经尖峰序列保留对数似然转化为以记录秒为单位的可解释单元，通过显著性时间τΔ实现模型排序，在小鼠头方向细胞数据上验证了其有效性。
Log-F-penalized Conditional Logistic Regression for Sparse Data
针对稀疏匹配病例对照研究中条件逻辑回归估计量偏差与收缩控制不足问题，提出log-F惩罚条件逻辑回归方法，其在模拟中表现出与Firth方法相当的置信区间覆盖率，且均方误差更低。
We consider the problem of detecting a `bump' in the intensity of a Poisson process or in a density. We analyze two types of likelihood ratio based statistics which allow for exact finite sample inference and asymptotically optimal detection: The maximum of the penalized square root of log likelihood ratios (`penalized scan') evaluated over a certain sparse set of intervals, and a certain average of log likelihood ratios (`condensed average likelihood ratio'). We show that penalizing the {\sl square root} of the log likelihood ratio - rather than the log likelihood ratio itself - leads to a si…
We consider Bayesian inference when only a limited number of noisy log-likelihood evaluations can be obtained. This occurs for example when complex simulator-based statistical models are fitted to data, and synthetic likelihood (SL) method is used to form the noisy log-likelihood estimates using computationally costly forward simulations. We frame the inference task as a sequential Bayesian experimental design problem, where the log-likelihood function is modelled with a hierarchical Gaussian process (GP) surrogate model, which is used to efficiently select additional log-likelihood evaluation…
Logistic regression models for binomial responses are routinely used in statistical practice. However, the maximum likelihood estimate may not exist due to data separability. We address this issue by considering a conjugate prior penalty which always produces finite estimates. Such a specification has a clear Bayesian interpretation and enjoys several invariance properties, making it an appealing prior choice. We show that the proposed method leads to an accurate approximation of the reduced-bias approach of Firth (1993), resulting in estimators with smaller asymptotic bias than the maximum-li…
An Accurate Standard Error Estimation for Quadratic Exponential Logistic Regressions by Applying Generalized Estimating Equations to Pseudo-Likelihoods
针对二次指数二元分布最大似然估计计算难、伪似然低估标准误的问题，本研究提出将广义估计方程应用于伪似然时采用独立工作相关结构可获一致估计，推导了理论性质并通过模拟与实例验证了方法有效性。
Composite likelihood inference for the Poisson log-normal model
本文提出了一种结合EM框架与复合似然和重要性采样估计的新型推断方法，用于泊松对数正态模型参数估计，解决了高维积分瓶颈问题，实现了计算可行性，同时保持了最大似然估计的渐近性质。
We consider the problem of estimating a parameter associated to a Bayesian inverse problem. Treating the unknown initial condition as a nuisance parameter, typically one must resort to a numerical approximation of gradient of the log-likelihood and also adopt a discretization of the problem in space and/or time. We develop a new methodology to unbiasedly estimate the gradient of the log-likelihood with respect to the unknown parameter, i.e. the expectation of the estimate has no discretization bias. Such a property is not only useful for estimation in terms of the original stochastic model of …
Classical penalized likelihood regression problems deal with the case that the independent variables data are known exactly. In practice, however, it is common to observe data with incomplete covariate information. We are concerned with a fundamentally important case where some of the observations do not represent the exact covariate information, but only a probability distribution. In this case, the maximum penalized likelihood method can be still applied to estimating the regression function. We first show that the maximum penalized likelihood estimate exists under a mild condition. In the c…
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。