Information criteria are an appropriate and widely used tool for solving model selection problems. However, different ways to use them exist, each leading to a more or less precise approximation of the sought model. In this paper, we mainly present two methods of utilisation of information criteria : the classical one which is generally used and an alternative one, more precise but requiring a little more calculations. Those methods are compared on 1-D and 2-D autoregressive models ; we use a synthetized process for the 1-D case and texture images for the 2-D case. We also work with the original phi_beta criterion which includes all others usual criteria such as AIC, BIC, and phi.
We test three common information criteria (IC) for selecting the order of a Hawkes process with an intensity kernel that can be expressed as a mixture of exponential terms. These processes find application in high-frequency financial data modelling. The information criteria are Akaike's information criterion (AIC), the Bayesian information criterion (BIC) and the Hannan-Quinn criterion (HQ). Since we work with simulated data, we are able to measure the performance of model selection by the success rate of the IC in selecting the model that was used to generate the data. In particular, we are i…
The theoretical foundation for a number of model selection criteria is established in the context of inhomogeneous point processes and under various asymptotic settings: infill, increasing domain, and combinations of these. For inhomogeneous Poisson processes we consider Akaike information criterion and the Bayesian information criterion, and in particular we identify the point process analogue of sample size needed for the Bayesian information criterion. Considering general inhomogeneous point processes we derive new composite likelihood and composite Bayesian information criteria for selecti…
This paper derives a Minimum Message Length (MML) criterion for the model selection of the Autoregressive Moving Average (ARMA) time series model. The MML87 performances on the ARMA model compared with other well known model selection criteria, Akaike Information Criterion (AIC), Corrected AIC (AICc), Bayesian Information Criterion (BIC), and Hannan Quinn (HQ). The experimental results show that the MML87 is outperformed the other model selection criteria as it select most of the models with lower prediction errors and the models selected by MML87 to have a lower mean squared error in differen…
Selecting important spatial-dependent variables under the nonhomogeneous spatial Poisson process model is an important topic of great current interest. In this paper, we use the Deviance Information Criterion (DIC) and Logarithm of the Pseudo Marginal Likelihood (LPML) for Bayesian variable selection under the nonhomogeneous spatial Poisson process model. We further derive the new Monte Carlo estimation formula for LPML in the spatial Poisson process setting. Extensive simulation studies are carried out to evaluate the empirical performance of the proposed criteria. The proposed methodology is…
Interpolation and prediction have been useful approaches in modeling data in many areas of applications. The aim of this paper is the prediction of the next value of a time series (time series forecasting) using the techniques in interpolation of the spatial data, for the tow approaches kernel interpolation and kriging. We are interested in finding some sufficient conditions for the kernels and provide a detailed analyse of the prediction using kernel interpolation. Finally, we provide a natural idea to select a good kernel among a given family of kernels using only the data. We illustrate our…
We consider the use of Bayesian information criteria for selection of the graph underlying an Ising model. In an Ising model, the full conditional distributions of each variable form logistic regression models, and variable selection techniques for regression allow one to identify the neighborhood of each node and, thus, the entire graph. We prove high-dimensional consistency results for this pseudo-likelihood approach to graph selection when using Bayesian information criteria for the variable selection problems in the logistic regressions. The results pertain to scenarios of sparsity and fol…
In objective Bayesian model selection, no single criterion has emerged as dominant in defining objective prior distributions. Indeed, many criteria have been separately proposed and utilized to propose differing prior choices. We first formalize the most general and compelling of the various criteria that have been suggested, together with a new criterion. We then illustrate the potential of these criteria in determining objective model selection priors by considering their application to the problem of variable selection in normal linear models. This results in a new model selection objective…
The paper obtains analytical results for the asymptotic properties of Model Selection Criteria -- widely used in practice -- for a general family of hidden Markov models (HMMs), thereby substantially extending the related theory beyond typical i.i.d.-like model structures and filling in an important gap in the relevant literature. In particular, we look at the Bayesian and Akaike Information Criteria (BIC and AIC) and the model evidence. In the setting of nested classes of models, we prove that BIC and the evidence are strongly consistent for HMMs (under regularity conditions), whereas AIC is …
This paper studies, under the setting of spline regression, the connection between finite-sample properties of selection criteria and their asymptotic counterparts, focusing on bridging the gap between the two. We introduce a bias-variance decomposition of the prediction error, using which it is shown that in the asymptotics the bias term dominates the variability term, providing an explanation of the gap. A geometric exposition is provided for intuitive understanding. The theoretical and geometric results are illustrated through a numerical example.
Multi-model mimicry (MMM) is a flexible model selection technique for comparison of multiple, non-nested models on any desired goodness-of-fit criteria. Applicable to any set of candidate models that are 1) able to be fit to observed data, 2) can simulate new sets of data under the models, and 3) have a metric by which a dataset's goodness-of-fit to the model can be calculated, MMM has a much broader range of applicability than many standard model selection techniques. This manuscript highlights the previous literature whilst presenting the theoretical framework underpinning MMM. The scope of …
We propose new model selection criteria based on generalized ridge estimators dominating the maximum likelihood estimator under the squared risk and the Kullback-Leibler risk in multivariate linear regression. Our model selection criteria have the following favorite properties: consistency, unbiasedness, uniformly minimum variance. Consistency is proven under an asymptotic structure $\frac{p}{n}\to c$ where $n$ is the sample size and $p$ is the parameter dimension of the response variables. In particular, our proposed class of estimators dominates the maximum likelihood estimator under the squ…
This paper is devoted to model selection in logistic regression. We extend the model selection principle introduced by Birgé and Massart (2001) to logistic regression model. This selection is done by using penalized maximum likelihood criteria. We propose in this context a completely data-driven criteria based on the slope heuristics. We prove non asymptotic oracle inequalities for selected estimators. Theoretical results are illustrated through simulation studies.
We consider the robust adaptive nonparametric estimation problem for a periodic function observed in the framework of a continuous time regression model with semimartingale noises.
This paper studies the model selection problem in a large class of causal time series models, which includes both the ARMA or AR($\infty$) processes, as well as the GARCH or ARCH($\infty$), APARCH, ARMA-GARCH and many others processes. To tackle this issue, we consider a penalized contrast based on the quasi-likelihood of the model. We provide sufficient conditions for the penalty term to ensure the consistency of the proposed procedure as well as the consistency and the asymptotic normality of the quasi-maximum likelihood estimator of the chosen model. It appears from these conditions that th…
Recent results in homotopy and solution paths demonstrate that certain well-designed greedy algorithms, with a range of values of the algorithmic parameter, can provide solution paths to a sequence of convex optimization problems. On the other hand, in regression many existing criteria in subset selection (including $C_p$, AIC, BIC, MDL, RIC, etc.) involve optimizing an objective function that contains a counting measure. The two optimization problems are formulated as (P1) and (P0) in the present paper. The latter is generally combinatoric and has been proven to be NP-hard. We study the condi…
We consider a new criterion-based approach to model selection in linear regression. Properties of selection criteria based on p-values of a likelihood ratio statistic are studied for families of linear regression models. We prove that such procedures are consistent i.e. the minimal true model is chosen with probability tending to 1 even when the number of models under consideration slowly increases with a sample size. The simulation study indicates that introduced methods perform promisingly when compared with Akaike and Bayesian Information Criteria.
The Integrated Completed Likelihood (ICL) criterion has been proposed by Biernacki et al. (2000) in the model-based clustering framework to select a relevant number of classes and has been used by statisticians in various application areas. A theoretical study of this criterion is proposed. A contrast related to the clustering objective is introduced: the conditional classification likelihood. This yields an estimator and a model selection criteria class. The properties of these new procedures are studied and ICL is proved to be an approximation of one of these criteria. We oppose these result…
In this paper we develop the James - Stein improved estimation method for a nonparametric periodic function observed with the Levy noises in continuous time. An adaptive model selection procedure based on the improved weighted least square estimates is constructed. The improvement effect for the nonparametric models is obtained. It turns out that in the nonasymptotic studies the accuracy improvement for nonparametric problems is more significantly than for the parametric one. Moreover, sharp oracle inequalities for the robust risks have been shown and the efficiency property for the improved m…
RIKEN Center for Brain Science（理化学研究所脑科学中心）；Cornell University（康奈尔大学）；University of Helsinki（赫尔辛基大学）
Many statistical models are given in the form of non-normalized densities with an intractable normalization constant. Since maximum likelihood estimation is computationally intensive for these models, several estimation methods have been developed which do not require explicit computation of the normalization constant, such as noise contrastive estimation (NCE) and score matching. However, model selection methods for general non-normalized models have not been proposed so far. In this study, we develop information criteria for non-normalized models estimated by NCE or score matching. They are …
This paper examines the limit properties of information criteria (such as AIC, BIC, HQIC) for distinguishing between the unit root model and the various kinds of explosive models. The explosive models include the local-to-unit-root model, the mildly explosive model and the regular explosive model. Initial conditions with different order of magnitude are considered. Both the OLS estimator and the indirect inference estimator are studied. It is found that BIC and HQIC, but not AIC, consistently select the unit root model when data come from the unit root model. When data come from the local-to-u…
