The extension of the classical Bayesian penalized spline method to inference on vector-valued functions is considered, with an emphasis on characterizing the suitability of the method for general application.We show that the standard quadratic penalty is exactly analogous to the energy of a stretched string, with the penalty parameter corresponding to its tension. This physical analogy motivates a discussion of resolution independence, which we define as the convergence of a computational function estimate to arbitrary accuracy with increasing resolution.The multidimensional context makes direct application of standard procedures for choosing the penalty parameter difficult, and a new method is proposed and compared to the established generalized cross-validation (GCV) and Akaike information criterion (AIC) functions.Our Bayesian method for choosing this parameter is derived by introducing a scal e independence criterion to ensure that simultaneously scaling the function samples and their variances does not significantly change the posterior parameter distribution. Due to the possibility of an exact polynomial fit, numerical issues prevent the use of this prior, and a solution is presented based on adding a st ring zero-point energy. This makes more complicated approaches recently propose d in the literature unnecessary, and eliminates the requirement for sensitivity analysis when the function deviates from the above mentioned polynomial. An important class of problems which can be analyzed by this method are stochastic numerical integrators, which are considered as an example problem. This work represents the first extension of penalized spline methods to inference on multidimensional numerical integrators reported in the literature. Several numerical calculations illustrate the above points and address practical application issues.
Bayesian P-splines assume an intrinsic Gaussian Markov random field prior on the spline coefficients, conditional on a precision hyper-parameter $τ$. Prior elicitation of $τ$ is difficult. To overcome this issue we aim to building priors on an interpretable property of the model, indicating the complexity of the smooth function to be estimated. Following this idea, we propose Penalized Complexity (PC) priors for the number of effective degrees of freedom. We present the general ideas behind the construction of these new PC priors, describe their properties and show how to implement them in P-s…
Despite their widespread use in practice, the asymptotic properties of Bayesian penalized splines have not been investigated so far. We close this gap and study posterior concentration rates for Bayesian penalized splines in a Gaussian nonparametric regression model. A key feature of the approach is the hyperprior on the smoothing variance, which allows for adaptive smoothing in practice but complicates the theoretical analysis considerably as it destroys conjugacy and precludes analytic expressions for the posterior moments. To derive our theoretical results, we rely on several new concepts i…
The paper motivates high dimensional smoothing with penalized splines and its numerical calculation in an efficient way. If smoothing is carried out over three or more covariates the classical tensor product spline bases explode in their dimension bringing the estimation to its numerical limits. A recent approach by Siebenborn and Wagner(2019) circumvents storage expensive implementations by proposing matrix-free calculations which allows to smooth over several covariates. We extend their approach here by linking penalized smoothing and its Bayesian formulation as mixed model which provides a …
Reducing the global burden of stillbirths is important to improving child and maternal health. Of interest is understanding patterns in the timing of stillbirths -- that is, whether they occur in the intra- or antepartum period -- because stillbirths that occur intrapartum are largely preventable. However, data availability on the timing of stillbirths is highly variable across the world, with low- and middle-income countries generally having few reliable observations. In this paper we develop a Bayesian penalized splines regression framework to estimate the proportion of stillbirths that are …
Generalized additive models (GAMs) are a well-established statistical tool for modeling complex nonlinear relationships between covariates and a response assumed to have a conditional distribution in the exponential family. In this article, P-splines and the Laplace approximation are coupled for flexible and fast approximate Bayesian inference in GAMs. The proposed Laplace-P-spline model contributes to the development of a new methodology to explore the posterior penalty space by considering a deterministic grid-based strategy or a Markov chain sampler, depending on the number of smooth additi…
Dynamic Contrast-enhanced Magnetic Resonance Imaging (DCE-MRI) is an important tool for detecting subtle kinetic changes in cancerous tissue. Quantitative analysis of DCE-MRI typically involves the convolution of an arterial input function (AIF) with a nonlinear pharmacokinetic model of the contrast agent concentration. Parameters of the kinetic model are biologically meaningful, but the optimization of the non-linear model has significant computational issues. In practice, convergence of the optimization algorithm is not guaranteed and the accuracy of the model fitting may be compromised. To …
Estimates of the under-five mortality rate (U5MR) are used to track progress in reducing child mortality and to evaluate countries' performance related to Millennium Development Goal 4. However, for the great majority of developing countries without well-functioning vital registration systems, estimating the U5MR is challenging due to limited data availability and data quality issues. We describe a Bayesian penalized B-spline regression model for assessing levels and trends in the U5MR for all countries in the world, whereby biases in data series are estimated through the inclusion of a multil…
The mixture cure model for analyzing survival data is characterized by the assumption that the population under study is divided into a group of subjects who will experience the event of interest over some finite time horizon and another group of cured subjects who will never experience the event irrespective of the duration of follow-up. When using the Bayesian paradigm for inference in survival models with a cure fraction, it is common practice to rely on Markov chain Monte Carlo (MCMC) methods to sample from posterior distributions. Although computationally feasible, the iterative nature of…
Bayesian learning is often hampered by large computational expense. As a powerful generalization of popular belief propagation, expectation propagation (EP) efficiently approximates the exact Bayesian computation. Nevertheless, EP can be sensitive to outliers and suffer from divergence for difficult cases. To address this issue, we propose a new approximate inference approach, relaxed expectation propagation (REP). It relaxes the moment matching requirement of expectation propagation by adding a relaxation factor into the KL minimization. We penalize this relaxation with a $l_1$ penalty. As a …
Handling Overlapping Asymmetric Datasets -- A Twice Penalized P-Spline Approach
Newcastle University（纽卡斯尔大学）；School of Computing（计算机学院）；School of Mathematics, Statistics and Physics（数学、统计与物理学院）；Translational & Clinical Research Institute（转化与临床研究院）
Penalized B-splines are routinely used in additive models to describe smooth changes in a response with quantitative covariates. It is typically done through the conditional mean in the exponential family using generalized additive models with an indirect impact on other conditional moments. Another common strategy consists in focussing on several low-order conditional moments, leaving the complete conditional distribution unspecified. Alternatively, a multi-parameter distribution could be assumed for the response with several of its parameters jointly regressed on covariates using additive ex…
Statistical inference with nonresponse is quite challenging, especially when the response mechanism is nonignorable. In this case, the validity of statistical inference depends on untestable correct specification of the response model. To avoid the misspecification, we propose semiparametric Bayesian estimation in which an outcome model is parametric, but the response model is semiparametric in that we do not assume any parametric form for the nonresponse variable. We adopt penalized spline methods to estimate the unknown function. We also consider a fully nonparametric approach to modeling th…
We use a discrete-time proportional hazards model of time to involuntary employment termination. This model enables us to examine both the continuous effect of the age of an employee and whether that effect has varied over time, generalizing earlier work [Kadane and Woodworth J. Bus. Econom. Statist. 22 (2004) 182--193]. We model the log hazard surface (over age and time) as a thin-plate spline, a Bayesian smoothness-prior implementation of penalized likelihood methods of surface-fitting [Wahba (1990) Spline Models for Observational Data. SIAM]. The nonlinear component of the surface has only …
The popular cubic smoothing spline estimate of a regression function arises as the minimizer of the penalized sum of squares $\sum_j(Y_j - μ(t_j))^2 + λ\int_a^b [μ"(t)]^2 dt$, where the data are $t_j,Y_j$, $j=1,..., n$. The minimization is taken over an infinite-dimensional function space, the space of all functions with square integrable second derivatives. But the calculations can be carried out in a finite-dimensional space. The reduction from minimizing over an infinite dimensional space to minimizing over a finite dimensional space occurs for more general objective functions: the data may…
Modeling dynamical systems with ordinary differential equations implies a mechanistic view of the process underlying the dynamics. However in many cases, this knowledge is not available. To overcome this issue, we introduce a general framework for nonparametric ODE models using penalized regression in Reproducing Kernel Hilbert Spaces (RKHS) based on operator-valued kernels. Moreover, we extend the scope of gradient matching approaches to nonparametric ODE. A smooth estimate of the solution ODE is built to provide an approximation of the derivative of the ODE solution which is in turn used to …
We study trend filtering, a recently proposed tool of Kim et al. [SIAM Rev. 51 (2009) 339-360] for nonparametric regression. The trend filtering estimate is defined as the minimizer of a penalized least squares criterion, in which the penalty term sums the absolute $k$th order discrete derivatives over the input points. Perhaps not surprisingly, trend filtering estimates appear to have the structure of $k$th degree spline functions, with adaptively chosen knot points (we say ``appear'' here as trend filtering estimates are not really functions over continuous domains, and are only defined over…
