The Generalized Additive Model (GAM) is a powerful tool and has been well studied. This model class helps to identify additive regression structure. Via available test procedures one may identify the regression structure even sharper if some component functions have parametric form. The Generalized Additive Partially Linear Models (GAPLM) enjoy the simplicity of the GLM and the flexibility of the GAM because they combine both parametric and nonparametric components. We use the hybrid spline-backfitted kernel estimation method, which combines the best features of both spline and kernel methods for making fast, efficient and reliable estimation under alpha-mixing condition. In addition, simultaneous confidence corridors (SCCs) for testing overall trends and empirical likelihood confidence region for parameters are provided under independent condition. The asymptotic properties are obtained and simulation results support the theoretical properties. For the application, we use the GAPLM to improve the accuracy ratio of the default predictions for $19610$ German companies. The quantlets for this paper are available on https://github.com.
The additive partially linear model (APLM) combines the flexibility of nonparametric regression with the parsimony of regression models, and has been widely used as a popular tool in multivariate nonparametric regression to alleviate the "curse of dimensionality". A natural question raised in practice is the choice of structure in the nonparametric part, that is, whether the continuous covariates enter into the model in linear or nonparametric form. In this paper, we present a comprehensive framework for simultaneous sparse model identification and learning for ultra-high-dimensional APLMs whe…
We consider an additive partially linear framework for modelling massive heterogeneous data. The major goal is to extract multiple common features simultaneously across all sub-populations while exploring heterogeneity of each sub-population. We propose an aggregation type of estimators for the commonality parameters that possess the asymptotic optimal bounds and the asymptotic distributions as if there were no heterogeneity. This oracle result holds when the number of sub-populations does not grow too fast and the tuning parameters are selected carefully. A plug-in estimator for the heterogen…
Efficient Poisson Subsampling for the Partially Linear Additive Cox Model
Nankai University（南开大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Changchun University of Technology（长春工业大学）；Liaoning University（辽宁大学）；University of Connecticut（康涅狄格大学）；Tianjin University（天津大学）
In this paper, we derive minimax rates for estimating both parametric and nonparametric components in partially linear additive models with high dimensional sparse vectors and smooth functional components. The minimax lower bound for Euclidean components is the typical sparse estimation rate that is independent of nonparametric smoothness indices. However, the minimax lower bound for each component function exhibits an interplay between the dimensionality and sparsity of the parametric component and the smoothness of the relevant nonparametric component. Indeed, the minimax risk for smooth non…
We propose a two-step estimating procedure for generalized additive partially linear models with clustered data using estimating equations. Our proposed method applies to the case that the number of observations per cluster is allowed to increase with the number of independent subjects. We establish oracle properties for the two-step estimator of each function component such that it performs as well as the univariate function estimator by assuming that the parametric vector and all other function components are known. Asymptotic distributions and consistency properties of the estimators are ob…
Discrete but ordered covariates are quite common in applied statistics, and some regularized fitting procedures have been proposed for proper handling of ordinal predictors in statistical modeling. In this study, we show how quadratic penalties on adjacent dummy coefficients of ordinal predictors proposed in the literature can be incorporated in the framework of generalized additive models, making tools for statistical inference developed there available for ordinal predictors as well. Motivated by an application from neonatal medicine, we discuss whether results obtained when constructing con…
We consider efficient estimation of the Euclidean parameters in a generalized partially linear additive models for longitudinal/clustered data when multiple covariates need to be modeled nonparametrically, and propose an estimation procedure based on a spline approximation of the nonparametric part of the model and the generalized estimating equations (GEE). Although the model in consideration is natural and useful in many practical applications, the literature on this model is very limited because of challenges in dealing with dependent data for nonparametric additive models. We show that the…
Statistical inference in high dimensional settings has recently attracted enormous attention within the literature. However, most published work focuses on the parametric linear regression problem. This paper considers an important extension of this problem: statistical inference for high dimensional sparse nonparametric additive models. To be more precise, this paper develops a methodology for constructing a probability density function on the set of all candidate models. This methodology can also be applied to construct confidence intervals for various quantities of interest (such as noise v…
We consider a flexible semiparametric quantile regression model for analyzing high dimensional heterogeneous data. This model has several appealing features: (1) By considering different conditional quantiles, we may obtain a more complete picture of the conditional distribution of a response variable given high dimensional covariates. (2) The sparsity level is allowed to be different at different quantile levels. (3) The partially linear additive structure accommodates nonlinearity and circumvents the curse of dimensionality. (4) It is naturally robust to heavy-tailed distributions. In this p…
A new statistical procedure, based on a modified spline basis, is proposed to identify the linear components in the panel data model with fixed effects. Under some mild assumptions, the proposed procedure is shown to consistently estimate the underlying regression function, correctly select the linear components, and effectively conduct the statistical inference. When compared to existing methods for detection of linearity in the panel model, our approach is demonstrated to be theoretically justified as well as practically convenient. We provide a computational algorithm that implements the pr…
Single-cell lineage tracking strategies enabled by recent experimental technologies have produced significant insights into cell fate decisions, but lack the quantitative framework necessary for rigorous statistical analysis of mechanistic models describing cell division and differentiation. In this paper, we develop such a framework with corresponding moment-based parameter estimation techniques for continuous-time, multi-type branching processes. Such processes provide a probabilistic model of how cells divide and differentiate, and we apply our method to study hematopoiesis, the mechanism o…
Recently the shape-restricted inference has gained popularity in statistical and econometric literature in order to relax the linear or quadratic covariate effect in regression analyses. The typical shape-restricted covariate effect includes monotonic increasing, decreasing, convexity or concavity. In this paper, we introduce the shape-restricted inference to the celebrated Cox regression model (SR-Cox), in which the covariate response is modeled as shape-restricted additive functions. The SR-Cox regression approximates the shape-restricted functions using a spline basis expansion with data dr…
Statistics has made tremendous advances since the times of Fisher, Neyman, Jeffreys, and others, but the fundamental and practically relevant questions about probability and inference that puzzled our founding fathers remain unanswered. To bridge this gap, I propose to look beyond the two dominating schools of thought and ask the following three questions: what do scientists need out of statistics, do the existing frameworks meet these needs, and, if not, how to fill the void? To the first question, I contend that scientists seek to convert their data, posited statistical model, etc., into cal…
We consider partially observed multiscale diffusion models that are specified up to an unknown vector parameter. We establish for a very general class of test functions that the filter of the original model converges to a filter of reduced dimension. Then, this result is used to justify statistical estimation for the unknown parameters of interest based on the model of reduced dimension but using the original available data. This allows to learn the unknown parameters of interest while working in lower dimensions, as opposed to working with the original high dimensional system. Simulation stud…
Missing data is a common problem in clinical data collection, which causes difficulty in the statistical analysis of such data. In this article, we consider the problem under a framework of a semiparametric partially linear model when observations are subject to missingness with complex patterns. If the correct model structure of the additive partially linear model is available, we propose to use a new imputation method called Partial Replacement IMputation Estimation (PRIME), which can overcome problems caused by incomplete data in the partially linear model. Also, we use PRIME in conjunction…
We use local polynomial fitting to estimate the nonparametric M-regression function for strongly mixing stationary processes $\{(Y_{i},\underline{X}_{i})\}$. We establish a strong uniform consistency rate for the Bahadur representation of estimators of the regression function and its derivatives. These results are fundamental for statistical inference and for applications that involve plugging in such estimators into other functionals where some control over higher order terms are required. We apply our results to the estimation of an additive M-regression model.
In this paper we discuss the estimation of a nonparametric component $f_1$ of a nonparametric additive model $Y=f_1(X_1) + ...+ f_q(X_q) + ε$. We allow the number $q$ of additive components to grow to infinity and we make sparsity assumptions about the number of nonzero additive components. We compare this estimation problem with that of estimating $f_1$ in the oracle model $Z= f_1(X_1) + ε$, for which the additive components $f_2,\dots,f_q$ are known. We construct a two-step presmoothing-and-resmoothing estimator of $f_1$ and state finite-sample bounds for the difference between our estimator…
We propose a novel framework for fitting additive quantile regression models, which provides well calibrated inference about the conditional quantiles and fast automatic estimation of the smoothing parameters, for model structures as diverse as those usable with distributional GAMs, while maintaining equivalent numerical efficiency and stability. The proposed methods are at once statistically rigorous and computationally efficient, because they are based on the general belief updating framework of Bissiri et al. (2016) to loss based inference, but compute by adapting the stable fitting methods…
We study semiparametric varying-coefficient partially linear models when some linear covariates are not observed, but ancillary variables are available. Semiparametric profile least-square based estimation procedures are developed for parametric and nonparametric components after we calibrate the error-prone covariates. Asymptotic properties of the proposed estimators are established. We also propose the profile least-square based ratio test and Wald test to identify significant parametric and nonparametric components. To improve accuracy of the proposed tests for small or moderate sample size…
In this article, we develop a semiparametric Bayesian estimation and model selection approach for partially linear additive models in conditional quantile regression. The asymmetric Laplace distribution provides a mechanism for Bayesian inferences of quantile regression models based on the check loss. The advantage of this new method is that nonlinear, linear and zero function components can be separated automatically and simultaneously during model fitting without the need of pre-specification or parameter tuning. This is achieved by spike-and-slab priors using two sets of indicator variables…
It is widely admitted that structured nonparametric modeling that circumvents the curse of dimensionality is important in nonparametric estimation. In this paper we show that the same holds for semi-parametric estimation. We argue that estimation of the parametric component of a semi-parametric model can be improved essentially when more structure is put into the nonparametric part of the model. We illustrate this for the partially linear model, and investigate efficiency gains when the nonparametric part of the model has an additive structure. We present the semi-parametric Fisher information…
Regression modelling of spatiotemporal extreme U.S. wildfires via partially-interpretable neural networks
School of Mathematics, University of Edinburgh（爱丁堡大学数学学院）；Statistics Program, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学计算机、电气与数学科学与工程（CEMSE）分部统计项目）
Statistical techniques used in air pollution modelling usually lack the possibility to understand which predictors affect air pollution in which functional form; and are not able to regress on exceedances over certain thresholds imposed by authorities directly. The latter naturally induce conditional quantiles and reflect the seriousness of particular events. In the present paper we focus on this important aspect by developing quantile regression models further. We propose a general Bayesian effect selection approach for additive quantile regression within a highly interpretable framework. We …
Current status data are commonly encountered in medical and epidemiological studies in which the failure time for study units is the outcome variable of interest. Data of this form are characterized by the fact that the failure time is not directly observed but rather is known relative to an observation time; i.e., the failure times are either left- or right-censored. Due to its structure, the analysis of such data can be challenging. To circumvent these challenges and to provide for a flexible modeling construct which can be used to analyze current status data, herein, a partially linear addi…
The generalized partially linear additive model (GPLAM) is a flexible and interpretable approach to building predictive models. It combines features in an additive manner, allowing each to have either a linear or nonlinear effect on the response. However, the choice of which features to treat as linear or nonlinear is typically assumed known. Thus, to make a GPLAM a viable approach in situations in which little is known $a~priori$ about the features, one must overcome two primary model selection challenges: deciding which features to include in the model and determining which of these features…
We observe the actions of a $K$ sub-sample of $N$ individuals up to time $t$ for some large $K<N$. We model the relationships of individuals by i.i.d. Bernoulli($p$)-random variables, where $p\in (0,1]$ is an unknown parameter. The rate of action of each individual depends on some unknown parameter $μ> 0$ and on the sum of some function $ϕ$ of the ages of the actions of the individuals which influence him. The function $ϕ$ is unknown but we assume it rapidly decays. The aim of this paper is to estimate the parameter $p$ asymptotically as $N\to \infty$, $K\to \infty$, and $t\to \infty$. Let $m_…
Deep learning has enjoyed tremendous success in a variety of applications but its application to quantile regressions remains scarce. A major advantage of the deep learning approach is its flexibility to model complex data in a more parsimonious way than nonparametric smoothing methods. However, while deep learning brought breakthroughs in prediction, it often lacks interpretability due to the black-box nature of multilayer structure with millions of parameters, hence it is not well suited for statistical inference. In this paper, we leverage the advantages of deep learning to apply it to quan…
A novel generalized additive scalar-on-function regression model for partially observed multidimensional functional data: An application to air quality classification
Additive models are popular in high--dimensional regression problems because of flexibility in model building and optimality in additive function estimation. Moreover, they do not suffer from the so-called {\it curse of dimensionality} generally arising in nonparametric regression setting. Less known is the model bias incurring from the restriction to the additive class of models. We introduce a new class of estimators that reduces additive model bias and at the same time preserves some stability of the additive estimator. This estimator is shown to partially relieve the dimensionality problem…
We propose a statistical model for networks of event count sequences built on a cascade structure. We assume that each event triggers successor events, whose counts follow additive probability distributions; the ensemble of counts is given by their superposition. These assumptions allow the marginal distribution of count sequences and the conditional distribution of event cascades to take analytic forms. We present our model framework using Poisson and negative binomial distributions as the building blocks. Based on this formulation, we describe a statistical method for estimating the model pa…
Partially observed Markov process (POMP) models, also known as hidden Markov models or state space models, are ubiquitous tools for time series analysis. The R package pomp provides a very flexible framework for Monte Carlo statistical investigations using nonlinear, non-Gaussian POMP models. A range of modern statistical methods for POMP models have been implemented in this framework including sequential Monte Carlo, iterated filtering, particle Markov chain Monte Carlo, approximate Bayesian computation, maximum synthetic likelihood estimation, nonlinear forecasting, and trajectory matching. …
This article deals with the problem of functional classification for L2-valued random covariates when some of the covariates may have missing or unobservable fragments. Here, it is allowed for both the training sample as well as the new unclassified observation to have missing fragments in their functional covariates. Furthermore, unlike most previous results in the literature, where covariate fragments are typically assumed to be missing completely at random, we do not impose any such assumptions here. Given the observed segments of the curves, we construct a kernel-type classifier which is q…
We propose a comprehensive framework for additive regression models for non-Gaussian functional responses, allowing for multiple (partially) nested or crossed functional random effects with flexible correlation structures for, e.g., spatial, temporal, or longitudinal functional data as well as linear and nonlinear effects of functional and scalar covariates that may vary smoothly over the index of the functional response. Our implementation handles functional responses from any exponential family distribution as well as many others like Beta- or scaled non-central $t$-distributions. Developmen…
Conditional copulas are flexible statistical tools that couple joint conditional and marginal conditional distributions. In a linear regression setting with more than one covariate and two dependent outcomes, we propose the use of additive models for conditional bivariate copula models and discuss computation and model selection tools for performing Bayesian inference. The method is illustrated using simulations and a real example.
Recently, conditional average treatment effect (CATE) estimation has been attracting much attention due to its importance in various fields such as statistics, social and biomedical sciences. This study proposes a partially linear nonparametric Bayes model for the heterogeneous treatment effect estimation. A partially linear model is a semiparametric model that consists of linear and nonparametric components in an additive form. A nonparametric Bayes model that uses a Gaussian process to model the nonparametric component has already been studied. However, this model cannot handle the heterogen…
Generalized additive models (GAMs) are a well-established statistical tool for modeling complex nonlinear relationships between covariates and a response assumed to have a conditional distribution in the exponential family. In this article, P-splines and the Laplace approximation are coupled for flexible and fast approximate Bayesian inference in GAMs. The proposed Laplace-P-spline model contributes to the development of a new methodology to explore the posterior penalty space by considering a deterministic grid-based strategy or a Markov chain sampler, depending on the number of smooth additi…
Additive regression models are actively researched in the statistical field because of their usefulness in the analysis of responses determined by non-linear relationships with multivariate predictors. In this kind of statistical models, the response depends linearly on unknown functions of predictor variables and typically, the goal of the analysis is to make inference about these functions. In this paper, we study the problem of additive regression using a least squares approach based on periodic orthogonal wavelets on the interval [0,1]. For this estimator, we obtain strong consistency (wit…
From Statistical to Structural Synergy: A Predictability Framework to Quantify the Effects due to High-Order Mechanisms
University of Palermo（巴勒莫大学）；Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, National Research Council（国家研究委员会先进制造智能工业技术与系统研究所）；University of Ghent（根特大学）；Faculty of Technical Sciences, University of Novi Sad（诺维萨德大学技术科学学院）；University of Bari Aldo Moro（巴里阿尔多·莫罗大学）；Istituto Nazionale di Fisica Nucleare, Sezione di Bari（意大利国家核物理研究所巴里分部）
Generalized additive model is a powerful statistical learning and predictive modeling tool that has been applied in a wide range of applications. The need of high-dimensional additive modeling is eminent in the context of dealing with high through-put data such as genetic data analysis. In this article, we studied a two step selection and estimation method for ultra high dimensional generalized additive models. The first step applies group lasso on the expanded bases of the functions. With high probability this selects all nonzero functions without having too much over selection. The second st…
Over the course of the COVID-19 pandemic, Generalised Additive Models (GAMs) have been successfully employed on numerous occasions to obtain vital data-driven insights. In this paper we further substantiate the success story of GAMs, demonstrating their flexibility by focusing on three relevant pandemic-related issues. First, we examine the interdepency among infections in different age groups, concentrating on school children. In this context, we derive the setting under which parameter estimates are independent of the (unknown) case-detection ratio, which plays an important role in COVID-19 …
