Generalized additive models have been popular among statisticians and data analysts in multivariate nonparametric regression with non-Gaussian responses including binary and count data. In this paper, a new likelihood approach for fitting generalized additive models is proposed. It aims to maximize a smoothed likelihood. The additive functions are estimated by solving a system of nonlinear integral equations. An iterative algorithm based on smooth backfitting is developed from the Newton--Kantorovich theorem. Asymptotic properties of the estimator and convergence of the algorithm are discussed. It is shown that our proposal based on local linear fit achieves the same bias and variance as the oracle estimator that uses knowledge of the other components. Numerical comparison with the recently proposed two-stage estimator [Ann. Statist. 32 (2004) 2412--2443] is also made.
In this paper, we study the ordinary backfitting and smooth backfitting as methods of fitting additive quantile models. We show that these backfitting quantile estimators are asymptotically equivalent to the corresponding backfitting estimators of the additive components in a specially-designed additive mean regression model. This implies that the theoretical properties of the backfitting quantile estimators are not unlike those of backfitting mean regression estimators. We also assess the finite sample properties of the two backfitting quantile estimators.
In this paper a new smooth backfitting estimate is proposed for additive regression models. The estimate has the simple structure of Nadaraya--Watson smooth backfitting but at the same time achieves the oracle property of local linear smooth backfitting. Each component is estimated with the same asymptotic accuracy as if the other components were known.
The additive model is one of the most popular semiparametric models. The backfitting estimation (Buja, Hastie and Tibshirani, 1989, \textit{Ann. Statist.}) for the model is intuitively easy to understand and theoretically most efficient (Opsomer and Ruppert, 1997, \textit{Ann. Statist.}); its implementation is equivalent to solving simple linear equations. However, convergence of the algorithm is very difficult to investigate and is still unsolved. For bivariate additive models, Opsomer and Ruppert (1997, \textit{Ann. Statist.}) proved the convergence under a very strong condition and conjectu…
The smooth backfitting introduced by Mammen, Linton and Nielsen [Ann. Statist. 27 (1999) 1443-1490] is a promising technique to fit additive regression models and is known to achieve the oracle efficiency bound. In this paper, we propose and discuss three fully automated bandwidth selection methods for smooth backfitting in additive models. The first one is a penalized least squares approach which is based on higher-order stochastic expansions for the residual sums of squares of the smooth backfitting estimates. The other two are plug-in bandwidth selectors which rely on approximations of the …
We discuss local linear smooth backfitting for additive non-parametric models. This procedure is well known for achieving optimal convergence rates under appropriate smoothness conditions. In particular, it allows for the estimation of each component of an additive model with the same asymptotic accuracy as if the other components were known. The asymptotic discussion of local linear smooth backfitting is rather complex because typically an overwhelming notation is required for a detailed discussion. In this paper we interpret the local linear smooth backfitting estimator as a projection of th…
Smooth backfitting has proven to have a number of theoretical and practical advantages in structured regression. Smooth backfitting projects the data down onto the structured space of interest providing a direct link between data and estimator. This paper introduces the ideas of smooth backfitting to survival analysis in a proportional hazard model, where we assume an underlying conditional hazard with multiplicative components. We develop asymptotic theory for the estimator and we use the smooth backfitter in a practical application, where we extend recent advances of in-sample forecasting me…
We investigate hypothesis testing in nonparametric additive models estimated using simplified smooth backfitting (Huang and Yu, Journal of Computational and Graphical Statistics, \textbf{28(2)}, 386--400, 2019). Simplified smooth backfitting achieves oracle properties under regularity conditions and provides closed-form expressions of the estimators that are useful for deriving asymptotic properties. We develop a generalized likelihood ratio (GLR) and a loss function (LF) based testing framework for inference. Under the null hypothesis, both the GLR and LF tests have asymptotically rescaled ch…
This paper is about optimal estimation of the additive components of a nonparametric, additive isotone regression model. It is shown that asymptotically up to first order, each additive component can be estimated as well as it could be by a least squares estimator if the other components were known. The algorithm for the calculation of the estimator uses backfitting. Convergence of the algorithm is shown. Finite sample properties are also compared through simulation experiments.
This paper is concerned with asymptotic theory for penalized spline estimator in bivariate additive model. The focus of this paper is put upon the penalized spline estimator obtained by the backfitting algorithm. The convergence of the algorithm as well as the uniqueness of its solution are shown. The asymptotic bias and variance of penalized spline estimator are derived by an efficient use of the asymptotic results for the penalized spline estimator in marginal univariate model. Asymptotic normality of estimator is also developed, by which an approximate confidence interval can be obtained. S…
Additive models and generalized additive models are effective semiparametric tools for multidimensional data. In this article we propose an online smoothing backfitting method for generalized additive models with local polynomial smoothers. The main idea is to use a second order expansion to approximate the nonlinear integral equations to maximize the local quasilikelihood and store the coefficients as the sufficient statistics which can be updated in an online manner by a dynamic candidate bandwidth method. The updating procedure only depends on the stored sufficient statistics and the curren…
Motivated by normalizing DNA microarray data and by predicting the interest rates, we explore nonparametric estimation of additive models with highly correlated covariates. We introduce two novel approaches for estimating the additive components, integration estimation and pooled backfitting estimation. The former is designed for highly correlated covariates, and the latter is useful for nonhighly correlated covariates. Asymptotic normalities of the proposed estimators are established. Simulations are conducted to demonstrate finite sample behaviors of the proposed estimators, and real data ex…
We propose to approximate the conditional expectation of a spatial random variable given its nearest-neighbour observations by an additive function. The setting is meaningful in practice and requires no unilateral ordering. It is capable of catching nonlinear features in spatial data and exploring local dependence structures. Our approach is different from both Markov field methods and disjunctive kriging. The asymptotic properties of the additive estimators have been established for $α$-mixing spatial processes by extending the theory of the backfitting procedure to the spatial case. This fac…
Due to the curse of dimensionality, estimation in a multidimensional nonparametric regression model is in general not feasible. Hence, additional restrictions are introduced, and the additive model takes a prominent place. The restrictions imposed can lead to serious bias. Here, a new estimator is proposed which allows penalizing the nonadditive part of a regression function. This offers a smooth choice between the full and the additive model. As a byproduct, this penalty leads to a regularization in sparse regions. If the additive model does not hold, a small penalty introduces an additional …
Application of nonparametric and semiparametric regression techniques to high-dimensional time series data has been hampered due to the lack of effective tools to address the ``curse of dimensionality.'' Under rather weak conditions, we propose spline-backfitted kernel estimators of the component functions for the nonlinear additive time series data that are both computationally expedient so they are usable for analyzing very high-dimensional time series, and theoretically reliable so inference can be made on the component functions with confidence. Simulation experiments have provided strong …
We consider estimation in a sparse additive regression model with the design points on a regular lattice. We establish the minimax convergence rates over Sobolev classes and propose a Fourier-based rate-optimal estimator which is adaptive to the unknown sparsity and smoothness of the response function. The estimator is derived within Bayesian formalism but can be naturally viewed as a penalized maximum likelihood estimator with the complexity penalties on the number of nonzero univariate additive components of the response and on the numbers of the nonzero coefficients of their Fourer expansio…
In this paper we discuss the estimation of a nonparametric component $f_1$ of a nonparametric additive model $Y=f_1(X_1) + ...+ f_q(X_q) + ε$. We allow the number $q$ of additive components to grow to infinity and we make sparsity assumptions about the number of nonzero additive components. We compare this estimation problem with that of estimating $f_1$ in the oracle model $Z= f_1(X_1) + ε$, for which the additive components $f_2,\dots,f_q$ are known. We construct a two-step presmoothing-and-resmoothing estimator of $f_1$ and state finite-sample bounds for the difference between our estimator…
We study generalized additive partial linear models, proposing the use of polynomial spline smoothing for estimation of nonparametric functions, and deriving quasi-likelihood based estimators for the linear parameters. We establish asymptotic normality for the estimators of the parametric components. The procedure avoids solving large systems of equations as in kernel-based procedures and thus results in gains in computational simplicity. We further develop a class of variable selection procedures for the linear parameters by employing a nonconcave penalized quasi-likelihood, which is shown to…
We consider the efficient estimation of the semiparametric additive transformation model with current status data. A wide range of survival models and econometric models can be incorporated into this general transformation framework. We apply the B-spline approach to simultaneously estimate the linear regression vector, the nondecreasing transformation function, and a set of nonparametric regression functions. We show that the parametric estimate is semiparametric efficient in the presence of multiple nonparametric nuisance functions. An explicit consistent B-spline estimate of the asymptotic …
We establish minimax optimal rates of convergence for estimation in a high dimensional additive model assuming that it is approximately sparse. Our results reveal an interesting phase transition behavior universal to this class of high dimensional problems. In the {\it sparse regime} when the components are sufficiently smooth or the dimensionality is sufficiently large, the optimal rates are identical to those for high dimensional linear regression, and therefore there is no additional cost to entertain a nonparametric model. Otherwise, in the so-called {\it smooth regime}, the rates coincide…
High-dimensional sparse modeling with censored survival data is of great practical importance, as exemplified by modern applications in high-throughput genomic data analysis and credit risk analysis. In this article, we propose a class of regularization methods for simultaneous variable selection and estimation in the additive hazards model, by combining the nonconcave penalized likelihood approach and the pseudoscore method. In a high-dimensional setting where the dimensionality can grow fast, polynomially or nonpolynomially, with the sample size, we establish the weak oracle property and ora…
