A basis expansion with regularization methods is much appealing to the flexible or robust nonlinear regression models for data with complex structures. When the underlying function has inhomogeneous smoothness, it is well known that conventional reguralization methods do not perform well. In this case, an adaptive procedure such as a free-knot spline or a local likelihood method is often introduced as an effective method. However, both methods need intensive computational loads. In this study, we consider a new efficient basis expansion by proposing a smoothly varying regularization method which is constructed by some special penalties. We call them adaptive-type penalties. In our modeling, adaptive-type penalties play key rolls and it has been successful in giving good estimation for inhomogeneous smoothness functions. A crucial issue in the modeling process is the choice of a suitable model among candidates. To select the suitable model, we derive an approximated generalized information criterion (GIC). The proposed method is investigated through Monte Carlo simulations and real data analysis. Numerical results suggest that our method performs well in various situations.
With a focus on linear models with smooth functional covariates, we propose a penalization framework (SACR) based on the nonzero centered ridge, where the center of the penalty is optimally reweighted in a supervised way, starting from the ordinary ridge solution as the initial centerfunction. In particular, we introduce a convex formulation that jointly estimates the model's coefficients and the weight function, with a roughness penalty on the centerfunction and constraints on the weights in order to recover a possibly smooth and/or sparse solution. This allows for a non-iterative and continu…
Penalized selection criteria like AIC or BIC are among the most popular methods for variable selection. Their theoretical properties have been studied intensively and are well understood, but making use of them in case of high-dimensional data is difficult due to the non-convex optimization problem induced by L0 penalties. An elegant solution to this problem is provided by the multi-step adaptive lasso, where iteratively weighted lasso problems are solved, whose weights are updated in such a way that the procedure converges towards selection with L0 penalties. In this paper we introduce an ada…
Features in predictive models are not exchangeable, yet common supervised models treat them as such. Here we study ridge regression when the analyst can partition the features into $K$ groups based on external side-information. For example, in high-throughput biology, features may represent gene expression, protein abundance or clinical data and so each feature group represents a distinct modality. The analyst's goal is to choose optimal regularization parameters $λ= (λ_1, \dotsc, λ_K)$ -- one for each group. In this work, we study the impact of $λ$ on the predictive risk of group-regularized …
The University of Washington eScience Institute（华盛顿大学电子科学研究所）；University of Washington（华盛顿大学）；Center for Magnetic Resonance Research（磁共振研究中心）；University of Minnesota（明尼苏达大学）
Ridge regression (RR) is a regularization technique that penalizes the L2-norm of the coefficients in linear regression. One of the challenges of using RR is the need to set a hyperparameter ($α$) that controls the amount of regularization. Cross-validation is typically used to select the best $α$ from a set of candidates. However, efficient and appropriate selection of $α$ can be challenging, particularly where large amounts of data are analyzed. Because the selected $α$ depends on the scale of the data and predictors, it is not straightforwardly interpretable. Here, we propose to reparameter…
Censored broken adaptive ridge rank regression via induced smoothing
School of Mathematics, Statistics and Data Science, Sungshin Women’s University（成均馆大学数学、统计与数据科学学院）；Sungshin Women’s University（诚信女子大学）；Department of Biostatistics, Virginia Commonwealth University（弗吉尼亚联邦大学生物统计系）；Virginia Commonwealth University（弗吉尼亚联邦大学）；Center for Data Science, Sungshin Women’s University（诚信女子大学数据科学中心）
We consider the problem of classification of functional data into two groups by linear classifiers based on one-dimensional projections of functions. We reformulate the task to find the best classifier as an optimization problem and solve it by regularization techniques, namely the conjugate gradient method with early stopping, the principal component method and the ridge method. We study the empirical version with finite training samples consisting of incomplete functions observed on different subsets of the domain and show that the optimal, possibly zero, misclassification probability can be…
This paper tackles the problem of selecting among several linear estimators in non-parametric regression; this includes model selection for linear regression, the choice of a regularization parameter in kernel ridge regression, spline smoothing or locally weighted regression, and the choice of a kernel in multiple kernel learning. We propose a new algorithm which first estimates consistently the variance of the noise, based upon the concept of minimal penalty, which was previously introduced in the context of model selection. Then, plugging our variance estimate in Mallows' $C_L$ penalty is pr…
