Reducing training time by efficient localized kernel regression

基于 tessellation 的局部迁移学习用于非参数回归

Tessellation Localized Transfer learning for nonparametric regression

本文提出一种非参数回归迁移学习框架，通过局部迁移假设建模源-目标关系的异质性，通过联合学习局部迁移函数和目标回归，实现高效迁移并缓解维度灾难。

Slope heuristics and V-Fold model selection in heteroscedastic regression using strongly localized bases

We investigate the optimality for model selection of the so-called slope heuristics, $V$-fold cross-validation and $V$-fold penalization in a heteroscedastic with random design regression context. We consider a new class of linear models that we call strongly localized bases and that generalize histograms, piecewise polynomials and compactly supported wavelets. We derive sharp oracle inequalities that prove the asymptotic optimality of the slope heuristics---when the optimal penalty shape is known---and $V$ -fold penalization. Furthermore, $V$-fold cross-validation seems to be suboptimal for a…

Robustness of shape-restricted regression estimators: an envelope perspective

Classical least squares estimators are well-known to be robust with respect to moment assumptions concerning the error distribution in a wide variety of finite-dimensional statistical problems; generally only a second moment assumption is required for least squares estimators to maintain the same rate of convergence that they would satisfy if the errors were assumed to be Gaussian. In this paper, we give a geometric characterization of the robustness of shape-restricted least squares estimators (LSEs) to error distributions with an $L_{2,1}$ moment, in terms of the `localized envelopes' of the…

Optimal upper and lower bounds for the true and empirical excess risks in heteroscedastic least-squares regression

We consider the estimation of a bounded regression function with nonparametric heteroscedastic noise and random design. We study the true and empirical excess risks of the least-squares estimator on finite-dimensional vector spaces. We give upper and lower bounds on these quantities that are nonasymptotic and optimal to first order, allowing the dimension to depend on sample size. These bounds show the equivalence between the true and empirical excess risks when, among other things, the least-squares estimator is consistent in sup-norm with the projection of the regression function onto the co…

非平稳时间序列局部部分和的泛函中心极限定理

A Functional Central Limit Theorem for Localized Partial Sums of Non-Stationary Time Series

研究非平稳时间序列局部部分和的泛函中心极限定理，通过核加权等方法，在特定条件下证明局部过程弱收敛到高斯分布，可扩展到特定空间，还用于非参数回归检验，展示了方法的性能和适用性。

$L_2$ boosting in kernel regression

In this paper, we investigate the theoretical and empirical properties of $L_2$ boosting with kernel regression estimates as weak learners. We show that each step of $L_2$ boosting reduces the bias of the estimate by two orders of magnitude, while it does not deteriorate the order of the variance. We illustrate the theoretical findings by some simulated examples. Also, we demonstrate that $L_2$ boosting is superior to the use of higher-order kernels, which is a well-known method of reducing the bias of the kernel estimate.