Selecting Local Models in Multiple Regression by Maximizing Power

Consistent Variable Selection for Functional Regression Models

The dual problem of testing the predictive significance of a particular covariate, and identification of the set of relevant covariates is common in applied research and methodological investigations. To study this problem in the context of functional linear regression models with predictor variables observed over a grid and a scalar response, we consider basis expansions of the functional covariates and apply the likelihood ratio test. Based on p-values from testing each predictor, we propose a new variable selection method, which is consistent in selecting the relevant predictors from set of…

Piecewise regression mixture for simultaneous functional data clustering and optimal segmentation

This paper introduces a novel mixture model-based approach for simultaneous clustering and optimal segmentation of functional data which are curves presenting regime changes. The proposed model consists in a finite mixture of piecewise polynomial regression models. Each piecewise polynomial regression model is associated with a cluster, and within each cluster, each piecewise polynomial component is associated with a regime (i.e., a segment). We derive two approaches for learning the model parameters. The former is an estimation approach and consists in maximizing the observed-data likelihood …

Likelihood approach for marginal proportional hazards regression in the presence of dependent censoring

In many public health problems, an important goal is to identify the effect of some treatment/intervention on the risk of failure for the whole population. A marginal proportional hazards regression model is often used to analyze such an effect. When dependent censoring is explained by many auxiliary covariates, we utilize two working models to condense high-dimensional covariates to achieve dimension reduction. Then the estimator of the treatment effect is obtained by maximizing a pseudo-likelihood function over a sieve space. Such an estimator is shown to be consistent and asymptotically nor…

通过错误覆盖率控制选择有信息量的保形预测集

Selecting informative conformal prediction sets with false coverage rate control

本文提出 InfoSP 与 InfoSCOP，在分类和回归中经规则选择小而有信息量的保形预测集，并对所选样本实现有限样本 FCR 控制。

Assessing prediction error of nonparametric regression and classification under Bregman divergence

Prediction error is critical to assessing the performance of statistical methods and selecting statistical models. We propose the cross-validation and approximated cross-validation methods for estimating prediction error under a broad q-class of Bregman divergence for error measures which embeds nearly all of the commonly used loss functions in regression, classification procedures and machine learning literature. The approximated cross-validation formulas are analytically derived, which facilitate fast estimation of prediction error under the Bregman divergence. We then study a data-driven op…

Latent function-on-scalar regression models for observed sequences of binary data: a restricted likelihood approach

In this paper, we study a functional regression setting where the random response curve is unobserved, and only its dichotomized version observed at a sequence of correlated binary data is available. We propose a practical computational framework for maximum likelihood analysis via the parameter expansion technique. Compared to existing methods, our proposal relies on the use of a complete data likelihood, with the advantage of being able to handle non-equally spaced and missing observations effectively. The proposed method is used in the Function-on-Scalar regression setting, with the latent …

Data-driven calibration of linear estimators with minimal penalties

This paper tackles the problem of selecting among several linear estimators in non-parametric regression; this includes model selection for linear regression, the choice of a regularization parameter in kernel ridge regression, spline smoothing or locally weighted regression, and the choice of a kernel in multiple kernel learning. We propose a new algorithm which first estimates consistently the variance of the noise, based upon the concept of minimal penalty, which was previously introduced in the context of model selection. Then, plugging our variance estimate in Mallows' $C_L$ penalty is pr…

Lasso回归推断的模拟校准检验

Simulation-calibration testing for inference in Lasso regressions

本文提出一种基于模拟校准的条件p值检验方法，用于检验Lasso路径上变量的显著性并控制族错误率，从而在控制假阳性风险的前提下选择Lasso路径模型。

通过核回归理解对抗训练中的过拟合

Understanding Overfitting in Adversarial Training via Kernel Regression

本文在再生核希尔伯特空间中建立对抗训练与噪声数据增强的极限公式，证明缺乏适当正则化时二者比标准核回归更易过拟合，而适当正则化可使其获得更小的泛化误差和Lipschitz常数。