Asymptotics for penalized spline estimators in quantile regression

极值分位数回归中GEV参数的惩罚估计

Penalized estimation of GEV parameters for extreme quantile regression

本文提出基于GEV分布和广义随机森林的惩罚似然方法，解决高概率分位数回归中的尾部外推难题和高维预测变量复杂结构问题。

Wild Residual Bootstrap Inference for Penalized Quantile Regression with Heteroscedastic Errors

We consider a heteroscedastic regression model in which some of the regression coefficients are zero but it is not known which ones. Penalized quantile regression is a useful approach for analyzing such data. By allowing different covariates to be relevant for modeling conditional quantile functions at different quantile levels, it provides a more complete picture of the conditional distribution of a response variable than mean regression. Existing work on penalized quantile regression has been mostly focused on point estimation. Although bootstrap procedures have recently been shown to be eff…

Fast Bivariate Penalized Splines: the Sandwich Smoother

We propose a fast penalized spline method for bivariate smoothing. Univariate P-spline smoothers (Eilers and Marx, 1996) are applied simultaneously along both coordinates. The new smoother has a sandwich form which suggested the name "sandwich smoother" to a referee. The sandwich smoother has a tensor product structure that simplifies an asymptotic analysis and it can be fast computed. We derive a local central limit theorem for the sandwich smoother, with simple expressions for the asymptotic bias and variance, by showing that the sandwich smoother is asymptotically equivalent to a bivariate …

A Unified Algorithm for Penalized Convolution Smoothed Quantile Regression

Penalized quantile regression (QR) is widely used for studying the relationship between a response variable and a set of predictors under data heterogeneity in high-dimensional settings. Compared to penalized least squares, scalable algorithms for fitting penalized QR are lacking due to the non-differentiable piecewise linear loss function. To overcome the lack of smoothness, a recently proposed convolution-type smoothed method brings an interesting tradeoff between statistical accuracy and computational efficiency for both standard and penalized quantile regressions. In this paper, we propose…

Risk Bounds for Quantile Trend Filtering

We study quantile trend filtering, a recently proposed method for nonparametric quantile regression with the goal of generalizing existing risk bounds known for the usual trend filtering estimators which perform mean regression. We study both the penalized and the constrained version (of order $r \geq 1$) of univariate quantile trend filtering. Our results show that both the constrained and the penalized version (of order $r \geq 1$) attain the minimax rate up to log factors, when the $(r-1)$th discrete derivative of the true vector of quantiles belongs to the class of bounded variation signal…

High-Dimensional Quantile Regression: Convolution Smoothing and Concave Regularization

$\ell_1$-penalized quantile regression is widely used for analyzing high-dimensional data with heterogeneity. It is now recognized that the $\ell_1$-penalty introduces non-negligible estimation bias, while a proper use of concave regularization may lead to estimators with refined convergence rates and oracle properties as the signal strengthens. Although folded concave penalized $M$-estimation with strongly convex loss functions have been well studied, the extant literature on quantile regression is relatively silent. The main difficulty is that the quantile loss is piecewise linear: it is non…

Quantile regression when the covariates are functions

This paper deals with a linear model of regression on quantiles when the explanatory variable takes values in some functional space and the response is scalar. We propose a spline estimator of the functional coefficient that minimizes a penalized L1 type criterion. Then, we study the asymptotic behavior of this estimator. The penalization is of primary importance to get existence and convergence.

样条分位数回归

Spline Quantile Regression

本文提出样条分位数回归方法，通过联合估计分位数回归系数作为平滑样条，提升估计性能。

使用跨分位数一致变量选择的惩罚分位数回归进行股票溢价的分位数预测

Quantile Predictions for Equity Premium using Penalized Quantile Regression with Consistent Variable Selection across Multiple Quantiles

本文提出一种跨分位数一致变量选择的惩罚分位数回归方法，用于股票溢价预测，解决均值回归在异方差和重尾下的问题，并通过模拟和实证验证其优于基准方法，且发现预测变量关系随分位数反转。

样条分位数回归与三次和线性平滑样条

Spline Quantile Regression with Cubic and Linear Smoothing Splines

本文提出了一种基于样条的分位数回归方法，通过引入三次和线性平滑样条，改进了回归系数的平滑估计，展示了其在实际数据中的应用价值。

Inference for penalized spline regression: Improving confidence intervals by reducing the penalty

Penalized spline regression is a popular method for scatterplot smoothing, but there has long been a debate on how to construct confidence intervals for penalized spline fits. Due to the penalty, the fitted smooth curve is a biased estimate of the target function. Many methods, including Bayesian intervals and the simple-shift bias-reduction, have been proposed to upgrade the coverage of the confidence intervals, but these methods usually fail to adequately improve the situation at predictor values where the function is sharply curved. In this paper, we develop a novel approach to improving th…

Local Asymptotics of P-splines

This report studies local asymptotics of P-splines with $p$th degree B-splines and a $m$th order difference penalty. Earlier work with $p$ and $m$ restricted is extended to the general case. Asymptotically, penalized splines are kernel estimators with equivalent kernels depending on $m$, but not on $p$. A central limit theorem provides simple expressions for the asymptotic mean and variance. Provided it is fast enough, the divergence rate of the number of knots does not affect the asymptotic distribution. The optimal convergence rate of the penalty parameter is given.

L1-Penalized Quantile Regression in High-Dimensional Sparse Models

We consider median regression and, more generally, a possibly infinite collection of quantile regressions in high-dimensional sparse models. In these models the overall number of regressors $p$ is very large, possibly larger than the sample size $n$, but only $s$ of these regressors have non-zero impact on the conditional quantile of the response variable, where $s$ grows slower than $n$. We consider quantile regression penalized by the $\ell_1$-norm of coefficients ($\ell_1$-QR). First, we show that $\ell_1$-QR is consistent at the rate $\sqrt{s/n} \sqrt{\log p}$. The overall number of regres…

Sparse Quantile Regression

We consider both $\ell _{0}$-penalized and $\ell _{0}$-constrained quantile regression estimators. For the $\ell _{0}$-penalized estimator, we derive an exponential inequality on the tail probability of excess quantile prediction risk and apply it to obtain non-asymptotic upper bounds on the mean-square parameter and regression function estimation errors. We also derive analogous results for the $\ell _{0}$-constrained estimator. The resulting rates of convergence are nearly minimax-optimal and the same as those for $\ell _{1}$-penalized and non-convex penalized estimators. Further, we charact…

样条自回归方法用于量纲谱估计

Spline Autoregression Method for Estimation of Quantile Spectrum

本文提出样条自回归方法用于估计时间序列的量纲谱，通过量纲离散傅里叶变换和时间域量纲序列，利用平滑样条建模量纲水平与自回归系数的关系，验证了该方法在非线性动态中的有效性。

Semismooth Newton Coordinate Descent Algorithm for Elastic-Net Penalized Huber Loss Regression and Quantile Regression

We propose an algorithm, semismooth Newton coordinate descent (SNCD), for the elastic-net penalized Huber loss regression and quantile regression in high dimensional settings. Unlike existing coordinate descent type algorithms, the SNCD updates each regression coefficient and its corresponding subgradient simultaneously in each iteration. It combines the strengths of the coordinate descent and the semismooth Newton algorithm, and effectively solves the computational challenges posed by dimensionality and nonsmoothness. We establish the convergence properties of the algorithm. In addition, we p…

Group Lasso for high dimensional sparse quantile regression models

This paper studies the statistical properties of the group Lasso estimator for high dimensional sparse quantile regression models where the number of explanatory variables (or the number of groups of explanatory variables) is possibly much larger than the sample size while the number of variables in "active" groups is sufficiently small. We establish a non-asymptotic bound on the $\ell_{2}$-estimation error of the estimator. This bound explains situations under which the group Lasso estimator is potentially superior/inferior to the $\ell_{1}$-penalized quantile regression estimator in terms of…

Copula based generalized additive models with non-random sample selection

Non-random sample selection is a commonplace amongst many empirical studies and it appears when an output variable of interest is available only for a restricted non-random sub-sample of data. We introduce an extension of the generalized additive model which accounts for non-random sample selection by using a selection equation. The proposed approach allows for different distributions of the outcome variable, various dependence structures between the (outcome and selection) equations through the use of copulae, and nonparametric effects on the responses. Parameter estimation is carried out wit…

Advanced Algorithms for Penalized Quantile and Composite Quantile Regression

In this paper, we discuss a family of robust, high-dimensional regression models for quantile and composite quantile regression, both with and without an adaptive lasso penalty for variable selection. We reformulate these quantile regression problems and obtain estimators by applying the alternating direction method of multipliers (ADMM), majorize-minimization (MM), and coordinate descent (CD) algorithms. Our new approaches address the lack of publicly available methods for (composite) quantile regression, especially for high-dimensional data, both with and without regularization. Through simu…

空间函数-函数分位数回归

Spatial function-on-function quantile regression

本文提出SpatialFoFReg R包实现的空间函数-函数分位数回归框架，通过函数型空间自回归结构与两阶段工具变量法，在PM₂.₅等数据上表现优于非空间及均值模型，用于环境风险管理等领域。

非交叉凸分位数回归

Non-crossing convex quantile regression

针对凸分位数回归中的分位数交叉问题，本文提出惩罚凸分位数回归方法，在避免交叉的同时保持分位数性质，蒙特卡洛实验验证其优越性。

不规则空间域上散布数据的非参数密度估计：一种使用二元惩罚样条平滑的似然方法

Nonparametric Density Estimation for Data Scattered on Irregular Spatial Domains: A Likelihood-Based Approach Using Bivariate Penalized Spline Smoothing

针对不规则空间域数据，本文提出基于三角剖分二元惩罚样条平滑的似然非参数密度估计方法，证明其渐近收敛速率，并通过模拟和波特兰盗车数据验证其优势。

On a phase transition in general order spline regression

In the Gaussian sequence model $Y= θ_0 + \varepsilon$ in $\mathbb{R}^n$, we study the fundamental limit of approximating the signal $θ_0$ by a class $Θ(d,d_0,k)$ of (generalized) splines with free knots. Here $d$ is the degree of the spline, $d_0$ is the order of differentiability at each inner knot, and $k$ is the maximal number of pieces. We show that, given any integer $d\geq 0$ and $d_0\in\{-1,0,\ldots,d-1\}$, the minimax rate of estimation over $Θ(d,d_0,k)$ exhibits the following phase transition: \begin{equation*} \begin{aligned} \inf_{\widetildeθ}\sup_{θ\inΘ(d,d_0, k)}\mathbb{E}_θ\|\wid…

惩罚函数型线性分位数回归

Penalized function-on-function linear quantile regression

本文提出一种函数型线性分位数回归模型，利用张量三次B样条和二次惩罚估计参数函数，通过BIC网格搜索选择平滑参数，在蒙特卡洛实验和玛丽河数据中优于现有方法。

扩展ADMM用于大数据中带线性约束的一般惩罚分位数回归

Extended ADMM for general penalized quantile regression with linear constraints in big data

本文提出基于多块ADMM的扩展算法，用于高效求解大数据中带线性约束的一般惩罚分位数回归，并开发了并行版本，通过显式参数更新避免嵌套循环，实验证明其计算效率显著提升。

使用成分样条的概率密度函数惩罚似然估计

Penalized likelihood estimation of probability density functions using compositional splines

该研究针对概率密度函数两阶段估计的偏差问题，提出了基于贝叶斯希尔伯特空间、中心对数比变换和ZB样条的惩罚最大似然框架，经模拟和地球化学数据验证了其有效性。

非凸惩罚下稀疏惩罚分位数回归的平滑ADMM算法

Smoothing ADMM for Sparse-Penalized Quantile Regression with Non-Convex Penalties

本文研究了非凸非光滑稀疏惩罚下的分位数回归问题，提出了一种名为SIAD的单循环平滑ADMM算法，通过递增惩罚参数加速收敛，理论上确认了子梯度界收敛率为o(k^{-1/4})，数值实验证明其优于现有方法。

Bayesian Non-parametric Simultaneous Quantile Regression for Complete and Grid Data

In this paper, we consider Bayesian methods for non-parametric quantile regressions with multiple continuous predictors ranging values in the unit interval. In the first method, the quantile function is assumed to be smooth over the explanatory variable and is expanded in tensor product of B-spline basis functions. While in the second method, the distribution function is assumed to be smooth over the explanatory variable and is expanded in tensor product of B-spline basis functions. Unlike other existing methods of non-parametric quantile regressions, the proposed methods estimate the whole qu…

基于深度回声状态网络与惩罚分位数回归的准周期气候过程校准预测

Calibrated Forecasts of Quasi-Periodic Climate Processes with Deep Echo State Networks and Penalized Quantile Regression

本文提出用深度回声状态网络与惩罚分位数回归预测海洋驱动的准周期气候事件，并通过快速集合方法校准预测不确定性，从而提升物理模型难以刻画过程的预测能力。

基于惩罚样条的LISA噪声贝叶斯功率谱密度估计方法

Bayesian power spectral density estimation for LISA noise based on penalized splines with a parametric boost

本文提出一种贝叶斯方法，利用惩罚样条和参数模型结合，准确估计长时序噪声功率谱密度，适用于LISA数据分析，减少误差并提高精度。

Quantile regression with varying coefficients

Quantile regression provides a framework for modeling statistical quantities of interest other than the conditional mean. The regression methodology is well developed for linear models, but less so for nonparametric models. We consider conditional quantiles with varying coefficients and propose a methodology for their estimation and assessment using polynomial splines. The proposed estimators are easy to compute via standard quantile regression algorithms and a stepwise knot selection algorithm. The proposed Rao-score-type test that assesses the model against a linear model is also easy to imp…

Asgl：用于惩罚线性和分位数回归的Python包

Asgl: A Python Package for Penalized Linear and Quantile Regression

Asgl是用于惩罚线性和分位数回归的Python包，实现多种惩罚技术，支持自适应惩罚，提供惩罚分位数回归功能，通过主要类Regressor与scikit-learn集成，在变量选择和预测任务中实用，是现代统计建模的综合工具。

通过 B 样条插值进行极端条件分位数的在线预测

Online Prediction of Extreme Conditional Quantiles via B-Spline Interpolation

针对条件极端分位数在线预测中GPD拟合计算成本高的问题，提出基于插值的EMI算法，结合分位数回归与B样条插值实现高效预测。

Quantile regression in high-dimension with breaking

The paper considers a linear regression model in high-dimension for which the predictive variables can change the influence on the response variable at unknown times (called change-points). Moreover, the particular case of the heavy-tailed errors is considered. In this case, least square method with LASSO or adaptive LASSO penalty can not be used since the theoretical assumptions do not occur or the estimators are not robust. Then, the quantile model with SCAD penalty or median regression with LASSO-type penalty allows, in the same time, to estimate the parameters on every segment and eliminat…

带案例权重调整解路径的惩罚分位数回归交叉验证

Cross Validation for Penalized Quantile Regression with a Case-Weight Adjusted Solution Path

针对惩罚分位数回归交叉验证计算量大、近似方案在极端分位数效果差的问题，提出基于案例权重调整解路径的精确留一交叉验证算法，借助同伦技术高效计算，还能考察案例影响，在实际应用中验证了实用性。

Censored quantile regression processes under dependence and penalization

We consider quantile regression processes from censored data under dependent data structures and derive a uniform Bahadur representation for those processes. We also consider cases where the dimension of the parameter in the quantile regression model is large. It is demonstrated that traditional penalized estimators such as the adaptive lasso yield sub-optimal rates if the coefficients of the quantile regression cross zero. New penalization techniques are introduced which are able to deal with specific problems of censored data and yield estimates with an optimal rate. In contrast to most of t…

Scalable estimation and inference for censored quantile regression process

Censored quantile regression (CQR) has become a valuable tool to study the heterogeneous association between a possibly censored outcome and a set of covariates, yet computation and statistical inference for CQR have remained a challenge for large-scale data with many covariates. In this paper, we focus on a smoothed martingale-based sequential estimating equations approach, to which scalable gradient-based algorithms can be applied. Theoretically, we provide a unified analysis of the smoothed sequential estimator and its penalized counterpart in increasing dimensions. When the covariate dimen…

具有非光滑目标函数的 M 型平滑样条的渐近理论

Asymptotics for M-type smoothing splines with non-smooth objective functions

本文研究具有非光滑目标函数的 M 型平滑样条的渐近理论，证明在仅假设凸性条件下最小二乘收敛速率可扩展至 M 型估计量，并建立导数最优收敛速率，填补了分位数和 Huber 型等稳健估计量的理论空白。

用于惩罚分位数回归的分布式随机平滑交替方向乘子法（DSS-ADMM）

Distributed Stochastic Smoothing ADMM for Penalized Quantile Regression

针对大型分布式数据下惩罚分位数回归的计算难题，提出分布式随机平滑ADMM算法，通过Huber平滑检查损失实现并行计算，在同质与异质数据分区下均表现良好，阐明了相关权衡关系。

HAL-MLE对数样条密度估计（第一部分：单变量）

HAL-MLE Log-Splines Density Estimation (Part I: Univariate)

本文提出基于HAL框架的HAL-MLE方法，用于单变量总变差惩罚密度估计，并建立了该方法的理论收敛性结果。

Penalized regression with multiple loss functions and selection by vote

This article considers a linear model in a high dimensional data scenario. We propose a process which uses multiple loss functions both to select relevant predictors and to estimate parameters, and study its asymptotic properties. Variable selection is conducted by a procedure called "vote", which aggregates results from penalized loss functions. Using multiple objective functions separately simplifies algorithms and allows parallel computing, which is convenient and fast. As a special example we consider a quantile regression model, which optimally combines multiple quantile levels. We show t…

考虑异质性的高维分位数回归的全局推断

Global Inference for High Dimensional Quantile Regression with Heterogeneity

针对高维分位数回归现有推断方法的局限，提出全局自适应LLA算法等方法，实现异方差模型下多量分位点线性假设的统一检验，扩展了部分惩罚推断的校准范围。

Partially linear additive quantile regression in ultra-high dimension

We consider a flexible semiparametric quantile regression model for analyzing high dimensional heterogeneous data. This model has several appealing features: (1) By considering different conditional quantiles, we may obtain a more complete picture of the conditional distribution of a response variable given high dimensional covariates. (2) The sparsity level is allowed to be different at different quantile levels. (3) The partially linear additive structure accommodates nonlinearity and circumvents the curse of dimensionality. (4) It is naturally robust to heavy-tailed distributions. In this p…

Variable Selection for Additive Partial Linear Quantile Regression with Missing Covariates

The standard quantile regression model assumes a linear relationship at the quantile of interest and that all variables are observed. We relax these assumptions by considering a partial linear model while allowing for missing linear covariates. To handle the potential bias caused by missing data we propose a weighted objective function using inverse probability weighting. Our work examines estimators using parametric and nonparametric estimates of the missing probability. For variable selection of the linear terms in the presence of missing data we consider a penalized and weighted objective f…

总变分去噪在分位数回归中的精确点wise描述

An Exact Pointwise Characterization for Total Variation Denoising in Quantile Regression

本文通过推导分位数总变分去噪估计器的精确minmax/maxmin表示，提供了其解集的完整点wise描述，并揭示了其结构特性及在分位数水平上的非交叉性。

Naive Penalized Spline Estimators of Derivatives Achieve Optimal Rates of Convergence

This paper studies the asymptotic behavior of penalized spline estimates of derivatives. In particular, we show that simply differentiating the penalized spline estimator of the mean regression function itself to estimate the corresponding derivative achieves the optimal L2 rate of convergence.

A data-driven P-spline smoother and the P-Spline-GARCH-models

Penalized spline smoothing of time series and its asymptotic properties are studied. A data-driven algorithm for selecting the smoothing parameter is developed. The proposal is applied to define a semiparametric extension of the well-known Spline-GARCH, called a P-Spline-GARCH, based on the log-data transformation of the squared returns. It is shown that now the errors process is exponentially strong mixing with finite moments of all orders. Asymptotic normality of the P-spline smoother in this context is proved. Practical relevance of the proposal is illustrated by data examples and simulatio…

Permutation tests for assessing potential non-linear associations between treatment use and multivariate clinical outcomes

In many psychometric applications, the relationship between the mean of an outcome and a quantitative covariate is too complex to be described by simple parametric functions; instead, flexible nonlinear relationships can be incorporated using penalized splines. Penalized splines can be conveniently represented as a linear mixed effects model (LMM), where the coefficients of the spline basis functions are random effects. The LMM representation of penalized splines makes the extension to multivariate outcomes relatively straightforward. In the LMM, no effect of the quantitative covariate on the …

Sparse Wavelet Estimation in Quantile Regression with Multiple Functional Predictors

In this manuscript, we study quantile regression in partial functional linear model where response is scalar and predictors include both scalars and multiple functions. Wavelet basis are adopted to better approximate functional slopes while effectively detect local features. The sparse group lasso penalty is imposed to select important functional predictors while capture shared information among them. The estimation problem can be reformulated into a standard second-order cone program and then solved by an interior point method. We also give a novel algorithm by using alternating direction met…

Nonparametric Quantile Regression: Non-Crossing Constraints and Conformal Prediction

We propose a nonparametric quantile regression method using deep neural networks with a rectified linear unit penalty function to avoid quantile crossing. This penalty function is computationally feasible for enforcing non-crossing constraints in multi-dimensional nonparametric quantile regression. We establish non-asymptotic upper bounds for the excess risk of the proposed nonparametric quantile regression function estimators. Our error bounds achieve optimal minimax rate of convergence for the Holder class, and the prefactors of the error bounds depend polynomially on the dimension of the pr…

高维复合分位数回归：最优统计保证与快速算法

High-Dimensional Composite Quantile Regression: Optimal Statistical Guarantees and Fast Algorithms

针对高维重尾误差回归，提出卷积平滑复合分位数回归结合迭代重加权ℓ1正则化，采用梯度算法实现近最优统计保证和显著计算加速。

Penalized Partial Least Squares Based on B-Splines Transformations

We propose a novel method to model nonlinear regression problems by adapting the principle of penalization to Partial Least Squares (PLS). Starting with a generalized additive model, we expand the additive component of each variable in terms of a generous amount of B-Splines basis functions. In order to prevent overfitting and to obtain smooth functions, we estimate the regression model by applying a penalized version of PLS. Although our motivation for penalized PLS stems from its use for B-Splines transformed data, the proposed approach is very general and can be applied to other penalty ter…

稀疏函数型数据主成分的惩罚样条估计：收敛速率

Penalized spline estimation of principal components for sparse functional data: rates of convergence

本文研究稀疏函数型数据下惩罚样条主成分估计的收敛速率，揭示平滑性、样条参数等因子对渐近行为的影响，并分类七种情形下的极小极大最优性。

Quantile Graphical Models: Prediction and Conditional Independence with Applications to Systemic Risk

We propose two types of Quantile Graphical Models (QGMs) --- Conditional Independence Quantile Graphical Models (CIQGMs) and Prediction Quantile Graphical Models (PQGMs). CIQGMs characterize the conditional independence of distributions by evaluating the distributional dependence structure at each quantile index. As such, CIQGMs can be used for validation of the graph structure in the causal graphical models (\cite{pearl2009causality, robins1986new, heckman2015causal}). One main advantage of these models is that we can apply them to large collections of variables driven by non-Gaussian and non…