The computational prediction algorithm of neural network, or deep learning, has drawn much attention recently in statistics as well as in image recognition and natural language processing. Particularly in statistical application for censored survival data, the loss function used for optimization has been mainly based on the partial likelihood from Cox's model and its variations to utilize existing neural network library such as Keras, which was built upon the open source library of TensorFlow. This paper presents a novel application of the neural network to the quantile regression for survival data with right censoring, which is adjusted by the inverse of the estimated censoring distribution in the check function. The main purpose of this work is to show that the deep learning method could be flexible enough to predict nonlinear patterns more accurately compared to existing quantile regression methods such as traditional linear quantile regression and nonparametric quantile regression with total variation regularization, emphasizing practicality of the method for censored survival data. Simulation studies were performed to generate nonlinear censored survival data and compare the deep learning method with existing quantile regression methods in terms of prediction accuracy. The proposed method is illustrated with two publicly available breast cancer data sets with gene signatures. The method has been built into a package and is freely available at \url{https://github.com/yicjia/DeepQuantreg}.
This paper studies a semiparametric quantile regression model with endogenous variables and random right censoring. The endogeneity issue is solved using instrumental variables. It is assumed that the structural quantile of the logarithm of the outcome variable is linear in the covariates and censoring is independent. The regressors and instruments can be either continuous or discrete. The specification generates a continuum of equations of which the quantile regression coefficients are a solution. Identification is obtained when this system of equations has a unique solution. Our estimation p…
University of California San Diego（加州大学圣地亚哥分校）；Alexa AI（Alexa人工智能部门）
Random forests are powerful non-parametric regression method but are severely limited in their usage in the presence of randomly censored observations, and naively applied can exhibit poor predictive performance due to the incurred biases. Based on a local adaptive representation of random forests, we develop its regression adjustment for randomly censored regression quantile models. Regression adjustment is based on a new estimating equation that adapts to censoring and leads to quantile score whenever the data do not exhibit censoring. The proposed procedure named {\it censored quantile regr…
Department of Mathematics University of California San Diego（加利福尼亚大学圣地亚哥分校数学系）
Random forests are powerful non-parametric regression method but are severely limited in their usage in the presence of randomly censored observations, and naively applied can exhibit poor predictive performance due to the incurred biases. Based on a local adaptive representation of random forests, we develop its regression adjustment for randomly censored regression quantile models. Regression adjustment is based on new estimating equations that adapt to censoring and lead to quantile score whenever the data do not exhibit censoring. The proposed procedure named censored quantile regression f…
In this paper, we develop a new censored quantile instrumental variable (CQIV) estimator and describe its properties and computation. The CQIV estimator combines Powell (1986) censored quantile regression (CQR) to deal with censoring, with a control variable approach to incorporate endogenous regressors. The CQIV estimator is obtained in two stages that are non-additive in the unobservables. The first stage estimates a non-additive model with infinite dimensional parameters for the control variable, such as a quantile or distribution regression model. The second stage estimates a non-additive …
Conditional quantiles provide a natural tool for reporting results from regression analyses based on semiparametric transformation models. We consider their estimation and construction of confidence sets in the presence of censoring.
The inactivity time, or lost lifespan specifically for mortality data, concerns time from occurrence of an event of interest to the current time point and has recently emerged as a new summary measure for cumulative information inherent in time-to-event data. This summary measure provides several benefits over the traditional methods, including more straightforward interpretation yet less sensitivity to heavy censoring. However, there exists no systematic modeling approach to inferring the quantile inactivity time in the literature. In this paper, we propose a regression method for the quantil…
In this paper, we study a novel approach for the estimation of quantiles when facing potential right censoring of the responses. Contrary to the existing literature on the subject, the adopted strategy of this paper is to tackle censoring at the very level of the loss function usually employed for the computation of quantiles, the so-called "check" function. For interpretation purposes, a simple comparison with the latter reveals how censoring is accounted for in the newly proposed loss function. Subsequently, when considering the inclusion of covariates for conditional quantile estimation, by…
In many applications, heterogeneous treatment effects on a censored response variable are of primary interest, and it is natural to evaluate the effects at different quantiles (e.g., median). The large number of potential effect modifiers, the unknown structure of the treatment effects, and the presence of right censoring pose significant challenges. In this paper, we develop a hybrid forest approach called Hybrid Censored Quantile Regression Forest (HCQRF) to assess the heterogeneous effects varying with high-dimensional variables. The hybrid estimation approach takes advantage of the random …
We consider a regression modeling of the quantiles of residual life, remaining lifetime at a specific time. We propose a smoothed induced version of the existing non-smooth estimating equations approaches for estimating regression parameters. The proposed estimating equations are smooth in regression parameters, so solutions can be readily obtained via standard numerical algorithms. Moreover, the smoothness in the proposed estimating equations enables one to obtain a robust sandwich-type covariance estimator of regression estimators aided by an efficient resampling method. To handle data subje…
In this paper, nonparametric estimation of the conditional Weibull-tail coefficient when the variable of interest is right random censored is addressed. A Weissman-type estimator of conditional extreme quantile is also proposed. In addition, a simulation study is conducted to assess the finite-sample behavior of the proposed estimators and a comparison with alternative strategies is provided. Finally, the practical applicability of the methodology is presented using a real datasets of men suffering from a larynx cancer.
Quantile regression has been advocated in survival analysis to assess evolving covariate effects. However, challenges arise when the censoring time is not always observed and may be covariate-dependent, particularly in the presence of continuously-distributed covariates. In spite of several recent advances, existing methods either involve algorithmic complications or impose a probability grid. The former leads to difficulties in the implementation and asymptotics, whereas the latter introduces undesirable grid dependence. To resolve these issues, we develop fundamental and general quantile cal…
This paper proposes a new test for the comparison of conditional quantile curves when the outcome of interest, typically a duration, is subject to right censoring. The test can be applied both in the case of two independent samples and for paired data, and can be used for the comparison of quantiles at a fixed quantile level, a finite set of levels or a range of quantile levels. The asymptotic distribution of the proposed test statistics is obtained both under the null hypothesis and under local alternatives. We describe a bootstrap procedure in order to approximate the critical values, and pr…
We investigate the estimation of the extreme value index when the data are subject to random censorship. We prove, in a unified way, detailed asymptotic normality results for various estimators of the extreme value index and use these estimators as the main building block for estimators of extreme quantiles. We illustrate the quality of these methods by a small simulation study and apply the estimators to medical data.
Since survival data occur over time, often important covariates that we wish to consider also change over time. Such covariates are referred as time-dependent covariates. Quantile regression offers flexible modeling of survival data by allowing the covariates to vary with quantiles. This paper provides a novel quantile regression model accommodating time-dependent covariates, for analyzing survival data subject to right censoring. Our simple estimation technique assumes the existence of instrumental variables. In addition, we present a doubly-robust estimator in the sense of Robins and Rotnitz…
We propose a censored quantile regression estimator motivated by unbiased estimating equations. Under the usual conditional independence assumption of the survival time and the censoring time given the covariates, we show that the proposed estimator is consistent and asymptotically normal. We develop an efficient computational algorithm which uses existing quantile regression code. As a result, bootstrap-type inference can be efficiently implemented. We illustrate the finite-sample performance of the proposed method by simulation studies and analysis of a survival data set.
Extreme value analysis in the presence of censoring is receiving much attention as it has applications in many disciplines, including survival and reliability studies. Estimation of extreme value index (EVI) is of primary importance as it is a critical parameter needed in estimating extreme events such as quantiles and exceedance probabilities. In this paper, we review several estimators of the extreme value index when data is subject to random censoring. In addition, four estimators are proposed, one based on the exponential regression approximation of log spacings, one based on a Zipf estima…
When facing multivariate covariates, general semiparametric regression techniques come at hand to propose flexible models that are unexposed to the curse of dimensionality. In this work a semiparametric copula-based estimator for conditional quantiles is investigated for complete or right-censored data. In spirit, the methodology is extending the recent work of Noh et al. (2013) and Noh et al. (2015), as the main idea consists in appropriately defining the quantile regression in terms of a multivariate copula and marginal distributions. Prior estimation of the latter and simple plug-in lead to…
We develop inference procedures for longitudinal data where some of the measurements are censored by fixed constants. We consider a semi-parametric quantile regression model that makes no distributional assumptions. Our research is motivated by the lack of proper inference procedures for data from biomedical studies where measurements are censored due to a fixed quantification limit. In such studies the focus is often on testing hypotheses about treatment equality. To this end, we propose a rank score test for large sample inference on a subset of the covariates. We demonstrate the importance …
One-shot devices analysis involves an extreme case of interval censoring, wherein one can only know whether the failure time is either before or after the test time. Some kind of one-shot devices do not get destroyed when tested, and so can continue within the experiment, providing extra information for inference, if they did not fail before an inspection time. In addition, their reliability can be rapidly estimated via accelerated life tests (ALTs) by running the tests at varying and higher stress levels than working conditions. In particular, step-stress tests allow the experimenter to incre…
This paper addresses the problem of estimating, in the presence of random censoring as well as competing risks, the extreme value index of the (sub)-distribution function associated to one particular cause, in the heavy-tail case. Asymptotic normality of the proposed estimator (which has the form of an Aalen-Johansen integral, and is the first estimator proposed in this context) is established. A small simulation study exhibits its performances for finite samples. Estimation of extreme quantiles of the cumulative incidence function is also addressed.
We develop a unified approach for classification and regression support vector machines for data subject to right censoring. We provide finite sample bounds on the generalization error of the algorithm, prove risk consistency for a wide class of probability measures, and study the associated learning rates. We apply the general methodology to estimation of the (truncated) mean, median, quantiles, and for classification problems. We present a simulation study that demonstrates the performance of the proposed approach.
