The Kaplan--Meier (KM) estimator, which provides a nonparametric estimate of a survival function for time-to-event data, has wide application in clinical studies, engineering, economics and other fields. The theoretical properties of the KM estimator including its consistency and asymptotic distribution have been extensively studied. We reconstruct the KM estimator as an M-estimator by maximizing a quadratic M-function based on concordance, which can be computed using the expectation--maximization (EM) algorithm. It is shown that the convergent point of the EM algorithm coincides with the traditional KM estimator, offering a new interpretation of the KM estimator as an M-estimator. Theoretical properties including the large-sample variance and limiting distribution of the KM estimator are established using M-estimation theory. Simulations and application on two real datasets demonstrate that the proposed M-estimator is exactly equivalent to the KM estimator, while the confidence interval and band can be derived as well.
In single-arm clinical trials with survival outcomes, the Kaplan-Meier estimator and its confidence interval are widely used to assess survival probability and median survival time. Since the asymptotic normality of the Kaplan-Meier estimator is a common result, the sample size calculation methods have not been studied in depth. An existing sample size calculation method is founded on the asymptotic normality of the Kaplan-Meier estimator using the log transformation. However, the small sample properties of the log transformed estimator are quite poor in small sample sizes (which are typical s…
In this paper, we study Kaplan-Meier V- and U-statistics respectively defined as $θ(\widehat{F}_n)=\sum_{i,j}K(X_{[i:n]},X_{[j:n]})W_iW_j$ and $θ_U(\widehat{F}_n)=\sum_{i\neq j}K(X_{[i:n]},X_{[j:n]})W_iW_j/\sum_{i\neq j}W_iW_j$, where $\widehat{F}_n$ is the Kaplan-Meier estimator, $\{W_1,\ldots,W_n\}$ are the Kaplan-Meier weights and $K:(0,\infty)^2\to\mathbb R$ is a symmetric kernel. As in the canonical setting of uncensored data, we differentiate between two asymptotic behaviours for $θ(\widehat{F}_n)$ and $θ_U(\widehat{F}_n)$. Additionally, we derive an asymptotic canonical V-statistic repr…
We derive process limit distribution results for the Nelson-Aalen estimator of a hasard function and for the Kaplan-Meier estimator of a distribution function, under different dependence assumptions. The data are assumed to be right censored observations of a stationary time series. We treat weakly dependent as well as long range dependent data, and allow for qualitative differences in the dependence for the censoring times versus the time of interest
This article is concerned with proving the consistency of Efron's (1981) bootstrap for the Kaplan-Meier estimator on the whole support of a survival function. While other works address the asymptotic Gaussianity of the estimator itself without restricting time (e.g. Gill, 1983, and Ying, 1989), we enable the construction of bootstrap-based time-simultaneous confidence bands for the whole survival function. Other practical applications include bootstrap-based confidence bands for the mean residual life-time function or the Lorenz curve as well as confidence intervals for the Gini index.
Comparing Two Survival Functions at a Fixed Timepoint
Biostatistics Research Branch, National Institute of Allergy and Infectious Diseases（过敏和传染病研究所生物统计研究分支）；Frederick National Laboratory for Cancer Research（弗雷德癌症国家实验室）；Axle Informatics（Axle信息学）
For studying or reducing the bias of functionals of the Kaplan-Meier survival estimator, the jackknifing approach of Stute and Wang (1994) is natural. We have studied the behavior of the jackknife estimate of bias under different configurations of the censoring level, sample size, and the censoring and survival time distributions. The empirical research reveals some new findings about robust calculation of the bias, particularly for higher censoring levels. We have extended their jackknifing approach to cover the case where the largest observation is censored, using the imputation methods for …
The setting of a right-censored random sample subject to contamination is considered. In various fields, expert information is often available and used to overcome the contamination. This paper integrates expert knowledge into the product-limit estimator in two different ways with distinct interpretations. Strong uniform consistency is proved for both cases under certain assumptions on the kind of contamination and the quality of expert information, which sheds light on the techniques and decisions that practitioners may take. The nuances of the techniques are discussed -- also with a view tow…
In this paper we test the composite hypothesis that lifetimes follow an exponential distribution based on observed randomly right censored data. Testing this hypothesis is complicated by the presence of this censoring, due to the fact that not all lifetimes are observed. To account for this complication, we propose modifications to tests based on the empirical characteristic function and Laplace transform. In the full sample case these empirical functions can be expressed as integrals with respect to the empirical distribution function of the lifetimes. We propose replacing this estimate of th…
Population attributable fractions aim to quantify the proportion of the cases of an outcome (for example, a disease) that would have been avoided had no individuals in the population been exposed to a given exposure. This quantity thus plays a crucial role in epidemiology and public health, notably to guide policies, interventions or to assess the burden of a disease due to a particular exposure. Various statistical methods have been proposed to estimate attributable fractions using observational data. When time-to-event data are used, several of these formulas yield invalid results. Alternati…
The Kaplan-Meier product-limit estimator is a simple and powerful tool in time to event analysis. An extension exists for populations stratified into cohorts where a population survival curve is generated by weighted averaging of cohort-level survival curves. For making population-level comparisons using this statistic, we analyze the statistics of the area between two such weighted survival curves. We derive the large sample behavior of this statistic based on an empirical process of product-limit estimators. This estimator was used by an interdisciplinary NIH-SSA team in the identification o…
Non-parametric maximum likelihood estimation encompasses a group of classic methods to estimate distribution-associated functions from potentially censored and truncated data, with extensive applications in survival analysis. These methods, including the Kaplan-Meier estimator and Turnbull's method, often result in overfitting, especially when the sample size is small. We propose an improvement to these methods by applying kernel smoothing to their raw estimates, based on a BIC-type loss function that balances the trade-off between optimizing model fit and controlling model complexity. In the …
Time to event outcomes are often evaluated on the hazard scale, but interpreting hazards may be difficult. Recently, there has been concern in the causal inference literature that hazards actually have a built in selection-effect that prevents simple causal interpretations. This is even a problem in randomized controlled trials, where hazard ratios have become a standard measure of treatment effects. Modeling on the hazard scale is nevertheless convenient, e.g. to adjust for covariates. Using hazards for intermediate calculations may therefore be desirable. Here, we provide a generic method fo…
In survey analysis, the estimation of the cumulative distribution function (cdf) is of great interest: it allows for instance to derive quantiles estimators or other non linear parameters derived from the cdf. We consider the case where the response variable is a right censored duration variable. In this framework, the classical estimator of the cdf is the Kaplan-Meier estimator. As an alternative, we propose a nonparametric model-based estimator of the cdf in a finite population. The new estimator uses auxiliary information brought by a continuous covariate and is based on nonparametric media…
Under a single-index regression assumption, we introduce a new semiparametric procedure to estimate a conditional density of a censored response. The regression model can be seen as a generalization of Cox regression model and also as a profitable tool to perform dimension reduction under censoring. This technique extends the results of Delecroix et al. (2003). We derive consistency and asymptotic normality of our estimator of the index parameter by proving its asymptotic equivalence with the (uncomputable) maximum likelihood estimator, using martingales results for counting processes and argu…
The delta method is a popular and elementary tool for deriving limiting distributions of transformed statistics, while applications of asymptotic distributions do not allow one to obtain desirable accuracy of approximation for tail probabilities. The large and moderate deviation theory can achieve this goal. Motivated by the delta method in weak convergence, a general delta method in large deviations is proposed. The new method can be widely applied to driving the moderate deviations of estimators and is illustrated by examples including the Wilcoxon statistic, the Kaplan--Meier estimator, the…
We propose a deep generative approach to nonparametric estimation of conditional survival and hazard functions with right-censored data. The key idea of the proposed method is to first learn a conditional generator for the joint conditional distribution of the observed time and censoring indicator given the covariates, and then construct the Kaplan-Meier and Nelson-Aalen estimators based on this conditional generator for the conditional hazard and survival functions. Our method combines ideas from the recently developed deep generative learning and classical nonparametric estimation in surviva…
Kaplan-Meier survival analysis represents the most objective measure of treatment efficacy in oncology, though subjected to potential bias, which is worrisome in an era of precision medicine. Independent of the bias inherent to the design and execution of clinical trials, bias may be the result of patient censoring, or incomplete observation. Unlike disease/progression free survival, overall survival is based on a well defined time point and thus avoids interval censoring, but right-censoring, due to incomplete follow-up, may still be a source of bias. We study three mechanisms of right-censor…
A reduced-bias nonparametric estimator of the cumulative distribution function (CDF) and the survival function is proposed using infinite-order kernels. Fourier transform theory on generalized functions is utilized to obtain the improved bias estimates. The new estimators are analyzed in terms of their relative deficiency to the empirical distribution function and Kaplan-Meier estimator, and even improvements in terms of asymptotic relative efficiency (ARE) are present under specified assumptions on the data. The deficiency analysis introduces a deficiency rate which provides a continuum betwe…
Right censored survival data collected on a cohort of prevalent cases with constant incidence are length-biased, and may be used to estimate the length-biased (i.e., prevalent-case) survival function. When the incidence rate is constant, so-called stationarity of the incidence, it is more efficient to use this structure for unconditional statistical inference than to carry out an analysis by conditioning on the observed truncation times. It is well known that, due to the informative censoring for prevalent cohort data, the Kaplan--Meier estimator is not the unconditional NPMLE of the length-bi…
Inferential challenges that arise when data are censored have been extensively studied under the classical frameworks. In this paper, we provide an alternative generalized inferential model approach whose output is a data-dependent plausibility function. This construction is driven by an association between the distribution of the relative likelihood function at the interest parameter and an unobserved auxiliary variable. The plausibility function emerges from the distribution of a suitably calibrated random set designed to predict that unobserved auxiliary variable. The evaluation of this pla…
Survival function estimation is used in many disciplines, but it is most common in medical analytics in the form of the Kaplan-Meier estimator. Sensitive data (patient records) is used in the estimation without any explicit control on the information leakage, which is a significant privacy concern. We propose a first differentially private estimator of the survival function and show that it can be easily extended to provide differentially private confidence intervals and test statistics without spending any extra privacy budget. We further provide extensions for differentially private estimati…
Let F be an unknown univariate distribution function to be estimated from a sample containing censored observations and tau be in dom(F). The author has derived a novel nonparametric estimator F_hat for F without making any assumptions regarding the nature of the censoring mechanism or the distribution function F. The distribution of F_hat(tau) can be easily and accurately estimated even for small sample sizes. The estimator F_hat has significantly outperformed the Kaplan Meier estimator in a simulation study with an exponential and a lognormal distribution functions F and a censoring mechanis…
