In this paper, we consider survival analysis with right-censored data which is a common situation in predictive maintenance and health field. We propose a model based on the estimation of two-parameter Weibull distribution conditionally to the features. To achieve this result, we describe a neural network architecture and the associated loss functions that takes into account the right-censored data. We extend the approach to a finite mixture of two-parameter Weibull distributions. We first validate that our model is able to precisely estimate the right parameters of the conditional Weibull distribution on synthetic datasets. In numerical experiments on two real-word datasets (METABRIC and SEER), our model outperforms the state-of-the-art methods. We also demonstrate that our approach can consider any survival time horizon.
The Weibull distribution, with shape parameter $k>0$ and scale parameter $λ>0$, is one of the most popular parametric distributions in survival analysis with complete or censored data. Although inference of the parameters of the Weibull distribution is commonly done through maximum likelihood, it is well established that the maximum likelihood estimate of the shape parameter is inadequate due to the associated large bias when the sample size is small or the proportion of censored data is large. This manuscript demonstrates how the Bayesian information-theoretic minimum message length principle…
In this paper we investigate the estimation of the unknown parameters of a competing risk model based on a Weibull distributed decreasing failure rate and an exponentially distributed constant failure rate, under right censored data.likelihood estimators.
The Weibull distribution is one of the most used tools in reliability analysis. In this paper, assuming a Bayesian approach, we propose necessary and sufficient conditions to verify when improper priors lead to proper posteriors for the parameters of the Weibull distribution in the presence of complete or right-censored data. Additionally, we proposed sufficient conditions to verify if the obtained posterior moments are finite. These results can be achieved by checking the behavior of the improper priors, which are applied in different objective priors to illustrate the usefulness of the new r…
The analysis of progressively censored data has received considerable attention in the last few years. In this paper we consider the joint progressive censoring scheme for two populations. It is assumed that the lifetime distribution of the items from the two populations follow Weibull distribution with the same shape but different scale parameters. Based on the joint progressive censoring scheme first we consider the maximum likelihood estimators of the unknown parameters whenever they exist. We provide the Bayesian inferences of the unknown parameters under a fairly general priors on the sha…
This paper addresses the problem of estimating the extreme value index in presence of random censoring for distributions in the Weibull domain of attraction. The methodologies introduced in [Worms (2014)], in the heavy-tailed case, are adapted here to the negative extreme value index framework, leading to the definition of weighted versions of the popular moments of relative excesses with arbitrary exponent. This leads to the definition of two families of estimators (with an adaptation of the so called Moment estimator as a particular case), for which the consistency is proved under a first or…
We develop flexible multi-parameter regression survival models for interval censored survival data arising in longitudinal prospective studies and longitudinal randomised controlled clinical trials. A multi-parameter Weibull regression survival model, which is wholly parametric, and has non-proportional hazards, is the main focus of the paper. We describe the basic model, develop the interval-censored likelihood and extend the model to include gamma frailty and a dispersion model. We evaluate the models by means of a simulation study and a detailed re-analysis of data from the Signal Tandmobie…
We introduce a kernel-based goodness-of-fit test for censored data, where observations may be missing in random time intervals: a common occurrence in clinical trials and industrial life-testing. The test statistic is straightforward to compute, as is the test threshold, and we establish consistency under the null. Unlike earlier approaches such as the Log-rank test, we make no assumptions as to how the data distribution might differ from the null, and our test has power against a very rich class of alternatives. In experiments, our test outperforms competing approaches for periodic and Weibul…
Fertility plans, measured by the number of planned children, have been found to be affected by education and family background via complex tail dependencies. This challenge was previously met with the use of non-parametric jittering approaches. This paper shows how a novel generalized additive model based on a discrete Weibull distribution provides partial effects of the covariates on fertility plans which are comparable to jittering, without the inherent drawback of crossing conditional quantiles. The model has some additional desirable features: both over- and under-dispersed data can be mod…
An optimal dynamic treatment regime (DTR) consists of a sequence of decision rules in maximizing long-term benefits, which is applicable for chronic diseases such as HIV infection or cancer. In this paper, we develop a novel angle-based approach to search the optimal DTR under a multicategory treatment framework for survival data. The proposed method targets maximization the conditional survival function of patients following a DTR. In contrast to most existing approaches which are designed to maximize the expected survival time under a binary treatment framework, the proposed method solves th…
Self-Consistent Equation-guided Neural Networks for Censored Time-to-Event Data
Harvard Pilgrim Health Care Institute（哈佛朝圣者医疗保健研究所）；Harvard Medical School（哈佛医学院）；Harvard School of Public Health（哈佛公共卫生学院）；North Carolina State University（北卡罗来纳州立大学）
In this paper we study bayesian analysis of Modified Weibull distribution under progressively censored competing risk model. This study is made for progressively censored data. We use deterministic scan Gibbs sampling combined with slice sampling to generate from the posterior distribution. Posterior distribution is formed by taking prior distribution as reference prior. A real life data analysis is shown for illustrative purpose.
In this paper, we have investigated the reliability of a K-out-of-N system for the components following Weibull distribution based on the generalized progressive hybrid censored data. We have obtained the maximum likelihood estimates (MLEs) of the unknown parameters and the reliability function of the system. Using asymptotic normality property of MLEs, the corresponding asymptotic confidence intervals are constructed. Furthermore, Bayes estimates are derived under squared error loss function with informative prior by using Markov Chain Monte Carlo (MCMC) technique. Highest posterior density (…
We propose a novel personalized concept for the optimal treatment selection for a situation where the response is a multivariate vector, that could contain right-censored variables such as survival time. The proposed method can be applied with any number of treatments and outcome variables, under a broad set of models. Following a working semiparametric Single Index Model that relates covariates and responses, we first define a patient-specific composite score, constructed from individual covariates. We then estimate conditional means of each response, given the patient score, correspond to ea…
In this paper, we establish uniform-in-bandwidth limit laws of the logarithm for nonparametric Inverse Probability of Censoring Weighted (I.P.C.W.) estimators of the multivariate regression function under random censorship. A similar result is deduced for estimators of the conditional distribution function. The uniform-in-bandwidth consistency for estimators of the conditional density and the conditional hazard rate functions are also derived from our main result. Moreover, the logarithm laws we establish are shown to yield almost sure simultaneous asymptotic confidence bands for the functions…
Estimation of reliability and hazard rate is one of the most important problems raised in many applications especially in engineering studies as well as human lifetime. In this regard, different methods of estimation have been used. Each method exploits various tools and suffers from problems such as complexity of computations, low precision, and so forth. This study is employed the E-Bayesian method, for estimating the parameter and survival functions of the Weibull Generalized Exponential distribution. The estimators are obtained under squared error and LINEX loss functions under progressive…
Interval-censored data analysis is important in biomedical statistics for any type of time-to-event response where the time of response is not known exactly, but rather only known to occur between two assessment times. Many clinical trials and longitudinal studies generate interval-censored data; one common example occurs in medical studies that entail periodic follow-up. In this paper we propose a survival forest method for interval-censored data based on the conditional inference framework. We describe how this framework can be adapted to the situation of interval-censored data. We show that…
Nonparametric and semiparametric methods are commonly used in survival analysis to mitigate the bias due to model misspecification. However, such methods often cannot estimate upper-tail survival quantiles when a sizable proportion of the data are censored, in which case parametric likelihood-based estimators present a viable alternative. In this article, we extend a popular family of parametric survival models which make the Accelerated Failure Time (AFT) assumption to account for heteroscedasticity in the survival times. The conditional variances can depend on arbitrary covariates, thus addi…
In survival studies, classical inferences for left-truncated data require quasi-independence, a property that the joint density of truncation time and failure time is factorizable into their marginal densities in the observable region. The quasi-independence hypothesis is testable; many authors have developed tests for left-truncated data with or without right-censoring. In this paper, we propose a class of test statistics for testing the quasi-independence which unifies the existing methods and generates new useful statistics such as conditional Spearman's rank correlation coefficient. Asympt…
Individualized treatment rules can lead to better health outcomes when patients have heterogeneous responses to treatment. Very few individualized treatment rule estimation methods are compatible with a multi-treatment observational study with right censored survival outcomes. In this paper we extend policy evaluation methods to the right censored data setting. Existing approaches either make restrictive assumptions about the structure of the data, or use inverse weighting methods that increase the variance of the estimator resulting in decreased performance. We propose a method which uses bal…
This paper focuses on quantifying and estimating the predictive accuracy of prognostic models for time-to-event outcomes with competing events. We consider the time-dependent discrimination and calibration metrics, including the receiver operating characteristics curve and the Brier score, in the context of competing risks. To address censoring, we propose a unified nonparametric estimation framework for both discrimination and calibration measures, by weighting the censored subjects with the conditional probability of the event of interest given the observed data. We demonstrate through simul…
This paper presents a hierarchical Bayesian approach to the estimation of components' reliability (survival) using a Weibull model for each of them. The proposed method can be used to estimation with general survival censored data, because the estimation of a component's reliability in a series (parallel) system is equivalent to the estimation of its survival function with right- (left-) censored data. Besides the Weibull parametric model for reliability data, independent gamma distributions are considered at the first hierarchical level for the Weibull parameters and independent uniform distr…
Estimating individualized treatment rules is a central task for personalized medicine. [zhao2012estimating] and [zhang2012robust] proposed outcome weighted learning to estimate individualized treatment rules directly through maximizing the expected outcome without modeling the response directly. In this paper, we extend the outcome weighted learning to right censored survival data without requiring either an inverse probability of censoring weighting or a semiparametric modeling of the censoring and failure times as done in [zhao2015doubly]. To accomplish this, we take advantage of the tree ba…
This paper studies prediction summary measures for a prediction function under a general setting in which the model is allowed to be misspecified and the prediction function is not required to be the conditional mean response. We show that the R2 measure based on a variance decomposition is insufficient to summarize the predictive power of a nonlinear prediction function. By deriving a prediction error decompo- sition, we introduce an additional measure, L2, to augment the R2 measure. When used together, the two measures provide a complete summary of the predictive power of a prediction functi…
Survival analysis of right censored data arises often in many areas of research including medical research. Effect of covariates (and their interactions) on survival distribution can be studied through existing methods which requires to pre-specify the functional form of the covariates including their interactions. Survival trees offer relatively flexible approach when the form of covariates' effects is unknown. Most of the currently available survival tree construction techniques are not based on a formal test of significance; however, recently proposed ctree algorithm (Hothorn et al., 2006) …
Survival analysis, or time-to-event modelling, is a classical statistical problem that has garnered a lot of interest for its practical use in epidemiology, demographics or actuarial sciences. Recent advances on the subject from the point of view of machine learning have been concerned with precise per-individual predictions instead of population studies, driven by the rise of individualized medicine. We introduce here a conditional normalizing flow based estimate of the time-to-event density as a way to model highly flexible and individualized conditional survival distributions. We use a nove…
A bivariate cure copula model with zero-inflated gamma frailty for right-censored data
Graduate Institute for Advanced Studies, SOKENDAI（SOKENDAI 高级研究研究生院）；Institute of Statistical Mathematics（统计数学研究所）；School of Informatics and Data Science, Hiroshima University（广岛大学信息科学与数据科学学院）
Most work in neural networks focuses on estimating the conditional mean of a continuous response variable given a set of covariates.In this article, we consider estimating the conditional distribution function using neural networks for both censored and uncensored data. The algorithm is built upon the data structure particularly constructed for the Cox regression with time-dependent covariates. Without imposing any model assumption, we consider a loss function that is based on the full likelihood where the conditional hazard function is the only unknown nonparametric parameter, for which uncon…
We develop a maximum likelihood estimating approach for time-to-event Weibull regression models with outcome-dependent sampling, where sampling of subjects is dependent on the residual fraction of the time left to developing the event of interest. Additionally, we propose a two-stage approach which proceeds by iteratively estimating, through a pseudo score, the Weibull parameters of interest (i.e., the regression parameters) conditional on the inverse probability of sampling weights; and then re-estimating these weights (given the updated Weibull parameter estimates) through the profiled full …
In this paper, nonparametric estimation of the conditional Weibull-tail coefficient when the variable of interest is right random censored is addressed. A Weissman-type estimator of conditional extreme quantile is also proposed. In addition, a simulation study is conducted to assess the finite-sample behavior of the proposed estimators and a comparison with alternative strategies is provided. Finally, the practical applicability of the methodology is presented using a real datasets of men suffering from a larynx cancer.
Through Alzheimer's Disease Neuroimaging Initiative (ADNI), time-to-event data: from the pre-dementia state of mild cognitive impairment (MCI) to the diagnosis of Alzheimer's disease (AD), is collected and analyzed by explicitly unraveling prognostic heterogeneity among 346 uncensored and 557 right censored subjects under structural dependency among covariate features. The non-informative censoring mechanism is tested and confirmed based on conditional-vs-marginal entropies evaluated upon contingency tables built by the Redistribute-to-the-right algorithm. The Categorical Exploratory Data Anal…
