We propose a new approach for estimating the parameters of a probability distribution. It consists on combining two new methods of estimation. The first is based on the definition of a new distance measuring the difference between variations of two distributions on a finite number of points from their support and on using this measure for estimation purposes by the method of minimum distance. For the second method, given an empirical discrete distribution, we build up an auxiliary discrete theoretical distribution having the same support of the first and depending on the same parameters of the parent distribution of the data from which the empirical distribution emanated. We estimate then the parameters from the empirical distribution by the usual statistical methods. In practice, we propose to compute the two estimations, the second based on maximum likelihood principle of known theoretical properties, and the first being as a control of the effectiveness of the obtained estimation, and for which we prove the convergence in probability, so we have also a criterion on the quality of the information contained in the observations. We apply the approach to truncated or grouped and censored data situations to give the flavour on the effectiveness of the approach. We give also some interesting perspectives of the approach including model selection from truncated data, estimation of the initial trial value in the celebrate EM algorithm in the case of truncation and merged normal populations, a test of goodness of fit based on the new distance, quality of estimations and data.
The analysis left truncated and right censored data is very common in survival and reliability analysis. In lifetime studies patients often subject to left truncation in addition to right censoring. For example, in bone marrow transplant studies based on International Bone Marrow Transplant Registry (IBMTR), the patients who die while waiting for the transplants will not be reported to the IBMTR. In this paper, we develop novel U-statistics under left truncation and right censoring. We prove the $\sqrt{n}$-consistency of the proposed U-statistics. We derive the asymptotic distribution of the U…
Tree methods (recursive partitioning) are a popular class of nonparametric methods for analyzing data. One extension of the basic tree methodology is the survival tree, which applies recursive partitioning to censored survival data. There are several existing survival tree methods in the literature, which are mainly designed for right-censored data. We propose two new survival trees for left-truncated and right-censored (LTRC) data, which can be seen as a generalization of the traditional survival tree for right-censored data. Further, we show that such trees can be used to analyze survival da…
In survival studies, classical inferences for left-truncated data require quasi-independence, a property that the joint density of truncation time and failure time is factorizable into their marginal densities in the observable region. The quasi-independence hypothesis is testable; many authors have developed tests for left-truncated data with or without right-censoring. In this paper, we propose a class of test statistics for testing the quasi-independence which unifies the existing methods and generates new useful statistics such as conditional Spearman's rank correlation coefficient. Asympt…
There is a surge in medical follow-up studies that include longitudinal covariates in the modeling of survival data. So far, the focus has been largely on right-censored survival data. We consider survival data that are subject to both left truncation and right censoring. Left truncation is well known to produce biased sample. The sampling bias issue has been resolved in the literature for the case which involves baseline or time-varying covariates that are observable. The problem remains open, however, for the important case where longitudinal covariates are present in survival models. A join…
This report is designed to clarify a few points about the article "Semiparametric modeling of grouped current duration data with preferential reporting" by McLain, Sundaram, Thoma and Louis in Statistics in Medicine (McLain et al., 2014, hereafter MSTL) regarding using the methods under right censoring. In simulation studies, it has been found that bias can occur when right censoring is present. Current duration data normally does not have censored values, but censoring can be induced at a value, say tau, after which the data values are thought to be unreliable. As noted in MSTL, some right ce…
We develop a unified approach for classification and regression support vector machines for data subject to right censoring. We provide finite sample bounds on the generalization error of the algorithm, prove risk consistency for a wide class of probability measures, and study the associated learning rates. We apply the general methodology to estimation of the (truncated) mean, median, quantiles, and for classification problems. We present a simulation study that demonstrates the performance of the proposed approach.
When data are collected subject to a detection limit, observations below the detection limit may be considered censored. In addition, the domain of such observations may be restricted; for example, values may be required to be non-negative. We propose a regression method for censored observations that also accounts for domain restriction. The method finds maximum likelihood estimates assuming an underlying truncated normal distribution. We show that our method, tcensReg, outperforms other methods commonly used for data with detection limits such as Tobit regression and single imputation of the…
A number of models for generating statistical data in various fields of insurance, including life insurance, pensions, and general insurance have been considered. It is shown that the insurance statistics data, as a rule, are truncated and censored, and often multivariate. We propose a non-parametric estimation of the distribution function for multivariate truncated-censored data in the form of a quasi-empirical distribution and a simple iterative algorithm for its construction. To check the accuracy of the proposed evaluation of the distribution function for truncated-censored data, simulatio…
This work was motivated by observational studies in pregnancy with spontaneous abortion (SAB) as outcome. Clearly some women experience the SAB event but the rest do not. In addition, the data are left truncated due to the way pregnant women are recruited into these studies. For those women who do experience SAB, their exact event times are sometimes unknown. Finally, a small percentage of the women are lost to follow-up during their pregnancy. All these give rise to data that are left truncated, partly interval and right-censored, and with a clearly defined cured portion. We consider the non-…
Spatial survival analysis has received a great deal of attention over the last 20 years due to the important role that geographical information can play in predicting survival. This paper provides an introduction to a set of programs for implementing some Bayesian spatial survival models in R using the package spBayesSurv. The function survregbayes includes the three most commonly-used semiparametric models: proportional hazards, proportional odds, and accelerated failure time. All manner of censored survival times are simultaneously accommodated including uncensored, interval censored, curren…
This paper describes prediction methods for the number of future events from a population of units associated with an on-going time-to-event process. Examples include the prediction of warranty returns and the prediction of the number of future product failures that could cause serious threats to property or life. Important decisions such as whether a product recall should be mandated are often based on such predictions. Data, generally right-censored (and sometimes left truncated and right-censored), are used to estimate the parameters of a time-to-event distribution. This distribution can th…
In this article, the analysis of left truncated and right censored competing risks data is carried out, under the assumption of the latent failure times model. It is assumed that there are two competing causes of failures, although most of the results can be extended for more than two causes of failures. The lifetimes corresponding to the competing causes of failures are assumed to follow Weibull distributions with the same shape parameter but different scale parameters. The maximum likelihood estimation procedure of the model parameters is discussed, and confidence intervals are provided usin…
Tree-based methods for length-biased survival data
National Cancer Center（韩国国立癌症中心）；Yonsei University（延世大学）；Seoul National University College of Medicine（首尔大学医学院）；Seoul Metropolitan Government-Seoul National University Boramae Medical Center（首尔市政府-首尔大学八良医疗中心）；Ewha Womans University（梨花女子大学）
Nonparametric Estimation of Event-Free Survival for Data with Left-Truncated Death and Intermittently Assessed Nonfatal Events
University of Minnesota（明尼苏达大学）；Columbia University（哥伦比亚大学）；University of Texas（德克萨斯大学）；University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）；University of Mississippi Medical Center（密西西比大学医学中心）；University of North Carolina（北卡罗来纳大学）；Wake Forest University School of Medicine（维克森林大学医学院）
A comprehensive, unified approach to modeling arbitrarily censored spatial survival data is presented for the three most commonly-used semiparametric models: proportional hazards, proportional odds, and accelerated failure time. Unlike many other approaches, all manner of censored survival times are simultaneously accommodated including uncensored, interval censored, current-status, left and right censored, and mixtures of these. Left-truncated data are also accommodated leading to models for time-dependent covariates. Both georeferenced (location exactly observed) and areally observed (locati…
Non-parametric maximum likelihood estimation encompasses a group of classic methods to estimate distribution-associated functions from potentially censored and truncated data, with extensive applications in survival analysis. These methods, including the Kaplan-Meier estimator and Turnbull's method, often result in overfitting, especially when the sample size is small. We propose an improvement to these methods by applying kernel smoothing to their raw estimates, based on a BIC-type loss function that balances the trade-off between optimizing model fit and controlling model complexity. In the …
In risk analysis, a global fit that appropriately captures the body and the tail of the distribution of losses is essential. Modelling the whole range of the losses using a standard distribution is usually very hard and often impossible due to the specific characteristics of the body and the tail of the loss distribution. A possible solution is to combine two distributions in a splicing model: a light-tailed distribution for the body which covers light and moderate losses, and a heavy-tailed distribution for the tail to capture large losses. We propose a splicing model with a mixed Erlang (ME)…
In many applied fields, such as genomics, different types of data are collected on the same system, and it is not uncommon that some of these datasets are subject to censoring as a result of the measurement technologies used, such as data generated by polymerase chain reactions and flow cytometer. When the overall objective is that of network inference, at possibly different levels of a system, information coming from different sources and/or different steps of the analysis can be integrated into one model with the use of conditional graphical models. In this paper, we develop a doubly penaliz…
This paper presents a score-based weighted likelihood estimator (SWLE) for robust estimations of generalized linear model (GLM) for insurance loss data. The SWLE exhibits a limited sensitivity to the outliers, theoretically justifying its robustness against model contaminations. Also, with the specially designed weight function to effectively diminish the contributions of extreme losses to the GLM parameter estimations, most statistical quantities can still be derived analytically, minimizing the computational burden for parameter calibrations. Apart from robust estimations, the SWLE can also …
Finite mixture models have been widely used to model and analyze data from a heterogeneous populations. Moreover, data of this kind can be missing or subject to some upper and/or lower detection limits because of the restriction of experimental apparatuses. Another complication arises when measures of each population depart significantly from normality, for instance, asymmetric behavior. For such data structures, we propose a robust model for censored and/or missing data based on finite mixtures of multivariate skew-normal distributions. This approach allows us to model data with great flexibi…
Best practices for estimating and reporting epidemiological delay distributions of infectious diseases using public health surveillance and healthcare data
Bimodal truncated count distributions are frequently observed in aggregate survey data and in user ratings when respondents are mixed in their opinion. They also arise in censored count data, where the highest category might create an additional mode. Modeling bimodal behavior in discrete data is useful for various purposes, from comparing shapes of different samples (or survey questions) to predicting future ratings by new raters. The Poisson distribution is the most common distribution for fitting count data and can be modified to achieve mixtures of truncated Poisson distributions. However,…
To obtain a probabilistic model for a dependent variable based on some set of explanatory variables, a distributional approach is often adopted where the parameters of the distribution are linked to regressors. In many classical models this only captures the location of the distribution but over the last decade there has been increasing interest in distributional regression approaches modeling all parameters including location, scale, and shape. Notably, so-called non-homogeneous Gaussian regression (NGR) models both mean and variance of a Gaussian response and is particularly popular in weath…
