Random double truncation refers a situation in which the variable of interest is observed only when it falls within two random limits. Such phenomenon occurs in many applications of Survival Analysis and Epidemiology, among many other fields. There exist several R packages to analyze doubly truncated data which implement, for instance, estimators for the cumulative distribution, the cumulative incidences of competing risks, or the regression coefficients in the Cox model. In this paper the main features of these packages are reviewed. The relative merits of the libraries are illustrated through the analysis of simulated and real data. This includes the study of the statistical accuracy of the implemented techniques as well as of their computational speed. Practical recommendations are given.
This article introduces the Python package gcimpute for missing data imputation. gcimpute can impute missing data with many different variable types, including continuous, binary, ordinal, count, and truncated values, by modeling data as samples from a Gaussian copula model. This semiparametric model learns the marginal distribution of each variable to match the empirical distribution, yet describes the interactions between variables with a joint Gaussian that enables fast inference, imputation with confidence intervals, and multiple imputation. The package also provides specialized extensions…
We present `latentcor`, an R package for correlation estimation from data with mixed variable types. Mixed variables types, including continuous, binary, ordinal, zero-inflated, or truncated data are routinely collected in many areas of science. Accurate estimation of correlations among such variables is often the first critical step in statistical analysis workflows. Pearson correlation as the default choice is not well suited for mixed data types as the underlying normality assumption is violated. The concept of semi-parametric latent Gaussian copula models, on the other hand, provides a uni…
Simulating from the multivariate truncated normal distribution (MTN) is required in various statistical applications yet remains challenging in high dimensions. Currently available algorithms and their implementations often fail when the number of parameters exceeds a few hundred. To provide a general computational tool to efficiently sample from high-dimensional MTNs, we introduce the hdtg package that implements two state-of-the-art simulation algorithms: harmonic Hamiltonian Monte Carlo (harmonic-HMC) and zigzag Hamiltonian Monte Carlo (Zigzag-HMC). Both algorithms exploit analytical soluti…
Doubly truncated data arise in many areas such as astronomy, econometrics, and medical studies. For the regression analysis with doubly truncated response variables, the existence of double truncation may bring bias for estimation as well as affect variable selection. We propose a simultaneous estimation and variable selection procedure for the doubly truncated regression, allowing a diverging number of regression parameters. To remove the bias introduced by the double truncation, a Mann-Whitney-type loss function is used. The adaptive LASSO penalty is then added into the loss function to achi…
Doubly truncated data are found in astronomy, econometrics and survival analysis literature. They arise when each observation is confined to an interval, i.e., only those which fall within their respective intervals are observed along with the intervals. Unlike the more widely studied one-sided truncation that can be handled effectively by the counting process-based approach, doubly truncated data are much more difficult to handle. In their analysis of an astronomical data set, Efron and Petrosian (1999) proposed some nonparametric methods, including a generalization of Kendall's tau test, for…
The NPMLE of a distribution function from doubly truncated data was introduced in the seminal paper of Efron and Petrosian. The consistency of the Efron-Petrosian estimator depends however on the assumption of independent truncation. In this work we introduce an extension of the Efron-Petrosian NPMLE when the variable of interest and the truncation variables may be dependent. The proposed estimator is constructed on the basis of a copula function which represents the dependence structure between the variable of interest and the truncation variables. Two different iterative algorithms to comput…
It is proved that the sum of n independent but non-identically distributed doubly truncated Normal distributions converges in distribution to a Normal distribution. It is also shown how the result can be applied in estimating a constrained mixed effects model.
This paper introduces the R package drpop to flexibly estimate total population size from incomplete lists. Total population estimation, also called capture-recapture, is an important problem in many biological and social sciences. A typical dataset consists of incomplete lists of individuals from the population of interest along with some covariate information. The goal is to estimate the number of unobserved individuals and equivalently, the total population size. drpop flexibly models heterogeneity using the covariate information, under the assumption that two lists are conditionally indepe…
The R package lcmm provides a series of functions to estimate statistical models based on linear mixed model theory. It includes the estimation of mixed models and latent class mixed models for Gaussian longitudinal outcomes (hlme), curvilinear and ordinal univariate longitudinal outcomes (lcmm) and curvilinear multivariate outcomes (multlcmm), as well as joint latent class mixed models (Jointlcmm) for a (Gaussian or curvilinear) longitudinal outcome and a time-to-event that can be possibly left-truncated right-censored and defined in a competing setting. Maximum likelihood esimators are obtai…
Time-smoothed inverse probability weighted estimation of effects of generalized time-varying treatment strategies on repeated outcomes truncated by death
The family of stable distributions received extensive applications in many fields of studies since it incorporates both the skewness and heavy tails. In this paper, we introduce a package written in the R language called alphastable. The alphastable performs a variety of tasks including: 1- generating random numbers from univariate, truncated, and multivariate stable distributions. 2- computing the probability density function of univariate and multivariate elliptically contoured stable distributions, 3- computing the distribution function of univariate stable distributions, 4- estimating the …
We develop an R package RMM to implement a Conditional Logit (CL) model using the Robust Demand Estimation (RDE) method introduced in Cho et al. (2020), a customer choice-based $\textbf{R}$evenue $\textbf{M}$anagement $\textbf{M}$odel. In business, it is important to understand customers' choice behavior and preferences when the product prices change over time and across various customers. However, it is difficult to estimate demand because of unobservable no-purchase customers (i.e., truncated demand issue). The CL model fitted using the RDE method, enables a more general utility model with f…
In clinical and epidemiological research doubly truncated data often appear. This is the case, for instance, when the data registry is formed by interval sampling. Double truncation generally induces a sampling bias on the target variable, so proper corrections of ordinary estimation and inference procedures must be used. Unfortunately, the nonparametric maximum likelihood estimator of a doubly truncated distribution has several drawbacks, like potential non-existence and non-uniqueness issues, or large estimation variance. Interestingly, no correction for double truncation is needed when the …
Finite mixture models have been widely used to model and analyze data from a heterogeneous populations. Moreover, data of this kind can be missing or subject to some upper and/or lower detection limits because of the restriction of experimental apparatuses. Another complication arises when measures of each population depart significantly from normality, for instance, asymmetric behavior. For such data structures, we propose a robust model for censored and/or missing data based on finite mixtures of multivariate skew-normal distributions. This approach allows us to model data with great flexibi…
Compositional data, which is data consisting of fractions or probabilities, is common in many fields including ecology, economics, physical science and political science. If these data would otherwise be normally distributed, their spread can be conveniently represented by a multivariate normal distribution truncated to the non-negative space under a unit simplex. Here this distribution is called the simplex-truncated multivariate normal distribution. For calculations on truncated distributions, it is often useful to obtain rapid estimates of their integral, mean and covariance; these quantiti…
Models such as the zero-inflated and zero-altered Poisson and zero-truncated binomial are well-established in modern regression analysis. We propose a super model that jointly and maximally unifies alteration, inflation, truncation and deflation for counts, given a 1- or 2-parameter parent (base) distribution. Seven disjoint sets of special value types are accommodated because all but truncation have parametric and nonparametric variants. Some highlights include: (i) the mixture distribution is exceeding flexible, e.g., up to seven modes; (ii) under-, equi- and over-dispersion can be handled u…
Linear Mixed Effects (LME) models have been widely applied in clustered data analysis in many areas including marketing research, clinical trials, and biomedical studies. Inference can be conducted using maximum likelihood approach if assuming Normal distributions on the random effects. However, in many applications of economy, business and medicine, it is often essential to impose constraints on the regression parameters after taking their real-world interpretations into account. Therefore, in this paper we extend the classical (unconstrained) LME models to allow for sign constraints on its o…
Patient reported outcomes including quality of life (QoL) assessments are increasingly being included as either primary or secondary outcomes in randomized controlled trials. While making the outcomes more relevant for patients it entails a challenge in cases where death or a similar event makes the outcome of interest undefined. A pragmatic - and much used - solution is to assign diseased patient with the lowest possible QoL score. This makes medical sense, but creates a statistical problem since traditional tests such as t-tests or Wilcox tests potentially looses large amounts of statistical…
Recent advances in computational methods for intractable models have made network data increasingly amenable to statistical analysis. Exponential random graph models (ERGMs) emerged as one of the main families of models capable of capturing the complex dependence structure of network data in a wide range of applied contexts. The Bergm package for R has become a popular package to carry out Bayesian parameter inference, missing data imputation, model selection and goodness-of-fit diagnostics for ERGMs. Over the last few years, the package has been considerably improved in terms of efficiency by…
Astronomical data often suffer from noise and incompleteness. We extend the common mixtures-of-Gaussians density estimation approach to account for situations with a known sample incompleteness by simultaneous imputation from the current model. The method, called GMMis, generalizes existing Expectation-Maximization techniques for truncated data to arbitrary truncation geometries and probabilistic rejection processes, as long as they can be specified and do not depend on the density itself. The method accounts for independent multivariate normal measurement errors for each of the observed sampl…
In Survival Analysis, the observed lifetimes often correspond to individuals for which the event occurs within a specific calendar time interval. With such interval sampling, the lifetimes are doubly truncated at values determined by the birth dates and the sampling interval. This double truncation may induce a systematic bias in estimation, so specific corrections are needed. A relevant target in Survival Analysis is the hazard rate function, which represents the instantaneous probability for the event of interest. In this work we introduce a flexible estimation approach for the hazard rate u…
