We study tail estimation in Pareto-like settings for datasets with a high percentage of randomly right-censored data, and where some expert information on the tail index is available for the censored observations. This setting arises for instance naturally for liability insurance claims, where actuarial experts build reserves based on the specificity of each open claim, which can be used to improve the estimation based on the already available data points from closed claims. Through an entropy-perturbed likelihood we derive an explicit estimator and establish a close analogy with Bayesian methods. Embedded in an extreme value approach, asymptotic normality of the estimator is shown, and when the expert is clair-voyant, a simple combination formula can be deduced, bridging the classical statistical approach with the expert information. Following the aforementioned combination formula, a combination of quantile estimators can be naturally defined. In a simulation study, the estimator is shown to often outperform the Hill estimator for censored observations and recent Bayesian solutions, some of which require more information than usually available. Finally we perform a case study on a motor third-party liability insurance claim dataset, where Hill-type and quantile plots incorporate ultimate values into the estimation procedure in an intuitive manner.
The classical mixture of linear experts (MoE) model is one of the widespread statistical frameworks for modeling, classification, and clustering of data. Built on the normality assumption of the error terms for mathematical and computational convenience, the classical MoE model has two challenges: 1) it is sensitive to atypical observations and outliers, and 2) it might produce misleading inferential results for censored data. The paper is then aimed to resolve these two challenges, simultaneously, by proposing a novel robust MoE model for model-based clustering and discriminant censored data …
This paper addresses the problem of estimating, in the presence of random censoring as well as competing risks, the extreme value index of the (sub)-distribution function associated to one particular cause, in the heavy-tail case. Asymptotic normality of the proposed estimator (which has the form of an Aalen-Johansen integral, and is the first estimator proposed in this context) is established. A small simulation study exhibits its performances for finite samples. Estimation of extreme quantiles of the cumulative incidence function is also addressed.
When observations are subject to right censoring, weighted least squares with appropriate weights (to adjust for censoring) is sometimes used for parameter estimation. With Stute's weighted least squares method, when the largest observation is censored ($Y_{(n)}^+$), it is natural to apply the redistribution to the right algorithm of Efron (1967). However, Efron's redistribution algorithm can lead to bias and inefficiency in estimation. This study explains the issues clearly and proposes some alternative ways of treating $Y_{(n)}^+$. The first four proposed approaches are based on the well kno…
Herein, we propose a Spearman rank correlation based screening procedure for ultrahigh-dimensional data with censored response case. The proposed method is model-free without specifying any regression forms of predictors or response variable and is robust under the unknown monotone transformations of these response variable and predictors. The sure-screening and rank-consistency properties are established under some mild regularity conditions. Simulation studies demonstrate that the new screening method performs well in the presence of a heavy-tailed distribution, strongly dependent predictors…
In the framework of censored data modeling, the classical linear regression model that assumes normally distributed random errors has received increasing attention in recent years, mainly for mathematical and computational convenience. However, practical studies have often criticized this linear regression model due to its sensitivity to departure from the normality and from the partial nonlinearity. This paper proposes to solve these potential issues simultaneously in the context of the partial linear regression model by assuming that the random errors follow a scale-mixture of normal (SMN) f…
A common assumption regarding the standard tobit model is the normality of the error distribution. However, asymmetry and bimodality may be present and alternative tobit models must be used. In this paper, we propose a tobit model based on the class of log-symmetric distributions, which includes as special cases heavy and light tailed distributions and bimodal distributions. We implement a likelihood-based approach for parameter estimation and derive a type of residual. We then discuss the problem of performing testing inference in the proposed class by using the likelihood ratio and gradient …
This paper addresses the problem of estimating the extreme value index in presence of random censoring for distributions in the Weibull domain of attraction. The methodologies introduced in [Worms (2014)], in the heavy-tailed case, are adapted here to the negative extreme value index framework, leading to the definition of weighted versions of the popular moments of relative excesses with arbitrary exponent. This leads to the definition of two families of estimators (with an adaptation of the so called Moment estimator as a particular case), for which the consistency is proved under a first or…
Nonparametric and semiparametric methods are commonly used in survival analysis to mitigate the bias due to model misspecification. However, such methods often cannot estimate upper-tail survival quantiles when a sizable proportion of the data are censored, in which case parametric likelihood-based estimators present a viable alternative. In this article, we extend a popular family of parametric survival models which make the Accelerated Failure Time (AFT) assumption to account for heteroscedasticity in the survival times. The conditional variances can depend on arbitrary covariates, thus addi…
In risk analysis, a global fit that appropriately captures the body and the tail of the distribution of losses is essential. Modelling the whole range of the losses using a standard distribution is usually very hard and often impossible due to the specific characteristics of the body and the tail of the loss distribution. A possible solution is to combine two distributions in a splicing model: a light-tailed distribution for the body which covers light and moderate losses, and a heavy-tailed distribution for the tail to capture large losses. We propose a splicing model with a mixed Erlang (ME)…
Predicting an individual's risk of experiencing a future clinical outcome is a statistical task with important consequences for both practicing clinicians and public health experts. Modern observational databases such as electronic health records (EHRs) provide an alternative to the longitudinal cohort studies traditionally used to construct risk models, bringing with them both opportunities and challenges. Large sample sizes and detailed covariate histories enable the use of sophisticated machine learning techniques to uncover complex associations and interactions, but observational databases…
In this paper, we suggest using a skew Gaussian-log Gaussian model for the analysis of spatial censored data from a Bayesian point of view. This approach furnishes an extension of the skew log Gaussian model to accommodate to both skewness and heavy tails and also censored data. All of the characteristics mentioned are three pervasive features of spatial data. We utilize data augmentation method and Markov chain Monte Carlo (MCMC) algorithms to do posterior calculations. The methodology is illustrated using simulated data, as well as applying it to a real data set. Keywords: Censored data, dat…
Stochastic weather generators (SWGs) are digital twins of complex weather processes and widely used in agriculture and urban design. Due to improved measuring instruments, an accurate SWG for high-frequency precipitation is now possible. However, high-frequency precipitation data are more zero-inflated, skewed, and heavy-tailed than common (hourly or daily) precipitation data. Therefore, classical methods that either model precipitation occurrence independently of their intensity or assume that the precipitation follows a censored meta-Gaussian process may not be appropriate. In this work, we …
The Student-$t$ distribution is widely used in statistical modeling of datasets involving outliers since its longer-than-normal tails provide a robust approach to hand such data. Furthermore, data collected over time may contain censored or missing observations, making it impossible to use standard statistical procedures. This paper proposes an algorithm to estimate the parameters of a censored linear regression model when the regression errors are autocorrelated and the innovations follow a Student-$t$ distribution. To fit the proposed model, maximum likelihood estimates are obtained througho…
Modelling of precipitation, including extremes, is important for hydrological and agricultural applications. Traditionally, because of large sample properties for data over a large threshold value, generalised Pareto (GP) distributions are often used for modelling extreme rainfall. It can be shown that under certain conditions the generalised hyperbolic (GH) distributions can approximate the power law decay of the GP distribution in the tails. Given their flexible form, this raises the possibility that distributions from the GH family serve as a model for the entire rainfall distribution thus …
The class of inhomogeneous phase-type distributions (IPH) was recently introduced in Albrecher and Bladt (2019) as an extension of the classical phase-type (PH) distributions. Like PH distributions, the class of IPH is dense in the class of distributions on the positive halfline, but leads to more parsimonious models in the presence of heavy tails. In this paper we propose a fitting procedure for this class to given data. We furthermore consider an analogous extension of Kulkarni's multivariate phase-type class (Kulkarni, 1989) to the inhomogeneous framework and study parameter estimation for …
Since the inception of Bitcoin in 2008, cryptocurrencies have played an increasing role in the world of e-commerce, but the recent turbulence in the cryptocurrency market in 2018 has raised some concerns about their stability and associated risks. For investors, it is crucial to uncover the dependence relationships between cryptocurrencies for a more resilient portfolio diversification. Moreover, the stochastic behavior in both tails is important, as long positions are sensitive to a decrease in prices (lower tail), while short positions are sensitive to an increase in prices (upper tail). In …
To disentangle the complex non-stationary dependence structure of precipitation extremes over the entire contiguous U.S., we propose a flexible local approach based on factor copula models. Our sub-asymptotic spatial modeling framework yields non-trivial tail dependence structures, with a weakening dependence strength as events become more extreme, a feature commonly observed with precipitation data but not accounted for in classical asymptotic extreme-value models. To estimate the local extremal behavior, we fit the proposed model in small regional neighborhoods to high threshold exceedances,…
We respond to Tetlock et al. (2022) showing 1) how expert judgment fails to reflect tail risk, 2) the lack of compatibility between forecasting tournaments and tail risk assessment methods (such as extreme value theory). More importantly, we communicate a new result showing a greater gap between the properties of tail expectation and those of the corresponding probability.
