In numerous regular statistical models, median bias reduction (Kenne Pagui et al., 2017) has proven to be a noteworthy improvement over maximum likelihood, alternative to mean bias reduction. The estimator is obtained as solution to a modified score equation ensuring smaller asymptotic median bias than the maximum likelihood estimator. This paper provides a simplified algebraic form of the adjustment term for general regular models. With the new formula, the estimation procedure benefits from a considerable computational gain by avoiding multiple summations and thus allows an efficient implementation. More importantly, the new formulation allows to highlight how the median bias reduction adjustment can be obtained by adding an extra term to the mean bias reduction adjustment. Illustrations are provided through new applications of median bias reduction to two regression models not belonging to the generalized linear models class, extended beta regression and beta-binomial regression. Mean bias reduction is also provided here for the latter model. Simulation studies show remarkable componentwise median centering of the median bias reduced estimator, while variability and coverage of related confidence intervals are comparable with those of mean bias reduction. Moreover, empirical results for the beta-binomial model show that the method is successful in solving maximum likelihood boundary estimate problem.
This paper presents an integrated framework for estimation and inference from generalized linear models using adjusted score equations that result in mean and median bias reduction. The framework unifies theoretical and methodological aspects of past research on mean bias reduction and accommodates, in a natural way, new advances on median bias reduction. General expressions for the adjusted score functions are derived in terms of quantities that are readily available in standard software for fitting generalized linear models. The resulting estimating equations are solved using a unifying quas…
Random-effects models are frequently used to synthesise information from different studies in meta-analysis. While likelihood-based inference is attractive both in terms of limiting properties and of implementation, its application in random-effects meta-analysis may result in misleading conclusions, especially when the number of studies is small to moderate. The current paper shows how methodology that reduces the asymptotic bias of the maximum likelihood estimator of the variance component can also substantially improve inference about the mean effect size. The results are derived for the mo…
With continuous outcomes, the average causal effect is typically defined using a contrast of expected potential outcomes. However, in the presence of skewed outcome data, the expectation may no longer be meaningful. In practice the typical approach is to either "ignore or transform" - ignore the skewness altogether or transform the outcome to obtain a more symmetric distribution, although neither approach is entirely satisfactory. Alternatively the causal effect can be redefined as a contrast of median potential outcomes, yet discussion of confounding-adjustment methods to estimate this parame…
Given samples $x_1,\cdots,x_n$, it is well known that any sample median value (not necessarily unique) minimizes the absolute loss $\sum_{i=1}^n |q-x_i|$. Interestingly, we show that the minimizer of the loss $\sum_{i=1}^n|q-x_i|^{1+ε}$ exhibits a singular perturbation behaviour that provides a unique definition for the sample median as $ε\rightarrow 0$. This definition is the unique point among all candidate median values that balances the $logarithmic$ moment of the empirical distribution. The result generalizes directly to breaking ties among sample quantiles when the quantile regression lo…
The length-biased Birnbaum-Saunders distribution is both useful and practical for environmental sciences. In this paper, we initially derive some new properties for the length-biased Birnbaum-Saunders distribution, showing that one of its parameters is the mode and that it is bimodal. We then introduce a new regression model based on this distribution. We implement use the maximum likelihood method for parameter estimation, approach interval estimation and consider three types of residuals. An elaborate Monte Carlo study is carried out for evaluating the performance of the likelihood-based est…
The manuscript discusses how to incorporate random effects for quantile regression models for clustered data with focus on settings with many but small clusters. The paper has three contributions: (i) documenting that existing methods may lead to severely biased estimators for fixed effects parameters; (ii) proposing a new two-step estimation methodology where predictions of the random effects are first computed {by a pseudo likelihood approach (the LQMM method)} and then used as offsets in standard quantile regression; (iii) proposing a novel bootstrap sampling procedure in order to reduce bi…
In this paper, we consider the beta prime regression model recently proposed by \cite{bour18}, which is tailored to situations where the response is continuous and restricted to the positive real line with skewed and long tails and the regression structure involves regressors and unknown parameters. We consider two different strategies of bias correction of the maximum-likelihood estimators for the parameters that index the model. In particular, we discuss bias-corrected estimators for the mean and the dispersion parameters of the model. Furthermore, as an alternative to the two analytically b…
We develop uniformly valid confidence regions for regression coefficients in a high-dimensional sparse median regression model with homoscedastic errors. Our methods are based on a moment equation that is immunized against non-regular estimation of the nuisance part of the median regression function by using Neyman's orthogonalization. We establish that the resulting instrumental median regression estimator of a target regression coefficient is asymptotically normally distributed uniformly with respect to the underlying sparse model and is semi-parametrically efficient. We also generalize our …
Biased sampling designs can be highly efficient when studying rare (binary) or low variability (continuous) endpoints. We consider longitudinal data settings in which the probability of being sampled depends on a repeatedly measured response through an outcome-related, auxiliary variable. Such auxiliary variable- or outcome-dependent sampling improves observed response and possibly exposure variability over random sampling, {even though} the auxiliary variable is not of scientific interest. {For analysis,} we propose a generalized linear model based approach using a sequence of two offsetted r…
Sensitivity analysis for measurement error can be applied in the absence of validation data by means of regression calibration and simulation-extrapolation. These have not been compared for this purpose. A simulation study was conducted comparing the performance of regression calibration and simulation-extrapolation in a multivariable model. The performance of the two methods was evaluated in terms of bias, mean squared error (MSE) and confidence interval coverage, for ranging reliability of the error-prone measurement (0.2-0.9), sample size (125-1,000), number of replicates (2-10), and R-squa…
Response-biased sampling, in which samples are drawn from a popula- tion according to the values of the response variable, is common in biomedical, epidemiological, economic and social studies. In particular, the complete obser- vations in data with censoring, truncation or missing covariates can be regarded as response-biased sampling under certain conditions. This paper proposes to use transformation models, known as the generalized accelerated failure time model in econometrics, for regression analysis with response-biased sampling. With unknown error distribution, the transformation models…
By the modified directed likelihood, higher order accurate confidence limits for a scalar parameter are obtained from the likelihood. They are conveniently described in terms of a confidence distribution, that is a sample dependent distribution function on the parameter space. In this paper we explore a different route to accurate confidence limits via tail-symmetric confidence curves, that is curves that describe equal tailed intervals at any level. Instead of modifying the directed likelihood, we consider inversion of the log-likelihood ratio when evaluated at the median of the maximum likel…
Two-step estimators often called upon to fit censored regression models in many areas of science and engineering. Since censoring incurs a bias in the naive least-squares fit, a two-step estimator first estimates the bias and then fits a corrected linear model. We develop a framework for performing valid /post-correction inference/ with two-step estimators. By exploiting recent results on post-selection inference, we obtain valid confidence intervals and significance tests for the fitted coefficients.
The Dirichlet distribution, also known as multivariate beta, is the most used to analyse frequencies or proportions data. Maximum likelihood is widespread for estimation of Dirichlet's parameters. However, for small sample sizes, the maximum likelihood estimator may shows a significant bias. In this paper, Dirchlet's parameters estimation is obtained through modified score functions aiming at mean and median bias reduction of the maximum likelihood estimator, respectively. A simulation study and an application compare the adjusted score approaches with maximum likelihood.
Correction for Weak IV Bias and Winner's Curse in Mendelian Randomization Egger Regression: Rerandomized Egger estimator
Huazhong University of Science and Technology（华中科技大学）；Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属同济医院）
We present a linear regression method for predictions on a small data set making use of a second possibly biased data set that may be much larger. Our method fits linear regressions to the two data sets while penalizing the difference between predictions made by those two models. The resulting algorithm is a shrinkage method similar to those used in small area estimation. We find a Stein-type finding for Gaussian responses: when the model has 5 or more coefficients and 10 or more error degrees of freedom, it becomes inadmissible to use only the small data set, no matter how large the bias is. …
Negative binomial regression is commonly employed to analyze overdispersed count data. With small to moderate sample sizes, the maximum likelihood estimator of the dispersion parameter may be subject to a significant bias, that in turn affects inference on mean parameters. This paper proposes inference for negative binomial regression based on adjustments of the score function aimed at mean and median bias reduction. The resulting estimating equations are similar to those available for improved inference in generalized linear models and, in particular, can be solved using a suitable extension …
University of Patras（帕特雷大学）；Roma Tre University（罗马第三大学）；National Technical University of Athens（雅典国立技术大学）
Machine and statistical learning algorithms can be reliably automated and applied at scale. Therefore, they can constitute a considerable asset for designing practical forecasting systems, such as those related to urban water demand. Quantile regression algorithms are statistical and machine learning algorithms that can provide probabilistic forecasts in a straightforward way, and have not been applied so far for urban water demand forecasting. In this work, we aim to fill this gap by automating and extensively comparing several quantile-regression-based practical systems for probabilistic one…
The issues of bias-correction and robustness are crucial in the strategy of divide-and-conquer (DC), especially for asymmetric nonparametric models with massive data. It is known that quantile-based methods can achieve the robustness, but the quantile estimation for nonparametric regression has non-ignorable bias when the error distribution is asymmetric. This paper explores a global bias-corrected DC by quantile-matched composite for nonparametric regressions with general error distributions. The proposed strategies can achieve the bias-correction and robustness, simultaneously. Unlike common…
This paper presents a novel estimation approach for cumulative link models, based on median bias reduction as developed in Kenne Pagui et al. (2017). The median bias reduced estimator is obtained as solution of an estimating equation based on an adjustment of the score. It allows to obtain higher-order median centering of maximum likelihood estimates without requiring their finiteness. Moreover, the estimator is equivariant under componentwise monotone reparameterizations and the method is effective in preventing boundary estimates. We evaluate the properties of the median bias reduced estimat…
Technische Universität Darmstadt（达姆施塔特工业大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
The Nadaraya-Watson kernel estimator is among the most popular nonparameteric regression technique thanks to its simplicity. Its asymptotic bias has been studied by Rosenblatt in 1969 and has been reported in a number of related literature. However, Rosenblatt's analysis is only valid for infinitesimal bandwidth. In contrast, we propose in this paper an upper bound of the bias which holds for finite bandwidths. Moreover, contrarily to the classic analysis we allow for discontinuous first order derivative of the regression function, we extend our bounds for multidimensional domains and we inclu…
This paper deals with the problem of estimating a slope parameter in a simple linear regression model, where independent variables have functional measurement errors. Measurement errors in independent variables, as is well known, cause biasedness of the ordinary least squares estimator. A general procedure for the bias reduction is presented in a finite sample situation, and some exact bias-reduced estimators are proposed. Also, it is shown that certain truncation procedures improve the mean square errors of the ordinary least squares and the bias-reduced estimators.
For regular parametric problems, we show how median centering of the maximum likelihood estimate can be achieved by a simple modification of the score equation. For a scalar parameter of interest, the estimator is equivariant under interest respecting parameterizations and third-order median unbiased. With a vector parameter of interest, componentwise equivariance and third-order median centering are obtained. Like Firth's (1993, Biometrika) implicit method for bias reduction, the new method does not require finiteness of the maximum likelihood estimate and is effective in preventing infinite …
Universidade Federal Pernambuco（伯南布哥联邦大学）；Aeronautics Institute of Technology (ITA)（航空技术学院）；Universidade Federal de Santa Maria（圣玛丽亚联邦大学）
The Rayleigh regression model was recently proposed for modeling amplitude values of synthetic aperture radar (SAR) image pixels. However, inferences from such model are based on the maximum likelihood estimators, which can be biased for small signal lengths. The Rayleigh regression model for SAR images often takes into account small pixel windows, which may lead to inaccurate results. In this letter, we introduce bias-adjusted estimators tailored for the Rayleigh regression model based on: (i) the Cox and Snell's method; (ii) the Firth's scheme; and (iii) the parametric bootstrap method. We p…
The median absolute deviation is a widely used robust measure of statistical dispersion. Using a scale constant, we can use it as an asymptotically consistent estimator for the standard deviation under normality. For finite samples, the scale constant should be corrected in order to obtain an unbiased estimator. The bias-correction factor depends on the sample size and the median estimator. When we use the traditional sample median, the factor values are well known, but this approach does not provide optimal statistical efficiency. In this paper, we present the bias-correction factors for the …
In this article, we propose a new nonparametric data analysis tool, which we call nonparametric modal regression, to investigate the relationship among interested variables based on estimating the mode of the conditional density of a response variable Y given predictors X. The nonparametric modal regression is distinguished from the conventional nonparametric regression in that, instead of the conditional average or median, it uses the "most likely" conditional values to measures the center. Better prediction performance and robustness are two important characteristics of nonparametric modal r…
The shape of the relationship between a continuous exposure variable and a binary disease variable is often central to epidemiologic investigations. This paper investigates a number of issues surrounding inference and the shape of the relationship. Presuming that the relationship can be expressed in terms of regression coefficients and a shape parameter, we investigate how well the shape can be inferred in settings which might typify epidemiologic investigations and risk assessment. We also consider a suitable definition of the median effect of exposure, and investigate how precisely this can …
Recently, there has been a growing need in analyzing data on manifolds owing to their important role in diverse fields of science and engineering. In the literature of manifold-valued data analysis up till now, however, only a few works have been carried out concerning the robustness of estimation against noises, outliers, and other sources of perturbations. In this regard, we introduce a novel extrinsic framework for analyzing manifold valued data in a robust manner. First, by extending the notion of the geometric median, we propose a new robust location parameter on manifolds, so-called the …
Salesforce Research（赛富时研究院）；University of California, Berkeley（加州大学伯克利分校）
Estimating the data uncertainty in regression tasks is often done by learning a quantile function or a prediction interval of the true label conditioned on the input. It is frequently observed that quantile regression -- a vanilla algorithm for learning quantiles with asymptotic guarantees -- tends to \emph{under-cover} than the desired coverage level in reality. While various fixes have been proposed, a more fundamental understanding of why this under-coverage bias happens in the first place remains elusive. In this paper, we present a rigorous theoretical study on the coverage of uncertainty…
Zuo (2019) (Z19) addressed the computation of the projection regression depth (PRD) and its induced median (the maximum depth estimator). Z19 achieved the exact computation of PRD via a modified version of regular univariate sample median, which resulted in the loss of invariance of PRD and the equivariance of depth induced median. This article achieves the exact computation without scarifying the invariance of PRD and the equivariance of the regression median. Z19 also addressed the approximate computation of PRD induced median, the naive algorithm in Z19 is very slow. This article modifies t…
The estimation of categorical response models using bias-reducing adjusted score equations has seen extensive theoretical research and applied use. The resulting estimates have been found to have superior frequentist properties to what maximum likelihood generally delivers and to be finite, even in cases where the maximum likelihood estimates are infinite. We briefly review mean and median bias reduction of maximum likelihood estimates via adjusted score equations in an illustration-driven way, and discuss their particular equivariance properties under parameter transformations. We then apply …
Scalable Gaussian Process Regression Via Median Posterior Inference for Estimating Multi-Pollutant Mixture Health Effects
Google Research（谷歌研究）；Department of Statistics University of California Davis（加州大学戴维斯分校统计系）；Department of Environmental Health Harvard University（哈佛大学环境健康系）；Department of Biostatistics Harvard University（哈佛大学生物统计学系）
Shanghai Jiao Tong University（上海交通大学）；Iowa State University（爱荷华州立大学）
Decentralized sparsity learning has attracted a significant amount of attention recently due to its rapidly growing applications. To obtain the robust and sparse estimators, a natural idea is to adopt the non-smooth median loss combined with a $\ell_1$ sparsity regularizer. However, most of the existing methods suffer from slow convergence performance caused by the {\em double} non-smooth objective. To accelerate the computation, in this paper, we proposed a decentralized surrogate median regression (deSMR) method for efficiently solving the decentralized sparsity learning problem. We show tha…
We consider the non-parametric regression problem under Huber's $ε$-contamination model, in which an $ε$ fraction of observations are subject to arbitrary adversarial noise. We first show that a simple local binning median step can effectively remove the adversary noise and this median estimator is minimax optimal up to absolute constants over the Hölder function class with smoothness parameters smaller than or equal to 1. Furthermore, when the underlying function has higher smoothness, we show that using local binning median as pre-preprocessing step to remove the adversarial noise, then we c…
We develop a Bayesian median autoregressive (BayesMAR) model for time series forecasting. The proposed method utilizes time-varying quantile regression at the median, favorably inheriting the robustness of median regression in contrast to the widely used mean-based methods. Motivated by a working Laplace likelihood approach in Bayesian quantile regression, BayesMAR adopts a parametric model bearing the same structure as autoregressive models by altering the Gaussian error to Laplace, leading to a simple, robust, and interpretable modeling strategy for time series forecasting. We estimate model…
