This paper discusses the problem of determining optimal designs for regression models, when the observations are dependent and taken on an interval. A complete solution of this challenging optimal design problem is given for a broad class of regression models and covariance kernels.
We propose a class of estimators which are only slightly more complicated than the ordinary least-squares estimators. We then demonstrate that we can design the experiments, such that asymptotically the new estimators achieve the same precision as the best linear unbiased estimator computed for the whole trajectory of the process. As a by-product we derive explicit expressions for the BLUE in the continuous time model and analytic expressions for the optimal designs in a wide class of regression models. We also demonstrate that for a finite number of observations the precision of the proposed procedure, which includes the estimator and design, is very close to the best achievable. The results are illustrated on a few numerical examples.
Log-symmetric regression models are particularly useful when the response variable is continuous, strictly positive and asymmetric. In this paper, we proposed a class of log-symmetric regression models in the context of correlated errors. The proposed models provide a novel alternative to the existing log-symmetric regression models due to its flexibility in accommodating correlation. We discuss some properties, parameter estimation by the conditional maximum likelihood method and goodness of fit of the proposed model. We also provide expressions for the observed Fisher information matrix. A M…
Traditionally, the least squares regression is mainly concerned with studying the effects of individual predictor variables, but strongly correlated variables generate multicollinearity which makes it difficult to study their effects. Existing methods for handling multicollinearity such as ridge regression are complicated. To resolve the multicollinearity issue without abandoning the simple least squares regression, for situations where predictor variables are in groups with strong within-group correlations but weak between-group correlations, we propose to study the effects of the groups with…
We consider the problem of efficient statistical inference for comparing two regression curves estimated from two samples of dependent measurements. Based on a representation of the best pair of linear unbiased estimators in continuous time models as a stochastic integral, an efficient pair of linear unbiased estimators with corresponding optimal designs for finite sample size is constructed. This pair minimises the width of the confidence band for the difference between the estimated curves. We thus extend results readily available in the literature to the case of correlated observations and …
Heavy-tailed continuous shrinkage priors, such as the horseshoe prior, are widely used for sparse estimation problems. However, there is limited work extending these priors to predictors with grouping structures. Of particular interest in this article, is regression coefficient estimation where pockets of high collinearity in the covariate space are contained within known covariate groupings. To assuage variance inflation due to multicollinearity we propose the group inverse-gamma gamma (GIGG) prior, a heavy-tailed prior that can trade-off between local and group shrinkage in a data adaptive f…
Lasso is a popular and efficient approach to simultaneous estimation and variable selection in high-dimensional regression models. In this paper, a robust LAD-lasso method for multiple outcomes is presented that addresses the challenges of non-normal outcome distributions and outlying observations. Measured covariate data from space or time, or spectral bands or genomic positions often have natural correlation structure arising from measuring distance between the covariates. The proposed multi-outcome approach includes handling of such covariate blocks by a group fusion penalty, which encourag…
Measurement error arises through a variety of mechanisms. A rich literature exists on the bias introduced by covariate measurement error and on methods of analysis to address this bias. By comparison, less attention has been given to errors in outcome assessment and non-classical covariate measurement error. We consider an extension of the regression calibration method to settings with errors in a continuous outcome, where the errors may be correlated with prognostic covariates or with covariate measurement error. This method adjusts for the measurement error in the data and can be applied wit…
Medical studies that depend on electronic health records (EHR) data are often subject to measurement error, as the data are not collected to support research questions under study. These data errors, if not accounted for in study analyses, can obscure or cause spurious associations between patient exposures and disease risk. Methodology to address covariate measurement error has been well developed; however, time-to-event error has also been shown to cause significant bias but methods to address it are relatively underdeveloped. More generally, it is possible to observe errors in both the cova…
Testing procedures for assessing a parametric regression model with circular response and $\mathbb{R}^d$-valued covariate are proposed and analyzed in this work both for independent and for spatially correlated data. The test statistics are based on a circular distance comparing a (non-smoothed or smoothed) parametric circular estimator and a nonparametric one. Properly designed bootstrap procedures for calibrating the tests in practice are also presented. Finite sample performance of the tests in different scenarios with independent and spatially correlated samples, is analyzed by simulations…
This paper studies optimal designs for linear regression models with correlated effects for single responses. We introduce the concept of rhombic design to reduce the computational complexity and find a semi-algebraic description for the D-optimality of a rhombic design via the Kiefer-Wolfowitz equivalence theorem. Subsequently, we show that the structure of an optimal rhombic design depends directly on the correlation structure of the random coefficients.
We propose an optimal estimating equation for logistic regression with linked data while accounting for false positives. It builds on a previous solution but estimates the regression coefficients with a smaller variance, in large samples.
We present a nonparametric method for estimating the value and several derivatives of an unknown, sufficiently smooth real-valued function of real-valued arguments from a finite sample of points, where both the function arguments and the corresponding values are known only up to measurement errors having some assumed distribution and correlation structure. The method, Moving Taylor Bayesian Regression (MOTABAR), uses Bayesian updating to find the posterior mean of the coefficients of a Taylor polynomial of the function at a moving position of interest. When measurement errors are neglected, MO…
Sparse regression is frequently employed in diverse scientific settings as a feature selection method. A pervasive aspect of scientific data that hampers both feature selection and estimation is the presence of strong correlations between predictive features. These fundamental issues are often not appreciated by practitioners, and jeapordize conclusions drawn from estimated models. On the other hand, theoretical results on sparsity-inducing regularized regression such as the Lasso have largely addressed conditions for selection consistency via asymptotics, and disregard the problem of model se…
This paper presents a new approach for trees-based regression, such as simple regression tree, random forest and gradient boosting, in settings involving correlated data. We show the problems that arise when implementing standard trees-based regression models, which ignore the correlation structure. Our new approach explicitly takes the correlation structure into account in the splitting criterion, stopping rules and fitted values in the leaves, which induces some major modifications of standard methodology. The superiority of our new approach over trees-based models that do not account for th…
We present a framework for performing regression when both covariate and response are probability distributions on a compact interval $Ω\subset\mathbb{R}$. Our regression model is based on the theory of optimal transportation and links the conditional Fréchet mean of the response distribution to the covariate distribution via an optimal transport map. We define a Fréchet-least-squares estimator of this regression map, and establish its consistency and rate of convergence to the true map, under both full and partial observation of the regression pairs. Computation of the estimator is shown to r…
The problem of estimating censored linear regression models with autocorrelated errors arises in many environmental and social studies. The present work proposes a Bayesian approach to estimate censored regression models with AR(p) errors. The algorithm developed here considers the Gibbs sampler with data augmentation(GDA), in which, at each iteration, both the model parameters and the latent variables are sampled. The data augmentation is achieved from multiple sampling of the latent variables from the corresponding conditional distributions. A suitable variable transformation allows the full…
In this paper we investigate the problem of designing experiments for series estimators in nonparametric regression models with correlated observations. We use projection based estimators to derive an explicit solution of the best linear oracle estimator in the continuous time model for all Markovian-type error processes. These solutions are then used to construct estimators, which can be calculated from the available data along with their corresponding optimal design points. Our results are illustrated by means of a simulation study, which demonstrates that the new series estimator has a bett…
In this paper we consider the problem of constructing $T$-optimal discriminating designs for Fourier regression models. We provide explicit solutions of the optimal design problem for discriminating between two Fourier regression models, which differ by at most three trigonometric functions. In general, the $T$-optimal discriminating design depends in a complicated way on the parameters of the larger model, and for special configurations of the parameters $T$-optimal discriminating designs can be found analytically. Moreover, we also study this dependence in the remaining cases by calculating …
We investigate R-optimal designs for multi-response regression models with multi-factors, where the random errors in these models are correlated. Several theoretical results are derived for Roptimal designs, including scale invariance, reflection symmetry, line and plane symmetry, and dependence on the covariance matrix of the errors. All the results can be applied to linear and nonlinear models. In addition, an efficient algorithm based on an interior point method is developed for finding R-optimal designs on discrete design spaces. The algorithm is very flexible, and can be applied to any mu…
A new bandwidth selection method for the fuzzy regression discontinuity estimator is proposed. The method chooses two bandwidths simultaneously, one for each side of the cut-off point by using a criterion based on the estimated asymptotic mean square error taking into account a second-order bias term. A simulation study demonstrates the usefulness of the proposed method.
Optimal designs are required to make efficient statistical experiments. D-optimal designs for some models are calculated by using canonical moments. On the other hand, integrable systems are dynamical systems whose solutions can be written down concretely. In this paper, polynomial regression models with prior information are discussed. In order to calculate D-optimal designs for these models, a useful relationship between canonical moments and discrete integrable systems is used. By using canonical moments and discrete integrable systems, an algorithm for calculating D-optimal designs for the…
Difference-based methods have been attracting increasing attention in nonparametric regression, in particular for estimating the residual variance.To implement the estimation, one needs to choose an appropriate difference sequence, mainly between {\em the optimal difference sequence} and {\em the ordinary difference sequence}. The difference sequence selection is a fundamental problem in nonparametric regression, and it remains a controversial issue for over three decades. In this paper, we propose to tackle this challenging issue from a very unique perspective, namely by introducing a new dif…
We typically construct optimal designs based on a single objective function. To better capture the breadth of an experiment's goals, we could instead construct a multiple objective optimal design based on multiple objective functions. While algorithms have been developed to find multi-objective optimal designs (e.g. efficiency-constrained and maximin optimal designs), it is far less clear how to verify the optimality of a solution obtained from an algorithm. In this paper, we provide theoretical results characterizing optimality for efficiency-constrained and maximin optimal designs on a discr…
This article develops a Bayesian approach for estimating panel quantile regression with binary outcomes in the presence of correlated random effects. We construct a working likelihood using an asymmetric Laplace (AL) error distribution and combine it with suitable prior distributions to obtain the complete joint posterior distribution. For posterior inference, we propose two Markov chain Monte Carlo (MCMC) algorithms but prefer the algorithm that exploits the blocking procedure to produce lower autocorrelation in the MCMC draws. We also explain how to use the MCMC draws to calculate the margin…
In multi-response regression models, the error covariance matrix is never known in practice. Thus, there is a need for optimal designs which are robust against possible misspecification of the error covariance matrix. In this paper, we approximate the error covariance matrix with a neighbourhood of covariance matrices, in order to define minimax D-optimal designs which are robust against small departures from an assumed error covariance matrix. It is well known that the optimization problems associated with robust designs are non-convex, which makes it challenging to construct robust designs a…
A new bandwidth selection rule that uses different bandwidths for the local linear regression estimators on the left and the right of the cut-off point is proposed for the sharp regression discontinuity estimator of the mean program impact at the cut-off point. The asymptotic mean squared error of the estimator using the proposed bandwidth selection rule is shown to be smaller than other bandwidth selection rules proposed in the literature. An extensive simulation study shows that the proposed method's performances for the sample sizes 500, 2000, and 5000 closely match the theoretical predicti…
We study identification and estimation of the average treatment effect in a correlated random coefficients model that allows for first stage heterogeneity and binary instruments. The model also allows for multiple endogenous variables and interactions between endogenous variables and covariates. Our identification approach is based on averaging the coefficients obtained from a collection of ordinary linear regressions that condition on different realizations of a control function. This identification strategy suggests a transparent and computationally straightforward estimator of a trimmed ave…
In nonlinear regression models the Fisher information depends on the parameters of the model. Consequently, optimal designs maximizing some functional of the information matrix cannot be implemented directly but require some preliminary knowledge about the unknown parameters. Bayesian optimality criteria provide an attractive solution to this problem. These criteria depend sensitively on a reasonable specification of a prior distribution for the model parameters which might not be available in all applications. In this paper we investigate Bayesian optimality criteria with non-informative prio…
In this article, the parameters of a hybrid log-linear model (log-Poisson) are estimated using the fuzzy least-squares (FLS) procedures (Celmiņš, 987a,b, D'Urso and Gastaldi, 2000, DUrso and Gastaldi, 2001). A goodness of fit have been derived in order to assess and compare this new model and the classical log-Poisson regression in loss reserving framework (Mack, 1991). Both the hybrid model and its goodness of fit are performed on a loss reserving data.
Considering parallel tempering and comparing post-treatment procedures in Bayesian Profile Regression Models for a survival outcome and correlated exposures
Autorité de Sûreté Nucléaire et de Radioprotection (ASNR)（核安全与辐射防护局）；PSE-SANTE/SESANE/LEPID；Université Paris Cité（巴黎西岱大学）；CNRS, Inserm, P-MIM, BioSTM（法国国家科学研究中心、INSERM、P-MIM、BioSTM）
We consider estimation in a high-dimensional linear model with strongly correlated variables. We propose to cluster the variables first and do subsequent sparse estimation such as the Lasso for cluster-representatives or the group Lasso based on the structure from the clusters. Regarding the first step, we present a novel and bottom-up agglomerative clustering algorithm based on canonical correlations, and we show that it finds an optimal solution and is statistically consistent. We also present some theoretical arguments that canonical correlation based clustering leads to a better-posed comp…
We propose a distributed quadratic inference function framework to jointly estimate regression parameters from multiple potentially heterogeneous data sources with correlated vector outcomes. The primary goal of this joint integrative analysis is to estimate covariate effects on all outcomes through a marginal regression model in a statistically and computationally efficient way. We develop a data integration procedure for statistical estimation and inference of regression parameters that is implemented in a fully distributed and parallelized computational scheme. To overcome computational and…
The paper proposes to analyze epidemiological data using regression models which enable subject-matter (epidemiological) interpretation of such data whether with uncorrelated or correlated predictors. To this end, response functions should include not only terms linear in predictors but also higher order ones (e.g. quadratic and cross terms). For epidemiological interpretation of a regression model, the suggestion is to construct conditional functions derived from the general regression function with the values of all predictor variables held fixed excepting one predictor. Unlike the conventio…
We consider T-optimal experiment design problems for discriminating multi-factor polynomial regression models where the design space is defined by polynomial inequalities and the regression parameters are constrained to given convex sets. Our proposed optimality criterion is formulated as a convex optimization problem with a moment cone constraint. When the regression models have one factor, an exact semidefinite representation of the moment cone constraint can be applied to obtain an equivalent semidefinite program. When there are two or more factors in the models, we apply a moment relaxatio…
In this paper we introduce a new methodology to determine an optimal coefficient of penalized functional regression. We assume the dependent, independent variables and the regression coefficients are functions of time and error dynamics follow a stochastic differential equation. First we construct our objective function as a time dependent residual sum of square and then minimize it with respect to regression coefficients subject to different error dynamics such as LASSO, group LASSO, fused LASSO and cubic smoothing spline. Then we use Feynman-type path integral approach to determine a Schrödi…
Two-phase designs involve measuring extra variables on a subset of the cohort where some variables are already measured. The goal of two-phase designs is to choose a subsample of individuals from the cohort and analyse that subsample efficiently. It is of interest to obtain an optimal design that gives the most efficient estimates of regression parameters. In this paper, we propose a multi-wave sampling design to approximate the optimal design for design-based estimators. Influences functions are used to compute the optimal sampling allocations. We propose to use informative priors on regressi…
We consider optimal non-sequential designs for a large class of (linear and nonlinear) regression models involving polynomials and rational functions with heteroscedastic noise also given by a polynomial or rational weight function. The proposed method treats D-, E-, A-, and $Φ_p$-optimal designs in a unified manner, and generates a polynomial whose zeros are the support points of the optimal approximate design, generalizing a number of previously known results of the same flavor. The method is based on a mathematical optimization model that can incorporate various criteria of optimality and c…
We investigate optimal subsampling for quantile regression. We derive the asymptotic distribution of a general subsampling estimator and then derive two versions of optimal subsampling probabilities. One version minimizes the trace of the asymptotic variance-covariance matrix for a linearly transformed parameter estimator and the other minimizes that of the original parameter estimator. The former does not depend on the densities of the responses given covariates and is easy to implement. Algorithms based on optimal subsampling probabilities are proposed and asymptotic distributions and asympt…
We characterize $D$-optimal designs in the two-dimensional Poisson regression model with synergetic interaction and provide an explicit proof. The proof is based on the idea of reparameterization of the design region in terms of contours of constant intensity. This approach leads to a substantial reduction of complexity as properties of the sensitivity can be treated along and across the contours separately. Furthermore, some extensions of this result to higher dimensions are presented.
We propose an optimal experimental design for a curvilinear regression model that minimizes the band-width of simultaneous confidence bands. Simultaneous confidence bands for curvilinear regression are constructed by evaluating the volume of a tube about a curve that is defined as a trajectory of a regression basis vector (Naiman, 1986). The proposed criterion is constructed based on the volume of a tube, and the corresponding optimal design that minimizes the volume of tube is referred to as the tube-volume optimal (TV-optimal) design. For Fourier and weighted polynomial regressions, the prob…
High-dimensional prediction typically comprises two steps: variable selection and subsequent least-squares refitting on the selected variables. However, the standard variable selection procedures, such as the lasso, hinge on tuning parameters that need to be calibrated. Cross-validation, the most popular calibration scheme, is computationally costly and lacks finite sample guarantees. In this paper, we introduce an alternative scheme, easy to implement and both computationally and theoretically efficient.
In this article, we propose the Sample Information Optimal Estimator (SIOE) and the Stochastic Restricted Optimal Estimator (SROE) for misspecified linear regression model when multicollinearity exists among explanatory variables. Further, we obtain the superiority conditions of proposed estimators over some other existing estimators in the Mean Square Error Matrix (MSEM) criterion in a standard form which can apply to all estimators considered in this study. Finally, a real world example and a Monte Carlo simulation study are presented for the proposed estimators to illustrate the theoretical…
