In this paper we propose a semiparametric spatial autoregressive model that combines a linear covariate component with a nonparametrically estimated spatial term, allowing flexible dependence modeling without restrictive covariance structure while preserving interpretability. We establish asymptotic properties, including consistency and asymptotic normality, and evaluate performance through simulations and real data. Results show competitive predictive accuracy relative to geostatistical methods and improved interpretability compared to spatial econometric models.
This paper considers an estimation of semiparametric functional (varying)-coefficient quantile regression with spatial data. A general robust framework is developed that treats quantile regression for spatial data in a natural semiparametric way. The local M-estimators of the unknown functional-coefficient functions are proposed by using local linear approximation, and their asymptotic distributions are then established under weak spatial mixing conditions allowing the data processes to be either stationary or nonstationary with spatial trends. Application to a soil data set is demonstrated wi…
Nonparametric methods have been very popular in the last couple of decades in time series and regression, but no such development has taken place for spatial models. A rather obvious reason for this is the curse of dimensionality. For spatial data on a grid evaluating the conditional mean given its closest neighbors requires a four-dimensional nonparametric regression. In this paper a semiparametric spatial regression approach is proposed to avoid this problem. An estimation procedure based on combining the so-called marginal integration technique with local linear kernel estimation is develop…
Stimulated by the Boston house price data, in this paper, we propose a semiparametric spatial dynamic model, which extends the ordinary spatial autoregressive models to accommodate the effects of some covariates associated with the house price. A profile likelihood based estimation procedure is proposed. The asymptotic normality of the proposed estimators are derived. We also investigate how to identify the parametric/nonparametric components in the proposed semiparametric model. We show how many unknown parameters an unknown bivariate function amounts to, and propose an AIC/BIC of nonparametr…
In this article, we propose a penalized high dimensional semiparametric model average quantile prediction approach that is robust for forecasting the conditional quantile of the response. We consider a two-step estimation procedure. In the first step, we use a local linear regression approach to estimate the individual marginal quantile functions, and approximate the conditional quantile of the response by an affine combination of one-dimensional marginal quantile regression functions. In the second step, based on the nonparametric kernel estimates of the marginal quantile regression functions…
Estimation of the covariance structure of spatial processes is of fundamental importance in spatial statistics. In the literature, several non-parametric and semi-parametric methods have been developed to estimate the covariance structure based on the spectral representation of covariance functions. However,they either ignore the high frequency properties of the spectral density, which are essential to determine the performance of interpolation procedures such as Kriging, or lack of theoretical justification. We propose a new semi-parametric method to estimate spectral densities of isotropic s…
The prevalence of spatially referenced multivariate data has impelled researchers to develop a procedure for the joint modeling of multiple spatial processes. This ordinarily involves modeling marginal and cross-process dependence for any arbitrary pair of locations using a multivariate spatial covariance function. However, building a flexible multivariate spatial covariance function that is nonnegative definite is challenging. Here, we propose a semiparametric approach for multivariate spatial covariance function estimation with approximate Matérn marginals and highly flexible cross-covarianc…
Inference over tails is performed by applying only the results of extreme value theory. Whilst such theory is well defined and flexible enough in the univariate case, multivariate inferential methods often require the imposition of arbitrary constraints not fully justifed by the underlying theory. In contrast, our approach uses only the constraints imposed by theory. We build on previous, theoretically justified work for marginal exceedances over a high, unknown threshold, by combining it with flexible, semiparametric copulae specifications to investigate extreme dependence. Whilst giving prob…
In real data analysis, the underlying model is usually unknown, modelling strategy plays a key role in the success of data analysis. Stimulated by the idea of model averaging, we propose a novel semiparametric modelling strategy for conditional quantile prediction, without assuming the underlying model is any specific parametric or semiparametric model. Thanks the optimality of the selected weights by cross-validation, the proposed modelling strategy results in a more accurate prediction than that based on some commonly used semiparametric models, such as the varying coefficient models and add…
Institute of Statistics and Data Science, Nanjing Audit University（南京审计大学统计与数据科学学院）
China has made great achievements in electric power industry during the long-term deepening of reform and opening up. However, the complex regional economic, social and natural conditions, electricity resources are not evenly distributed, which accounts for the electricity deficiency in some regions of China. It is desirable to develop a robust electricity forecasting model. Motivated by which, we propose a Panel Semiparametric Quantile Regression Neural Network (PSQRNN) by utilizing the artificial neural network and semiparametric quantile regression. The PSQRNN can explore a potential linear…
We propose a series-based nonparametric specification test for a regression function when data are spatially dependent, the `space' being of a general economic or social nature. Dependence can be parametric, parametric with increasing dimension, semiparametric or any combination thereof, thus covering a vast variety of settings. These include spatial error models of varying types and levels of complexity. Under a new smooth spatial dependence condition, our test statistic is asymptotically standard normal. To prove the latter property, we establish a central limit theorem for quadratic forms i…
Linear quantile regression is a powerful tool to investigate how predictors may affect a response heterogeneously across different quantile levels. Unfortunately, existing approaches find it extremely difficult to adjust for any dependency between observation units, largely because such methods are not based upon a fully generative model of the data. For analyzing spatially indexed data, we address this difficulty by generalizing the joint quantile regression model of Yang and Tokdar (2017) and characterizing spatial dependence via a Gaussian or $t$ copula process on the underlying quantile le…
In this paper, we focus on the variable selection techniques for a class of semiparametric spatial regression models which allow one to study the effects of explanatory variables in the presence of the spatial information. The spatial smoothing problem in the nonparametric part is tackled by means of bivariate splines over triangulation, which is able to deal efficiently with data distributed over irregularly shaped regions. In addition, we develop a unified procedure for variable selection to identify significant covariates under a double penalization framework, and we show that the penalized…
Spatial point pattern data are routinely encountered. A flexible regression model for the underlying intensity is essential to characterizing the spatial point pattern and understanding the impacts of potential risk factors on such pattern. We propose a Bayesian semiparametric regression model where the observed spatial points follow a spatial Poisson process with an intensity function which adjusts a nonparametric baseline intensity with multiplicative covariate effects. The baseline intensity is piecewise constant, approached with a powered Chinese restaurant process prior which prevents an …
To better understand the spatial structure of large panels of economic and financial time series and provide a guideline for constructing semiparametric models, this paper first considers estimating a large spatial covariance matrix of the generalized $m$-dependent and $β$-mixing time series (with $J$ variables and $T$ observations) by hard thresholding regularization as long as ${\log J \, \cx^*(\ct)}/{T} = \Co(1)$ (the former scheme with some time dependence measure $\cx^*(\ct)$) or $\log J /{T} = \Co(1)$ (the latter scheme with some upper bounded mixing coefficient). We quantify the interpl…
Discrete data are abundant and often arise as counts or rounded data. These data commonly exhibit complex distributional features such as zero-inflation, over-/under-dispersion, boundedness, and heaping, which render many parametric models inadequate. Yet even for parametric regression models, approximations such as MCMC typically are needed for posterior inference. This paper introduces a Bayesian modeling and algorithmic framework that enables semiparametric regression analysis for discrete data with Monte Carlo (not MCMC) sampling. The proposed approach pairs a nonparametric marginal model …
In this work, we estimate extreme sea surface temperature (SST) hotspots, i.e., high threshold exceedance regions, for the Red Sea, a vital region of high biodiversity. We analyze high-resolution satellite-derived SST data comprising daily measurements at 16703 grid cells across the Red Sea over the period 1985-2015. We propose a semiparametric Bayesian spatial mixed-effects linear model with a flexible mean structure to capture spatially-varying trend and seasonality, while the residual spatial variability is modeled through a Dirichlet process mixture (DPM) of low-rank spatial Student-$t$ pr…
We introduce a general hierarchical Bayesian framework that incorporates a flexible nonparametric data model specification through the use of empirical likelihood methodology, which we term semiparametric hierarchical empirical likelihood (SHEL) models. Although general dependence structures can be readily accommodated, we focus on spatial modeling, a relatively underdeveloped area in the empirical likelihood literature. Importantly, the models we develop naturally accommodate spatial association on irregular lattices and irregularly spaced point-referenced data. We illustrate our proposed fra…
Distributed Prediction under Heterogeneity with Unidentifiable Parameter
Center for Applied Statistics, School of Statistics, Renmin University of China（中国人民大学统计学院应用统计中心）；Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing（北京未来区块链与隐私计算高级创新中心）
Testing for point-impact effects in a spatial semi-functional linear regression model
Laboratoire de Probabilités, Statistique et Informatique（概率、统计与信息实验室）；Unité de Recherche en Mathématiques et Informatique（数学与信息技术研究单位）；Université des Sciences et Techniques de Masuku（马苏库科学技术大学）
Diffusion tensor imaging (DTI) is a popular magnetic resonance imaging technique used to characterize microstructural changes in the brain. DTI studies quantify the diffusion of water molecules in a voxel using an estimated 3x3 symmetric positive definite diffusion tensor matrix. Statistical analysis of DTI data is challenging because the data are positive definite matrices. Matrix-variate information is often summarized by a univariate quantity, such as the fractional anisotropy (FA), leading to a loss of information. Furthermore, DTI analyses often ignore the spatial association of neighbori…
A comprehensive, unified approach to modeling arbitrarily censored spatial survival data is presented for the three most commonly-used semiparametric models: proportional hazards, proportional odds, and accelerated failure time. Unlike many other approaches, all manner of censored survival times are simultaneously accommodated including uncensored, interval censored, current-status, left and right censored, and mixtures of these. Left-truncated data are also accommodated leading to models for time-dependent covariates. Both georeferenced (location exactly observed) and areally observed (locati…
There has been substantial recent work on methods for estimating the slope function in linear regression for functional data analysis. However, as in the case of more conventional finite-dimensional regression, much of the practical interest in the slope centers on its application for the purpose of prediction, rather than on its significance in its own right. We show that the problems of slope-function estimation, and of prediction from an estimator of the slope function, have very different characteristics. While the former is intrinsically nonparametric, the latter can be either nonparametr…
The cause of failure in cohort studies that involve competing risks is frequently incompletely observed. To address this, several methods have been proposed for the semiparametric proportional cause-specific hazards model under a missing at random assumption. However, these proposals provide inference for the regression coefficients only, and do not consider the infinite dimensional parameters, such as the covariate-specific cumulative incidence function. Nevertheless, the latter quantity is essential for risk prediction in modern medicine. In this paper we propose a unified framework for infe…
We consider a spatial functional linear regression, where a scalar response is related to a square integrable spatial functional process. We use a smoothing spline estimator for the functional slope parameter and establish a finite sample bound for variance of this estimator under mixing spatial dependence. Then, we give a bound of the prediction error. Finally, we illustrate our results by simulations
In this paper, we present a statistical framework for modeling conditional quantiles of spatial processes assumed to be strongly mixing in space. We establish the $L_1$ consistency and the asymptotic normality of the kernel conditional quantile estimator in the case of random fields. We also define a nonparametric spatial predictor and illustrate the methodology used with some simulations.
We tackle estimation and prediction at non-visted sites in a spatial semi-functional linear regression model with derivatives that combines a functional linear model with a nonparametric regression one. The parametric part is estimated by a method of moments and the other one by a local linear estimator. We establish the convergence rate of the resulting estimators and predictor. A simulation study and an application to ozone pollution prediction at non-visted sites are proposed to illustrate our results.
Readily available proxies for time of disease onset such as time of the first diagnostic code can lead to substantial risk prediction error if performing analyses based on poor proxies. Due to the lack of detailed documentation and labor intensiveness of manual annotation, it is often only feasible to ascertain for a small subset the current status of the disease by a follow up time rather than the exact time. In this paper, we aim to develop risk prediction models for the onset time efficiently leveraging both a small number of labels on current status and a large number of unlabeled observat…
Deep learning has enjoyed tremendous success in a variety of applications but its application to quantile regressions remains scarce. A major advantage of the deep learning approach is its flexibility to model complex data in a more parsimonious way than nonparametric smoothing methods. However, while deep learning brought breakthroughs in prediction, it often lacks interpretability due to the black-box nature of multilayer structure with millions of parameters, hence it is not well suited for statistical inference. In this paper, we leverage the advantages of deep learning to apply it to quan…
Quantile regression, that is the prediction of conditional quantiles, has steadily gained importance in statistical modeling and financial applications. The authors introduce a new semiparametric quantile regression method based on sequentially fitting a likelihood optimal D-vine copula to given data resulting in highly flexible models with easily extractable conditional quantiles. As a subclass of regular vine copulas, D-vines enable the modeling of multivariate copulas in terms of bivariate building blocks, a so-called pair-copula construction (PCC). The proposed algorithm works fast and acc…
Large wildfires pose a major environmental concern, and precise maps of fire risk can improve disaster relief planning. Fosberg Fire Weather Index (FFWI) is often used to measure wildfire risk; FFWI exhibits non-Gaussian marginal distributions as well as strong spatiotemporal extremal dependence and thus, modeling FFWI using geostatistical models like Gaussian processes is questionable. Extreme value theory (EVT)-driven models like max-stable processes are theoretically appealing but are computationally demanding and applicable only for threshold exceedances or block maxima. Disaster managemen…
Spatial extent inference (SEI) is widely used across neuroimaging modalities to study brain-phenotype associations that inform our understanding of disease. Recent studies have shown that Gaussian random field (GRF) based tools can have inflated family-wise error rates (FWERs). This has led to fervent discussion as to which preprocessing steps are necessary to control the FWER using GRF-based SEI. The failure of GRF-based methods is due to unrealistic assumptions about the covariance function of the imaging data. The permutation procedure is the most robust SEI tool because it estimates the co…
Debiased Machine Learning for Conformal Prediction of Counterfactual Outcomes Under Runtime Confounding
Department of Biostatistics, Harvard T.H. Chan School of Public Health（哈佛大学陈曾熙公共卫生学院生物统计系）；Department of Biostatistics and Informatics, Colorado School of Public Health（科罗拉多公共卫生学院生物统计与信息学系）；Department of Data Science, Dana Farber Cancer Institute（丹娜-法伯癌症研究所数据科学系）
Conformal prediction has received tremendous attention in recent years and has offered new solutions to problems in missing data and causal inference; yet these advances have not leveraged modern semiparametric efficiency theory for more robust and efficient uncertainty quantification. In this paper, we consider the problem of obtaining distribution-free prediction regions accounting for a shift in the distribution of the covariates between the training and test data. Under an explainable covariate shift assumption analogous to the standard missing at random assumption, we propose three varian…
We propose a multiplicative semiparametric model for the intensity function of replicated point processes. Two examples of applications are given: a temporal one, about the dynamics of Internet auctions, and a spatial one, about the spatial distribution of street robberies in Chicago.
