Panel count data describes aggregated counts of recurrent events observed at discrete time points. To understand dynamics of health behaviors, the field of quantitative behavioral research has evolved to increasingly rely upon panel count data collected via multiple self reports, for example, about frequencies of smoking using in-the-moment surveys on mobile devices. However, missing reports are common and present a major barrier to downstream statistical learning. As a first step, under a missing completely at random assumption (MCAR), we propose a simple yet widely applicable functional EM algorithm to estimate the counting process mean function, which is of central interest to behavioral scientists. The proposed approach wraps several popular panel count inference methods, seamlessly deals with incomplete counts and is robust to misspecification of the Poisson process assumption. Theoretical analysis of the proposed algorithm provides finite-sample guarantees by expanding parametric EM theory to our general non-parametric setting. We illustrate the utility of the proposed algorithm through numerical experiments and an analysis of smoking cessation data. We also discuss useful extensions to address deviations from the MCAR assumption and covariate effects.
Panel count data is common when the study subjects are exposed to recurrent events, observed only at discrete time points. In this article, we consider the regression analysis of panel count data with multiple modes of recurrence. We propose a proportional mean model to estimate the effect of covariates on the underlying counting process due to different modes of recurrence. The simultaneous estimation of baseline cumulative mean functions and regression parameters of $(k>1)$ recurrence modes are studied in detail. Asymptotic properties of the proposed estimators are also established. A Monte …
We develop a Bayesian non-parametric quantile panel regression model. Within each quantile, the response function is a convex combination of a linear model and a non-linear function, which we approximate using Bayesian Additive Regression Trees (BART). Cross-sectional information at the pth quantile is captured through a conditionally heteroscedastic latent factor. The non-parametric feature of our model enhances flexibility, while the panel feature, by exploiting cross-country information, increases the number of observations in the tails. We develop Bayesian Markov chain Monte Carlo (MCMC) m…
In this paper, we present a method of maximum a posteriori estimation of parameters in dynamic factor models with incomplete data. We extend maximum likelihood expectation maximization iterations by Bańbura & Modugno (2014) to penalized counterparts by applying parameter shrinkage in a Minnesota prior style fashion, also considering factors loading onto variables dynamically. A heuristic and adapting shrinkage scheme is considered. The algorithm is applicable to any arbitrary pattern of missing data, including different publication dates, sample lengths and frequencies. The method is evaluated…
Panel count data refer to the data arising from studies concerning recurrent events where study subjects are observed only at distinct time points. If these study subjects are exposed to recurrent events of several types, we obtain panel count data with multiple modes of recurrence. In the present paper, we propose a nonparametric test for comparing cause specific rate functions of panel count data with more than one mode of recurrence. The test can also be employed to assess whether the competing modes of recurrence are affecting the recurrence times identically. We carry out simulation studi…
Panel count data arise from longitudinal studies on recurrent events where each subject is observed only at discrete time points. If recurrent events of several types are possible, we obtain panel count data with multiple modes of recurrence. Such data is commonly encountered in medical studies, reliability experiments as well as in sociological studies. In this article, we present cause specific rate functions for the analysis of panel count data with multiple modes of recurrence and develop nonparametric estimation procedures for the same. We derive empirical estimators for the cause specifi…
In this paper, the panel count data analysis for recurrent events is considered. Such analysis is useful for studying tumor or infection recurrences in both clinical trial and observational studies. A bivariate Gaussian Cox process model is proposed to jointly model the observation process and the recurrent event process. Bayesian nonparametric inference is proposed for simultaneously estimating regression parameters, bivariate frailty effects and baseline intensity functions. Inference is done through Markov chain Monte Carlo, with fully developed computational techniques. Predictive inferenc…
This paper proposes a new approach for testing Granger non-causality on panel data. Instead of aggregating panel member statistics, we aggregate their corresponding p-values and show that the resulting p-value approximately bounds the type I error by the chosen significance level even if the panel members are dependent. We compare our approach against the most widely used Granger causality algorithm on panel data and show that our approach yields lower FDR at the same power for large sample sizes and panels with cross-sectional dependencies. Finally, we examine COVID-19 data about confirmed ca…
The spatial panel regression model has shown great success in modelling econometric and other types of data that are observed both spatially and temporally with associated predictor variables. However, model checking via testing for spatial correlations in spatial-temporal residuals is still lacking. We propose a general methodology for fast permutation testing of local and global indicators of spatial association. This methodology extends past statistics for univariate spatial data that can be written as a gamma index for matrix similarity to the multivariate and panel data settings. This inc…
Frequently, a set of objects has to be evaluated by a panel of assessors, but not every object is assessed by every assessor. A problem facing such panels is how to take into account different standards amongst panel members and varying levels of confidence in their scores. Here, a mathematically-based algorithm is developed to calibrate the scores of such assessors, addressing both of these issues. The algorithm is based on the connectivity of the graph of assessors and objects evaluated, incorporating declared confidences as weights on its edges. If the graph is sufficiently well connected, …
For a panel model considered by Abadie et al. (2010), the counterfactual outcomes constructed by Abadie et al., Hsiao et al. (2012), and Doudchenko and Imbens (2017) may all be confounded by uncontrolled heterogenous trends. Based on exact-matching on the trend predictors, I propose new methods of estimating the model-specific treatment effects, which are free from heterogenous trends. When applied to Abadie et al.'s (2010) model and data, the new estimators suggest considerably smaller effects of California's tobacco control program.
A useful technique for analyzing incomplete tables is to model the missing data mechanisms of the variables using log-linear models. In this paper, we use log-linear parametrization and propose estimation methods for arbitrary three-way and $n$-dimensional incomplete tables. All possible cases in which data on one or more of the variables may be missing are considered. We provide simple closed form estimates of expected cell counts and parameters for the various missing data models. We also obtain explicit boundary estimates under nonignorable nonresponse models. Finally, a real-life dataset i…
Factor structures or interactive effects are convenient devices to incorporate latent variables in panel data models. We consider fixed effect estimation of nonlinear panel single-index models with factor structures in the unobservables, which include logit, probit, ordered probit and Poisson specifications. We establish that fixed effect estimators of model parameters and average partial effects have normal distributions when the two dimensions of the panel grow large, but might suffer of incidental parameter bias. We show how models with factor structures can also be applied to capture impor…
Causal Data Fusion for Panel Data without a Pre-Intervention Period
Faculty of Computing and Data Sciences, Boston University（波士顿大学计算与数据科学学院）；Division of Infectious Diseases, Boston Children’s Hospital & Harvard Medical School（波士顿儿童医院感染病科及哈佛医学院）；Department of Mathematics and Statistics, Boston University（波士顿大学数学与统计系）
The presence of outlying observations may adversely affect statistical testing procedures that result in unstable test statistics and unreliable inferences depending on the distortion in parameter estimates. In spite of the fact that the adverse effects of outliers in panel data models, there are only a few robust testing procedures available for model specification. In this paper, a new weighted likelihood based robust specification test is proposed to determine the appropriate approach in panel data including individual-specific components. The proposed test has been shown to have the same a…
Accurate estimation for extent of cross{sectional dependence in large panel data analysis is paramount to further statistical analysis on the data under study. Grouping more data with weak relations (cross{sectional dependence) together often results in less efficient dimension reduction and worse forecasting. This paper describes cross-sectional dependence among a large number of objects (time series) via a factor model and parameterizes its extent in terms of strength of factor loadings. A new joint estimation method, benefiting from unique feature of dimension reduction for high dimensional…
We develop new higher-order asymptotic techniques for the Gaussian maximum likelihood estimator in a spatial panel data model, with fixed effects, time-varying covariates, and spatially correlated errors. Our saddlepoint density and tail area approximation feature relative error of order $O(1/(n(T-1)))$ with $n$ being the cross-sectional dimension and $T$ the time-series dimension. The main theoretical tool is the tilted-Edgeworth technique in a non-identically distributed setting. The density approximation is always non-negative, does not need resampling, and is accurate in the tails. Monte C…
The analysis of count data is commonly done using Poisson models. Negative binomial models are a straightforward and readily motivated generalization for the case of overdispersed data, i.e., when the observed variance is greater than expected under a Poissonian model. Rate and overdispersion parameters then need to be considered jointly, which in general is not trivial. Here we are concerned with evidence synthesis in the case where the reporting of data is rather heterogeneous, i.e., events are reported either in terms of mean event counts, the proportion of event-free patients, or rate esti…
This study proposes a new Bayesian approach to infer binary treatment effects. The approach treats counterfactual untreated outcomes as missing observations and infers them by completing a matrix composed of realized and potential untreated outcomes using a data augmentation technique. We also develop a tailored prior that helps in the identification of parameters and induces the matrix of untreated outcomes to be approximately low rank. Posterior draws are simulated using a Markov Chain Monte Carlo sampler. While the proposed approach is similar to synthetic control methods and other related …
In order to evaluate the impact of a policy intervention on a group of units over time, it is important to correctly estimate the average treatment effect (ATE) measure. Due to lack of robustness of the existing procedures of estimating ATE from panel data, in this paper, we introduce a robust estimator of the ATE and the subsequent inference procedures using the popular approach of minimum density power divergence inference. Asymptotic properties of the proposed ATE estimator are derived and used to construct robust test statistics for testing parametric hypotheses related to the ATE. Besides…
The analysis of incomplete contingency tables is an important problem, which is also of practical interest. In this paper, we consider boundary solutions under nonignorable nonresponse models in two-way incomplete tables with data on both variables missing. We establish a result similar to Park {\it et al.} (2014) on sufficient conditions for the occurrence of boundary solutions. We also provide a new result, which connects the forms of boundary solutions under various parameterizations of the missing data models. This result helps us to obtain the exact form of boundary solutions in the above…
