In light of recent work studying massive functional/longitudinal data, such as the resulting data from the COVID-19 pandemic, we propose a novel functional/longitudinal data model which is a combination of the popular varying coefficient (VC) model and additive model. We call it Semi-VCAM in which the response could be a functional/longitudinal variable, and the explanatory variables could be a mixture of functional/longitudinal and scalar variables. Notably some of the scalar variables could be categorical variables as well. The Semi-VCAM simultaneously allows for both substantial flexibility and the maintaining of one-dimensional rates of convergence. A local linear smoothing with the aid of an initial B spline series approximation is developed to estimate the unknown functional effects in the model. To avoid the subjective choice between the sparse and dense cases of the data, we establish the asymptotic theories of the resultant Pilot Estimation Based Local Linear Estimators (PEBLLE) on a unified framework of sparse, dense and ultra-dense cases of the data. Moreover, we construct unified consistent tests to justify whether a parsimony submodel is sufficient or not. These test methods also avoid the subjective choice between the sparse, dense and ultra dense cases of the data. Extensive Monte Carlo simulation studies investigating the finite sample performance of the proposed methodologies confirm our asymptotic results. We further illustrate our methodologies via analyzing the COVID-19 data from China and the CD4 data.
Modern longitudinal data, for example from wearable devices, measures biological signals on a fixed set of participants at a diverging number of time points. Traditional statistical methods are not equipped to handle the computational burden of repeatedly analyzing the cumulatively growing dataset each time new data is collected. We propose a new estimation and inference framework for dynamic updating of point estimates and their standard errors across serially collected dependent datasets. The key technique is a decomposition of the extended score function of the quadratic inference function …
A new single-index model that reflects the time-dynamic effects of the single index is proposed for longitudinal and functional response data, possibly measured with errors, for both longitudinal and time-invariant covariates. With appropriate initial estimates of the parametric index, the proposed estimator is shown to be $\sqrt{n}$-consistent and asymptotically normally distributed. We also address the nonparametric estimation of regression functions and provide estimates with optimal convergence rates. One advantage of the new approach is that the same bandwidth is used to estimate both the…
The distribution of the grid points at which a response function is observed in longitudinal or functional data applications is often informative and not independent of the response process. In this paper we introduce a covariation model to estimate and make inferences about this interrelation, by treating the data as replicated realizations of a marked point process. We derive maximum likelihood estimators, the asymptotic distribution of the estimators, and study the estimators' behavior by simulation. We apply the model to an online auction data set and show that there is a strong correlatio…
In this paper, we consider the problem of estimating the eigenvalues and eigenfunctions of the covariance kernel (i.e., the functional principal components) from sparse and irregularly observed longitudinal data. We approach this problem through a maximum likelihood method assuming that the covariance kernel is smooth and finite dimensional. We exploit the smoothness of the eigenfunctions to reduce dimensionality by restricting them to a lower dimensional space of smooth functions. The estimation scheme is developed based on a Newton-Raphson procedure using the fact that the basis coefficients…
Sliced inverse regression (Duan and Li [Ann. Statist. 19 (1991) 505-530], Li [J. Amer. Statist. Assoc. 86 (1991) 316-342]) is an appealing dimension reduction method for regression models with multivariate covariates. It has been extended by Ferré and Yao [Statistics 37 (2003) 475-488, Statist. Sinica 15 (2005) 665-683] and Hsing and Ren [Ann. Statist. 37 (2009) 726-755] to functional covariates where the whole trajectories of random functional covariates are completely observed. The focus of this paper is to develop sliced inverse regression for intermittently and sparsely measured longitudin…
Ron et al (1998) introduced a rich family of models for discrete longitudinal data, called acyclic probabilistic finite automata. These may be described as context-specific graphical models, since they are represented as directed multigraphs that embody context-specific conditional independence relations. Here we develop the methodology from a statistical modelling perspective. We show how likelihood ratio tests may be constructed using standard contingency table methods, and indicate how these may be used in model selection. We also show that the models generalize certain subclasses of conven…
We consider the marginal models of Liang and Zeger [Biometrika 73 (1986) 13-22] for the analysis of longitudinal data and we develop a theory of statistical inference for such models. We prove the existence, weak consistency and asymptotic normality of a sequence of estimators defined as roots of pseudo-likelihood equations.
Functional linear regression analysis aims to model regression relations which include a functional predictor. The analog of the regression parameter vector or matrix in conventional multivariate or multiple-response linear regression models is a regression parameter function in one or two arguments. If, in addition, one has scalar predictors, as is often the case in applications to longitudinal studies, the question arises how to incorporate these into a functional regression model. We study a varying-coefficient approach where the scalar covariates are modeled as additional arguments of the …
There is a difficulty in finding an estimate of variance of the profile likelihood estimator in the joint model of longitudinal and survival data. We solve the difficulty by introducing the ``statistical generalized derivative''. The derivative is used to show the asymptotic normality of the estimator without assuming the second derivative of the density function in the model exists.
Segmented regression is a standard statistical procedure used to estimate the effect of a policy intervention on time series outcomes. This statistical method assumes the normality of the outcome variable, a large sample size, no autocorrelation in the observations, and a linear trend over time. Also, segmented regression is very sensitive to outliers. In a small sample study, if the outcome variable does not follow a Gaussian distribution, then using segmented regression to estimate the intervention effect leads to incorrect inferences. To address the small sample problem and non-normality in…
Scalable regression calibration approaches to correcting measurement error in multi-level generalized functional linear regression models with heteroscedastic measurement errors
We address the statistical estimation of composite functionals which may be nonlinear in the probability measure. Our study is motivated by the need to estimate coherent measures of risk, which become increasingly popular in finance, insurance, and other areas associated with optimization under uncertainty and risk. We establish central limit formulae for composite risk functionals. Furthermore, we discuss the asymptotic behavior of optimization problems whose objectives are composite risk functionals and we establish a central limit formula of their optimal values when an estimator of the ris…
The study of the dynamic behavior of cross-sectional ranks over time for functional data and the ranks of the observed curves at each time point and their temporal evolution can yield valuable insights into the time dynamics of functional data. This approach is of interest in various application areas. For the analysis of the dynamics of ranks, estimation of the cross-sectional ranks of functional data is a first step. Several statistics of interest for ranked functional data are proposed. To quantify the evolution of ranks over time, a model for rank derivatives is introduced, where rank dyna…
Quantile regression is a powerful statistical methodology that complements the classical linear regression by examining how covariates influence the location, scale, and shape of the entire response distribution and offering a global view of the statistical landscape. In this paper we propose a new quantile regression model for longitudinal data. The proposed approach incorporates the correlation structure between repeated measures to enhance the efficiency of the inference. In order to use the Newton-Raphson iteration method to obtain convergent estimates, the estimating functions are redefin…
In this paper, we propose a novel association measure for longitudinal studies based on the traditional definition of relative risk. In a Markovian fashion, such a proposal takes into account the information content regarding the previous time. We derive its corresponding confidence interval by means of the Delta method having in mind the crude association between factor and event. Also, we study the properties of our uncertainty quantification scheme through an exhaustive simulation study. Our findings show that the coverage probability is quite close to the level of confidence. Finally, our …
Rooted in genetics, human complex diseases are largely influenced by environmental factors. Existing literature has shown the power of integrative gene-environment interaction analysis by considering the joint effect of environmental mixtures on a disease risk. In this work, we propose a functional varying index coefficient model for longitudinal measurements of a phenotypic trait together with multiple environmental variables, and assess how the genetic effects on a longitudinal disease trait are nonlinearly modified by a mixture of environmental influences. We derive an estimation procedure …
This paper develops a two-part finite mixture quantile regression model for semi-continuous longitudinal data. The proposed methodology allows heterogeneity sources that influence the model for the binary response variable, to influence also the distribution of the positive outcomes. As is common in the quantile regression literature, estimation and inference on the model parameters are based on the Asymmetric Laplace distribution. Maximum likelihood estimates are obtained through the EM algorithm without parametric assumptions on the random effects distribution. In addition, a penalized versi…
In many applications, geodesic hierarchical models are adequate for the study of temporal observations. We employ such a model derived for manifold-valued data to Kendall's shape space. In particular, instead of the Sasaki metric, we adapt a functional-based metric, which increases the computational efficiency and does not require the implementation of the curvature tensor. We propose the corresponding variational time discretization of geodesics and employ the approach for longitudinal analysis of 2D rat skulls shapes as well as 3D shapes derived from an imaging study on osteoarthritis. Parti…
We propose a nonparametric method to explicitly model and represent the derivatives of smooth underlying trajectories for longitudinal data. This representation is based on a direct Karhunen--Loève expansion of the unobserved derivatives and leads to the notion of derivative principal component analysis, which complements functional principal component analysis, one of the most popular tools of functional data analysis. The proposed derivative principal component scores can be obtained for irregularly spaced and sparsely observed longitudinal data, as typically encountered in biomedical studie…
In this paper, we consider the exponential functional \(A_{\infty}=\int_0^\infty e^{-ξ_s}ds\) of a L{é}vy process \(ξ_s\) and aim to estimate the characteristics of \(ξ_{s}\) from the distribution of \(A_{\infty}\). We present a new approach, which allows to statistically infer on the L{é}vy triplet of \(ξ_{t}\), and study the theoretical properties of the proposed estimators. The suggested algorithms are illustrated with numerical simulations.
We present a robust regression estimator for longitudinal data, which is especially suited for functional data that has been observed on sparse or irregular time grids. We show by simulation that the proposed estimators possess good outlier-resistance properties compared with the traditional functional least-squares estimator. As an example of application, we study the relationship between levels of oxides of nitrogen and ozone in the city of San Francisco.
A new framework is developed to intrinsically analyze sparsely observed Riemannian functional data. It features four innovative components: a frame-independent covariance function, a smooth vector bundle termed covariance vector bundle, a parallel transport and a smooth bundle metric on the covariance vector bundle. The introduced intrinsic covariance function links estimation of covariance structure to smoothing problems that involve raw covariance observations derived from sparsely observed Riemannian functional data, while the covariance vector bundle provides a rigorous mathematical founda…
We consider analysis of dependent functional data that are correlated because of a longitudinal-based design: each subject is observed at repeated time visits and for each visit we record a functional variable. We propose a novel parsimonious modeling framework for the repeatedly observed functional variables that allows to extract low dimensional features. The proposed methodology accounts for the longitudinal design, is designed for the study of the dynamic behavior of the underlying process, and is computationally fast. Theoretical properties of this framework are studied and numerical inve…
Functional principal component analysis is essential in functional data analysis, but the inferences will become unconvincing when some non-Gaussian characteristics occur, such as heavy tail and skewness. The focus of this paper is to develop a robust functional principal component analysis methodology in dealing with non-Gaussian longitudinal data, for which sparsity and irregularity along with non-negligible measurement errors must be considered. We introduce a Kendall's $τ$ function whose particular properties make it a nice proxy for the covariance function in the eigenequation when handli…
This paper addresses estimation in a longitudinal regression model for association between a scalar outcome and a set of longitudinally-collected functional covariates or predictor curves. The framework consists of estimating a time-varying coefficient function that is modeled as a linear combination of time-invariant functions but having time-varying coefficients. The estimation procedure exploits the equivalence between penalized least squares estimation and a linear mixed model representation. The process is empirically evaluated with several simulations and it is applied to analyze the neu…
We propose novel optimal designs for longitudinal data for the common situation where the resources for longitudinal data collection are limited, by determining the optimal locations in time where measurements should be taken. As for all optimal designs, some prior information is needed to implement the proposed optimal designs. We demonstrate that this prior information may come from a pilot longitudinal study that has irregularly measured and noisy measurements, where for each subject one has available a small random number of repeated measurements that are randomly located on the domain. A …
In many longitudinal settings, time-varying covariates may not be measured at the same time as responses and are often prone to measurement error. Naive last-observation-carried-forward methods incur estimation biases, and existing kernel-based methods suffer from slow convergence rates and large variations. To address these challenges, we propose a new functional calibration approach to efficiently learn longitudinal covariate processes based on sparse functional data with measurement error. Our approach, stemming from functional principal component analysis, calibrates the unobserved synchro…
Modeling non-linear temporal trajectories is of fundamental interest in many application areas, such as in longitudinal microbiome analysis. Many existing methods focus on estimating mean trajectories, but it is also often of value to assess temporal patterns of individual subjects. Sparse principal components analysis (SFPCA) serves as a useful tool for assessing individual variation in non-linear trajectories; however its application to real data often requires careful model selection criteria and diagnostic tools. Here, we propose a Bayesian approach to SFPCA, which allows users to use the …
In this paper we review existing methods for robust functional principal component analysis (FPCA) and propose a new method for FPCA that can be applied to longitudinal data where only a few observations per trajectory are available. This method is robust against the presence of atypical observations, and can also be used to derive a new non-robust FPCA approach for sparsely observed functional data. We use local regression to estimate the values of the covariance function, taking advantage of the fact that for elliptically distributed random vectors the conditional location parameter of some …
Functional forms in joint models for longitudinal and time-to-event data: A practical guide with application and interpretation
Erasmus University Medical Center（伊拉斯姆斯大学医学中心）；European Organisation for Research and Treatment of Cancer（欧洲癌症研究与治疗组织）；University Medical Center Utrecht（乌得勒支大学医学中心）
This paper introduces a new spatial scan statistic designed to adjust cluster detection for longitudinal confounding factors indexed in space. The functional-model-adjusted statistic was developed using generalized functional linear models in which longitudinal confounding factors were considered to be functional covariates. A general framework was developed for application to various probability models. Application to a Poisson model showed that the new method is equivalent to a conventional spatial scan statistic that adjusts the underlying population for covariates. In a simulation study wi…
