Latent variable (LV) models are widely used in psychological research to investigate relationships among unobservable constructs. When one-stage estimation of the overall LV model is challenging, two-stage factor score regression (FSR) serves as a convenient alternative: the measurement model is fitted to obtain factor scores in the first stage, which are then used to fit the structural model in the subsequent stage. However, naive application of FSR is known to yield biased estimates of structural parameters. In this paper, we develop a generic bias-correction framework for two-stage estimation of parametric statistical models and tailor it specifically to FSR. Unlike existing bias-corrected FSR solutions, the proposed method applies to a broader class of LV models and does not require computing specific types of factor scores. We establish the root-n consistency of the proposed bias-corrected two-stage estimator under mild regularity conditions. To ensure broad applicability and minimize reliance on complex analytical derivations, we introduce a stochastic approximation algorithm for point estimation and a Monte Carlo-based procedure for variance estimation. In a sequence of Monte Carlo experiments, we demonstrate that the bias-corrected FSR estimator performs comparably to the ``gold standard'' one-stage maximum likelihood estimator. These results suggest that our approach offers a straightforward yet effective alternative for estimating LV models.
Quantile regression has demonstrated promising utility in longitudinal data analysis. Existing work is primarily focused on modeling cross-sectional outcomes, while outcome trajectories often carry more substantive information in practice. In this work, we develop a trajectory quantile regression framework that is designed to robustly and flexibly investigate how latent individual trajectory features are related to observed subject characteristics. The proposed models are built under modeling with usual parametric assumptions lifted or relaxed. We derive our estimation procedure by novelly tra…
Longitudinal data are characterized by the dependence between observations coming from the same individual. In a regression perspective, such a dependence can be usefully ascribed to unobserved features (covariates) specific to each individual. On these grounds, random parameter models with time-constant or time-varying structure are well established in the generalized linear model context. In the quantile regression framework, specifications based on random parameters have only recently known a flowering interest. We start from the recent proposal by Farcomeni (2012) on longitudinal quantile …
We propose a goodness-of-fit test for the distribution of errors from a multivariate indirect regression model. The test statistic is based on the Khmaladze transformation of the empirical process of standardized residuals. This goodness-of-fit test is consistent at the root-n rate of convergence, and the test can maintain power against local alternatives converging to the null at a root-n rate.
Regression models, in which the observed features $X \in \R^p$ and the response $Y \in \R$ depend, jointly, on a lower dimensional, unobserved, latent vector $Z \in \R^K$, with $K< p$, are popular in a large array of applications, and mainly used for predicting a response from correlated features. In contrast, methodology and theory for inference on the regression coefficient $β$ relating $Y$ to $Z$ are scarce, since typically the un-observable factor $Z$ is hard to interpret. Furthermore, the determination of the asymptotic variance of an estimator of $β$ is a long-standing problem, with solu…
Data collected from wearable devices and smartphones can shed light on an individual's pattern of behavioral and circadian routine. Phone use can be modeled as alternating event process, between the state of active use and the state of being idle. Markov chains and alternating recurrent event models are commonly used to model state transitions in cases such as these, and the incorporation of random effects can be used to introduce diurnal effects. While state labels can be derived prior to modeling dynamics, this approach omits informative regression covariates that can influence state members…
Popularity Regression-Based Latent Space Model for Large-Scale Sparse Network Analysis
Guanghua School of Management, Peking University（北京大学光华管理学院）；Center for Applied Statistics and School of Statistics, Renmin University of China（中国人民大学统计学院与应用统计中心）；School of Statistics and Mathematics, Central University of Finance and Economics（中央财经大学统计与数学学院）
This paper considers estimating functional-coefficient models in panel quantile regression with individual effects, allowing the cross-sectional and temporal dependence for large panel observations. A latent group structure is imposed on the heterogenous quantile regression models so that the number of nonparametric functional coefficients to be estimated can be reduced considerably. With the preliminary local linear quantile estimates of the subject-specific functional coefficients, a classic agglomerative clustering algorithm is used to estimate the unknown group structure and an easy-to-imp…
This paper develops the asymptotic theory for parametric and nonparametric regression models when the errors have a fractional local to unity root (FLUR) model structure. FLUR models are stationary time series with semi-long range dependence property in the sense that their covariance function resembles that of a long memory model for moderate lags but eventually diminishes exponentially fast according to the presence of a decay factor governed by a noncentrality parameter. When this parameter is sample size dependent, the asymptotic normality for these regression models admit a wide range of …
We propose a generalization of the linear panel quantile regression model to accommodate both \textit{sparse} and \textit{dense} parts: sparse means while the number of covariates available is large, potentially only a much smaller number of them have a nonzero impact on each conditional quantile of the response variable; while the dense part is represent by a low-rank matrix that can be approximated by latent factors and their loadings. Such a structure poses problems for traditional sparse estimators, such as the $\ell_1$-penalised Quantile Regression, and for traditional latent factor estim…
Seemingly unrelated regressions are statistical regression models based on the Gaussian distribution. They are popular in econometrics but also arise in graphical modeling of multivariate dependencies. In maximum likelihood estimation, the parameters of the model are estimated by maximizing the likelihood function, which maps the parameters to the likelihood of observing the given data. By transforming this optimization problem into a polynomial optimization problem, it was recently shown that the likelihood function of a simple bivariate seemingly unrelated regressions model may have several …
We study tail risk dynamics in high-frequency financial markets and their connection with trading activity and market uncertainty. We introduce a dynamic extreme value regression model accommodating both stationary and local unit-root predictors to appropriately capture the time-varying behaviour of the distribution of high-frequency extreme losses. To characterize trading activity and market uncertainty, we consider several volatility and liquidity predictors, and propose a two-step adaptive $L_1$-regularized maximum likelihood estimator to select the most appropriate ones. We establish the o…
Using latent representations to link disjoint longitudinal data for mixed-effects regression
Institute of Medical Biometry and Statistics (IMBI), Medical Faculty and Medical Center – University of Freiburg, Germany（医学生物统计学研究所（IMBI）、医学 faculty 和医疗中心——弗赖堡大学，德国）；Freiburg Center for Data Analysis, Modeling and AI – University of Freiburg, Germany（弗赖堡数据分析、建模和人工智能中心——弗赖堡大学，德国）；Department of Mathematical Stochastics, Mathematical Institute – University of Freiburg, Germany（数学随机过程系、数学研究所——弗赖堡大学，德国）；Department of Neuropediatrics and Muscle Disorders, Medical Faculty and Medical Center – University of Freiburg, Germany（神经发育学与肌肉疾病系、医学 faculty 和医疗中心——弗赖堡大学，德国）；Bonn Center for Mathematical Life Sciences – University of Bonn, Germany（波恩数学生命科学中心——波恩大学，德国）；Life and Medical Sciences (LIMES) Institute – University of Bonn, Germany（生命科学与医学科学（LIMES）研究所——波恩大学，德国）；CIBSS, Centre for Integrative Biological Signalling Studies – University of Freiburg, Germany（整合生物信号研究中心（CIBSS）——弗赖堡大学，德国）
We introduce a random partition model for Bayesian nonparametric regression. The model is based on infinitely-many disjoint regions of the range of a latent covariate-dependent Gaussian process. Given a realization of the process, the cluster of dependent variable responses that share a common region are assumed to arise from the same distribution. Also, the latent Gaussian process prior allows for the random partitions (i.e., clusters of the observations) to exhibit dependencies among one another. The model is illustrated through the analysis of a real data set arising from education, and thr…
Recently, graph (network) data is an emerging research area in artificial intelligence, machine learning and statistics. In this work, we are interested in whether node's labels (people's responses) are affected by their neighbor's features (friends' characteristics). We propose a novel latent logistic regression model to describe the network dependence with binary responses. The key advantage of our proposed model is that a latent binary indicator is introduced to indicate whether a node is susceptible to the influence of its neighbour. A score-type test is proposed to diagnose the existence …
The study of network data in the social and health sciences frequently concentrates on two distinct tasks (1) detecting community structures among nodes and (2) associating covariate information to edge formation. In much of this data, it is likely that the effects of covariates on edge formation differ between communities (e.g. age might play a different role in friendship formation in communities across a city). In this work, we introduce a latent space network model where coefficients associated with certain covariates can depend on latent community membership of the nodes. We show that ign…
Flexibly modeling how an entire density changes with covariates is an important but challenging generalization of mean and quantile regression. While existing methods for density regression primarily consist of covariate-dependent discrete mixture models, we consider a continuous latent variable model in general covariate spaces, which we call DR-BART. The prior mapping the latent variable to the observed data is constructed via a novel application of Bayesian Additive Regression Trees (BART). We prove that the posterior induced by our model concentrates quickly around true generative function…
Most prediction models that are used in medical research fail to accurately predict health outcomes due to methodological limitations. Using routinely collected patient data, we explore the use of a Cox proportional hazard (PH) model within a latent class framework to model survival of patients with chronic heart failure (CHF). We identify subgroups of patients based on their risk with the aid of available covariates. We allow each subgroup to have its own risk model.We choose an optimum number of classes based on the reported Bayesian information criteria (BIC). We assess the discriminative a…
The Gaussian graphical model (GGM) has become a popular tool for analyzing networks of psychological variables. In a recent paper in this journal, Forbes, Wright, Markon, and Krueger (FWMK) voiced the concern that GGMs that are estimated from partial correlations wrongfully remove the variance that is shared by its constituents. If true, this concern has grave consequences for the application of GGMs. Indeed, if partial correlations only capture the unique covariances, then the data that come from a unidimensional latent variable model ULVM should be associated with an empty network (no edges)…
Univariate or multivariate ordinal responses are often assumed to arise from a latent continuous parametric distribution, with covariate effects which enter linearly. We introduce a Bayesian nonparametric modeling approach for univariate and multivariate ordinal regression, which is based on mixture modeling for the joint distribution of latent responses and covariates. The modeling framework enables highly flexible inference for ordinal regression relationships, avoiding assumptions of linearity or additivity in the covariate effects. In standard parametric ordinal regression models, computat…
Bivariate observations of binary and ordinal data arise frequently and require a bivariate modelling approach in cases where one is interested in aspects of the marginal distributions as separate outcomes along with the association between the two. We consider methods for constructing such bivariate models with logistic marginals and propose a model based on the Ali-Mikhail-Haq bivariate logistic distribution. We motivate the model as an extension of that based on the Gumbel type 2 distribution as considered by other authors and as a bivariate extension of the logistic distribution which prese…
Now over 20 years old, functional MRI (fMRI) has a large and growing literature that is best synthesised with meta-analytic tools. As most authors do not share image data, only the peak activation coordinates (foci) reported in the paper are available for Coordinate-based Meta-analysis (CBMA). Neuroimaging meta-analysis is used to 1) identify areas of consistent activation; and 2) build a predictive model of task type or cognitive process for new studies (reverse inference). To simultaneously address these aims, we propose a Bayesian point process hierarchical model for CBMA. We model the foci…
We propose the Factor Augmented sparse linear Regression Model (FARM) that not only encompasses both the latent factor regression and sparse linear regression as special cases but also bridges dimension reduction and sparse regression together. We provide theoretical guarantees for the estimation of our model under the existence of sub-Gaussian and heavy-tailed noises (with bounded (1+x)-th moment, for all x>0), respectively. In addition, the existing works on supervised learning often assume the latent factor regression or the sparse linear regression is the true underlying model without just…
Standard linear modeling approaches make potentially simplistic assumptions regarding the structure of categorical effects that may obfuscate more complex relationships governing data. For example, recent work focused on the two-way unreplicated layout has shown that hidden groupings among the levels of one categorical predictor frequently interact with the ungrouped factor. We extend the notion of a "latent grouping factor" to linear models in general. The proposed work allows researchers to determine whether an apparent grouping of the levels of a categorical predictor reveals a plausible hi…
We introduce Bayesian additive regression trees (BART) for log-linear models including multinomial logistic regression and count regression with zero-inflation and overdispersion. BART has been applied to nonparametric mean regression and binary classification problems in a range of settings. However, existing applications of BART have been limited to models for Gaussian "data", either observed or latent. This is primarily because efficient MCMC algorithms are available for Gaussian likelihoods. But while many useful models are naturally cast in terms of latent Gaussian variables, many others …
Institut National de Recherche sur les Transports et leur Sécurité（法国国家交通、运输与安全研究所）；Université de Technologie de Compiègne（贡比涅技术大学）
A non linear regression approach which consists of a specific regression model incorporating a latent process, allowing various polynomial regression models to be activated preferentially and smoothly, is introduced in this paper. The model parameters are estimated by maximum likelihood performed via a dedicated expecation-maximization (EM) algorithm. An experimental study using simulated and real data sets reveals good performances of the proposed approach.
We develop Bayesian models for density regression with emphasis on discrete outcomes. The problem of density regression is approached by considering methods for multivariate density estimation of mixed scale variables, and obtaining conditional densities from the multivariate ones. The approach to multivariate mixed scale outcome density estimation that we describe represents discrete variables, either responses or covariates, as discretised versions of continuous latent variables. We present and compare several models for obtaining these thresholds in the challenging context of count data ana…
Regularized regression models are well studied and, under appropriate conditions, offer fast and statistically interpretable results. However, large data in many applications are heterogeneous in the sense of harboring distributional differences between latent groups. Then, the assumption that the conditional distribution of response Y given features X is the same for all samples may not hold. Furthermore, in scientific applications, the covariance structure of the features may contain important signals and its learning is also affected by latent group structure. We propose a class of mixture …
This paper studies the confounding effects from the unmeasured confounders and the imbalance of observed confounders in IV regression and aims at unbiased causal effect estimation. Recently, nonlinear IV estimators were proposed to allow for nonlinear model in both stages. However, the observed confounders may be imbalanced in stage 2, which could still lead to biased treatment effect estimation in certain cases. To this end, we propose a Confounder Balanced IV Regression (CB-IV) algorithm to jointly remove the bias from the unmeasured confounders and the imbalance of observed confounders. The…
In this paper, we study a functional regression setting where the random response curve is unobserved, and only its dichotomized version observed at a sequence of correlated binary data is available. We propose a practical computational framework for maximum likelihood analysis via the parameter expansion technique. Compared to existing methods, our proposal relies on the use of a complete data likelihood, with the advantage of being able to handle non-equally spaced and missing observations effectively. The proposed method is used in the Function-on-Scalar regression setting, with the latent …
Products manufactured from the same batch or utilized in the same region often exhibit correlated lifetime observations due to the latent heterogeneity caused by the influence of shared but unobserved covariates. The unavailable group-shared covariates involve multiple different types (e.g., discrete, continuous, or mixed-type) and induce different structures of indispensable group-shared latent heterogeneity. Without carefully capturing such latent heterogeneity, the lifetime modeling accuracy will be significantly undermined. In this work, we propose a generic Bayesian lifetime modeling appr…
