We introduce a class of semiparametric time series models by assuming a quasi-likelihood approach driven by a latent factor process. More specifically, given the latent process, we only specify the conditional mean and variance of the time series and enjoy a quasi-likelihood function for estimating parameters related to the mean. This proposed methodology has three remarkable features: (i) no parametric form is assumed for the conditional distribution of the time series given the latent process; (ii) able for modelling non-negative, count, bounded/binary and real-valued time series; (iii) dispersion parameter is not assumed to be known. Further, we obtain explicit expressions for the marginal moments and for the autocorrelation function of the time series process so that a method of moments can be employed for estimating the dispersion parameter and also parameters related to the latent process. Simulated results aiming to check the proposed estimation procedure are presented. Real data analysis on unemployment rate and precipitation time series illustrate the potencial for practice of our methodology.
Time-series data in population health and epidemiology often involve non-Gaussian responses. In this note, we propose a semiparametric generalized linear models framework for time-series data that does not require specification of a working conditional response distribution for the data. Instead, the underlying response distribution is treated as an infinite-dimensional parameter which is estimated simultaneously with the usual finite-dimensional parameters via a maximum empirical likelihood approach. A general consistency result for the resulting estimators is given. Simulations suggest that …
I describe ongoing work developing Bayesian methods for flexible modeling of arrival time series data without binning, aiming to improve detection and measurement of X-ray and gamma-ray pulsars, and of pulses in gamma-ray bursts. The methods use parametric and semiparametric Poisson point process models for the event rate, and by design have close connections to conventional frequentist methods currently used in time-domain astronomy.
Stochastic volatility (SV) models mimic many of the stylized facts attributed to time series of asset returns, while maintaining conceptual simplicity. The commonly made assumption of conditionally normally distributed or Student-t-distributed returns, given the volatility, has however been questioned. In this manuscript, we introduce a novel maximum penalized likelihood approach for estimating the conditional distribution in an SV model in a nonparametric way, thus avoiding any potentially critical assumptions on the shape. The considered framework exploits the strengths both of the powerful …
We formulate nonparametric and semiparametric hypothesis testing of multivariate stationary linear time series in a unified fashion and propose new test statistics based on estimators of the spectral density matrix. The limiting distributions of these test statistics under null hypotheses are always normal distributions, and they can be implemented easily for practical use. If null hypotheses are false, as the sample size goes to infinity, they diverge to infinity and consequently are consistent tests for any alternative. The approach can be applied to various null hypotheses such as the indep…
To better understand the spatial structure of large panels of economic and financial time series and provide a guideline for constructing semiparametric models, this paper first considers estimating a large spatial covariance matrix of the generalized $m$-dependent and $β$-mixing time series (with $J$ variables and $T$ observations) by hard thresholding regularization as long as ${\log J \, \cx^*(\ct)}/{T} = \Co(1)$ (the former scheme with some time dependence measure $\cx^*(\ct)$) or $\log J /{T} = \Co(1)$ (the latter scheme with some upper bounded mixing coefficient). We quantify the interpl…
We study a well-known estimator of the fractal index of a stochastic process. Our framework is very general and encompasses many models of interest; we show how to extend the theory of the estimator to a large class of non-Gaussian processes. Particular focus is on clarity and ease of implementation of the estimator and the associated asymptotic results, making it easy for practitioners to apply the methods. We additionally show how measurement noise in the observations will bias the estimator, potentially resulting in the practitioner erroneously finding evidence of fractal characteristics in…
Semiparametric forecasting and filtering are introduced as a method of addressing model errors arising from unresolved physical phenomena. While traditional parametric models are able to learn high-dimensional systems from small data sets, their rigid parametric structure makes them vulnerable to model error. On the other hand, nonparametric models have a very flexible structure, but they suffer from the curse-of-dimensionality and are not practical for high-dimensional systems. The semiparametric approach loosens the structure of a parametric model by fitting a data-driven nonparametric model…
We develop a Bayesian hierarchical semiparametric model for phenomena related to time series of counts. The main feature of the model is its capability to learn a latent pattern of heterogeneity in the distribution of the process innovation rates, which are softly clustered through time with the help of a Dirichlet process placed at the top of the model hierarchy. The probabilistic forecasting capabilities of the model are put to test in the analysis of crime data in Pittsburgh, with favorable results.
Area Under the Curve (AUC) is arguably the most popular measure of classification accuracy. We use a semiparametric framework to introduce a latent scale-invariant $R^2$, a novel measure of variation explained for an observed binary outcome and an observed continuous predictor, and then directly link the latent $R^2$ to AUC. This enables a mutually consistent simultaneous use of AUC as a measure of classification accuracy and the latent $R^2$ as a scale-invariant measure of explained variation. Specifically, we employ Semiparametric Gaussian Copula (SGC) to model a joint dependence between obs…
In this paper we propose and study a general class of Gaussian Semiparametric Estimators (GSE) of the fractional differencing parameter in the context of long-range dependent multivariate time series. We establish large sample properties of the estimator without assuming Gaussianity. The class of models considered here satisfies simple conditions on the spectral density function, restricted to a small neighborhood of the zero frequency and includes important class of VARFIMA processes. We also present a simulation study to assess the finite sample properties of the proposed estimator based on …
Item response theory (IRT) models typically rely on a normality assumption for subject-specific latent traits, which is often unrealistic in practice. Semiparametric extensions based on Dirichlet process mixtures offer a more flexible representation of the unknown distribution of the latent trait. However, the use of such models in the IRT literature has been extremely limited, in good part because of the lack of comprehensive studies and accessible software tools. This paper provides guidance for practitioners on semiparametric IRT models and their implementation. In particular, we rely on NI…
We consider a time series model involving a fractional stochastic component, whose integration order can lie in the stationary/invertible or nonstationary regions and be unknown, and an additive deterministic component consisting of a generalized polynomial. The model can thus incorporate competing descriptions of trending behavior. The stationary input to the stochastic component has parametric autocorrelation, but innovation with distribution of unknown form. The model is thus semiparametric, and we develop estimates of the parametric component which are asymptotically normal and achieve an …
We introduce a new class of semiparametric latent variable models for long memory discretized event data. The proposed methodology is motivated by a study of bird vocalizations in the Amazon rain forest; the timings of vocalizations exhibit self-similarity and long range dependence ruling out models based on Poisson processes. The proposed class of FRActional Probit (FRAP) models is based on thresholding of a latent process consisting of an additive expansion of a smooth Gaussian process with a fractional Brownian motion. We develop a Bayesian approach to inference using Markov chain Monte Car…
University of Southern California（南加州大学）；IBM T.J. Watson Research Center（IBM T.J. Watson研究中心）
In many applications of time series models, such as climate analysis and social media analysis, we are often interested in extreme events, such as heatwave, wind gust, and burst of topics. These time series data usually exhibit a heavy-tailed distribution rather than a Gaussian distribution. This poses great challenges to existing approaches due to the significantly different assumptions on the data distributions and the lack of sufficient past data on extreme events. In this paper, we propose the Sparse-GEV model, a latent state model based on the theory of extreme value modeling to automatic…
This paper reviews recent developments of robust estimation in linear time series models, with short and long memory correlation structures, in the presence of additive outliers. Based on the manuscripts Fajardo et al. (2009) and Lévy-Leduc et al. (2011a), the emphasis in this paper is given in the following directions; the influence of additive outliers in the estimation of a time series, the asymptotic properties of a robust autocovariance function and a robust semiparametric estimation method of the fractional parameter d in ARFIMA(p, d, q) models. Some simulations are used to support the u…
This paper proposes a flexible Bayesian approach to multiple imputation using conditional Gaussian mixtures. We introduce novel shrinkage priors for covariate-dependent mixing proportions in the mixture models to automatically select the suitable number of components used in the imputation step. We develop an efficient sampling algorithm for posterior computation and multiple imputation via Markov Chain Monte Carlo methods. The proposed method can be easily extended to the situation where the data contains not only continuous variables but also discrete variables such as binary and count value…
This paper develops the theory and methods for modeling a stationary count time series via Gaussian transformations. The techniques use a latent Gaussian process and a distributional transformation to construct stationary series with very flexible correlation features that can have any pre-specified marginal distribution, including the classical Poisson, generalized Poisson, negative binomial, and binomial structures. Gaussian pseudo-likelihood and implied Yule-Walker estimation paradigms, based on the autocovariance function of the count series, are developed via a new Hermite expansion. Part…
We discuss Bayesian analysis of multivariate time series with dynamic factor models that exploit time-adaptive sparsity in model parametrizations via the latent threshold approach. One central focus is on the transfer responses of multiple interrelated series to underlying, dynamic latent factor processes. Structured priors on model hyper-parameters are key to the efficacy of dynamic latent thresholding, and MCMC-based computation enables model fitting and analysis. A detailed case study of electroencephalographic (EEG) data from experimental psychiatry highlights the use of latent threshold e…
We study the identification of direct and indirect causes on time series and provide conditions in the presence of latent variables, which we prove to be necessary and sufficient under some graph constraints. Our theoretical results and estimation algorithms require two conditional independence tests for each observed candidate time series to determine whether or not it is a cause of an observed target time series. We provide experimental results in simulations, as well as real data. Our results show that our method leads to very low false positives and relatively low false negative rates, out…
We assume a second-order source separation model where the observed multivariate time series is a linear mixture of latent, temporally uncorrelated time series with some components pure white noise. To avoid the modelling of noise, we extract the non-noise latent components using some standard method, allowing the modelling of the extracted univariate time series individually. An important question is the determination of which of the latent components are of interest in modelling and which can be considered as noise. Bootstrap-based methods have recently been used in determining the latent di…
This paper proposes a nonparametric Bayesian method for exploratory data analysis and feature construction in continuous time series. Our method focuses on understanding shared features in a set of time series that exhibit significant individual variability. Our method builds on the framework of latent Diricihlet allocation (LDA) and its extension to hierarchical Dirichlet processes, which allows us to characterize each series as switching between latent ``topics'', where each topic is characterized as a distribution over ``words'' that specify the series dynamics. However, unlike standard app…
We study semiparametric time series models with innovations following a log-concave distribution. We propose a general maximum likelihood framework which allows us to estimate simultaneously the parameters of the model and the density of the innovations. This framework can be easily adapted to many well-known models, including ARMA, GARCH and ARMA-GARCH. Furthermore, we show that the estimator under our new framework is consistent in both ARMA and ARMA-GARCH settings. We demonstrate its finite sample performance via a thorough simulation study and apply it to model the daily log-return of FTSE…
Current understanding holds that financial contagion is driven mainly by the system-wide interconnectedness of institutions. A distinction has been made between systematic and idiosyncratic channels of contagion, with shocks transmitted through the latter expected to be substantially more likely to lead to systemic crisis than through the former. Idiosyncratic connectivity is thought to be driven not simply by obviously shared characteristics among institutions, but more by latent characteristics that lead to the holding of related securities. We develop a graphical model for multivariate fina…
Multivariate time series exhibit two types of dependence: across variables and across time points. Vine copulas are graphical models for the dependence and can conveniently capture both types of dependence in the same model. We derive the maximal class of graph structures that guarantee stationarity under a natural and verifiable condition called translation invariance. We propose computationally efficient methods for estimation, simulation, prediction, and uncertainty quantification and show their validity by asymptotic results and simulations. The theoretical results allow for misspecified m…
Count-valued time series data are routinely collected in many application areas. We are particularly motivated to study the count time series of daily new cases, arising from COVID-19 spread. We propose two Bayesian models, a time-varying semiparametric AR(p) model for count and then a time-varying INGARCH model considering the rapid changes in the spread. We calculate posterior contraction rates of the proposed Bayesian methods with respect to average Hellinger metric. Our proposed structures of the models are amenable to Hamiltonian Monte Carlo (HMC) sampling for efficient computation. We su…
This paper is concerned with modeling the dependence structure of two (or more) time-series in the presence of a (possible multivariate) covariate which may include past values of the time series. We assume that the covariate influences only the conditional mean and the conditional variance of each of the time series but the distribution of the standardized innovations is not influenced by the covariate and is stable in time. The joint distribution of the time series is then determined by the conditional means, the conditional variances and the marginal distributions of the innovations, which …
Time Series Domain Adaptation via Latent Invariant Causal Mechanism
School of Computer Science, Guangdong University of Technology（广东工业大学计算机学院）；Peng Cheng Laboratory（鹏城实验室）；School of Computing, Guangdong University of Technology（广东工业大学计算机学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局信息通信研究所）；Centre for Frontier AI Research, A*STAR（新加坡科技研究局前沿人工智能研究中心）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
We consider a time series $X=\{X_k, k\in\mathbb{Z}\}$ with memory parameter $d\in\mathbb{R}$. This time series is either stationary or can be made stationary after differencing a finite number of times. We study the "Local Whittle Wavelet Estimator" of the memory parameter $d$. This is a wavelet-based semiparametric pseudo-likelihood maximum method estimator. The estimator may depend on a given finite range of scales or on a range which becomes infinite with the sample size. We show that the estimator is consistent and rate optimal if $X$ is a linear process and is asymptotically normal if $X$…
This paper explores seasonal and long-memory time series properties by using the seasonal fractional ARIMA model when the seasonal data has one and two seasonal periods and short-memory counterparts. The stationarity and invertibility parameter conditions are established for the model studied. To estimate the memory parameters, the method given in Reisen, Rodrigues and Palma (2006 a,b) is generalized here to deal with a time series with two seasonal fractional long-memory parameters. The asymptotic properties are established and the accuracy of the method is investigated through Monte Carlo ex…
The empirical likelihood inference is extended to a class of semiparametric models for stationary, weakly dependent series. A partially linear single-index regression is used for the conditional mean of the series given its past, and the present and past values of a vector of covariates. A parametric model for the conditional variance of the series is added to capture further nonlinear effects. We propose a fixed number of suitable moment equations which characterize the mean and variance model. We derive an empirical log-likelihood ratio which includes nonparametric estimators of several func…
Two-sample hypothesis testing for random graphs arises naturally in neuroscience, social networks, and machine learning. In this paper, we consider a semiparametric problem of two-sample hypothesis testing for a class of latent position random graphs. We formulate a notion of consistency in this context and propose a valid test for the hypothesis that two finite-dimensional random dot product graphs on a common vertex set have the same generating latent positions or have generating latent positions that are scaled or diagonal transformations of one another. Our test statistic is a function of …
Identifying the causal structure of systems with multiple dynamic elements is critical to several scientific disciplines. The conventional approach is to conduct statistical tests of causality, for example with Granger Causality, between observed signals that are selected a priori. Here it is posited that, in many systems, the causal relations are embedded in a latent space that is expressed in the observed data as a linear mixture. A technique for blindly identifying the latent sources is presented: the observations are projected into pairs of components -- driving and driven -- to maximize t…
University of Toronto（多伦多大学）；Vector Institute（向量研究所）
Time series with non-uniform intervals occur in many applications, and are difficult to model using standard recurrent neural networks (RNNs). We generalize RNNs to have continuous-time hidden dynamics defined by ordinary differential equations (ODEs), a model we call ODE-RNNs. Furthermore, we use ODE-RNNs to replace the recognition network of the recently-proposed Latent ODE model. Both ODE-RNNs and Latent ODEs can naturally handle arbitrary time gaps between observations, and can explicitly model the probability of observation times using Poisson processes. We show experimentally that these …
Analyzing multivariate time series data is important to predict future events and changes of complex systems in finance, manufacturing, and administrative decisions. The expressiveness power of Gaussian Process (GP) regression methods has been significantly improved by compositional covariance structures. In this paper, we present a new GP model which naturally handles multiple time series by placing an Indian Buffet Process (IBP) prior on the presence of shared kernels. Our selective covariance structure decomposition allows exploiting shared parameters over a set of multiple, selected time s…
University of Cantabria（坎塔布里亚大学）；IMT Lille Douai（里尔-杜埃国立高等矿业电信学院）；Tecnatom S.A.（泰克纳通公司）
In this paper, we study the problem of locating a predefined sequence of patterns in a time series. In particular, the studied scenario assumes a theoretical model is available that contains the expected locations of the patterns. This problem is found in several contexts, and it is commonly solved by first synthesizing a time series from the model, and then aligning it to the true time series through dynamic time warping. We propose a technique that increases the similarity of both time series before aligning them, by mapping them into a latent correlation space. The mapping is learned from t…
Continuous latent time series models are prevalent in Bayesian modeling; examples include the Kalman filter, dynamic collaborative filtering, or dynamic topic models. These models often benefit from structured, non mean field variational approximations that capture correlations between time steps. Black box variational inference with reparameterization gradients (BBVI) allows us to explore a rich new class of Bayesian non-conjugate latent time series models; however, a naive application of BBVI to such structured variational models would scale quadratically in the number of time steps. We desc…
In the context of time series forecasting, it is a common practice to evaluate multiple methods and choose one of these methods or an ensemble for producing the best forecasts. However, choosing among different ensembles over multiple methods remains a challenging task that undergoes a combinatorial explosion as the number of methods increases. In the context of demand forecasting or revenue forecasting, this challenge is further exacerbated by a large number of time series as well as limited historical data points available due to changing business context. Although deep learning forecasting …
