The concept of Granger causality is increasingly being applied for the characterization of directional interactions in different applications. A multivariate framework for estimating Granger causality is essential in order to account for all the available information from multivariate time series. However, the inclusion of non-informative or non-significant variables creates estimation problems related to the 'curse of dimensionality'. To deal with this issue, direct causality measures using variable selection and dimension reduction techniques have been introduced. In this comparative work, the performance of an ensemble of bivariate and multivariate causality measures in the time domain is assessed, focusing on dimension reduction causality measures. In particular, different types of high-dimensional coupled discrete systems are used (involving up to 100 variables) and the robustness of the causality measures to time series length and different noise types is examined. The results of the simulation study highlight the superiority of the dimension reduction measures, especially for high-dimensional systems.
Granger causality and variants of this concept allow the study of complex dynamical systems as networks constructed from multivariate time series. In this work, a large number of Granger causality measures used to form causality networks from multivariate time series are assessed. These measures are in the time domain, such as model-based and information measures, the frequency domain and the phase domain. The study aims also to compare bivariate and multivariate measures, linear and nonlinear measures, as well as the use of dimension reduction in linear model-based measures and information me…
Causal analysis based on non-uniform embedding schemes is an important way to detect the underlying interactions between dynamic systems. However, there are still some obstacles to estimate high-dimensional conditional mutual information and form optimal mixed embedding vector in traditional non-uniform embedding schemes. In this study, we present a new non-uniform embedding method framed in information theory to detect causality for multivariate time series, named LM-PMIME, which integrates the low-dimensional approximation of conditional mutual information and the mixed search strategy for t…
We propose a nonparametric method for detecting nonlinear causal relationship within a set of multidimensional discrete time series, by using sparse additive models (SpAMs). We show that, when the input to the SpAM is a $β$-mixing time series, the model can be fitted by first approximating each unknown function with a linear combination of a set of B-spline bases, and then solving a group-lasso-type optimization problem with nonconvex regularization. Theoretically, we characterize the oracle statistical properties of the proposed sparse estimator in function estimation and model selection. Num…
Electroencephalograms (EEG) are noninvasive measurement signals of electrical neuronal activity in the brain. One of the current major statistical challenges is formally measuring functional dependency between those complex signals. This paper, proposes the spectral causality model (SCAU), a robust linear model, under a causality paradigm, to reflect inter- and intra-frequency modulation effects that cannot be identifiable using other methods. SCAU inference is conducted with three main steps: (a) signal decomposition into frequency bins, (b) intermediate spectral band mapping, and (c) depende…
Inferring nonlinear and asymmetric causal relationships between multivariate longitudinal data is a challenging task with wide-ranging application areas including clinical medicine, mathematical biology, economics and environmental research. A number of methods for inferring causal relationships within complex dynamic and stochastic systems have been proposed but there is not a unified consistent definition of causality in this context. We evaluate the performance of ten prominent bivariate causality indices for time series data, across four simulated model systems that have different coupling…
Centre de Mathématiques Appliquées, CNRS, Ecole Polytechnique, UMR 7641（应用数学中心，CNRS，巴黎综合理工学院，UMR 7641）；Capital Fund Management；Laboratoire Sciences Pour l’Environnement, Université de Corse（环境科学实验室，科西嘉大学）
We design a new nonparametric method that allows one to estimate the matrix of integrated kernels of a multivariate Hawkes process. This matrix not only encodes the mutual influences of each nodes of the process, but also disentangles the causality relationships between them. Our approach is the first that leads to an estimation of this matrix without any parametric modeling and estimation of the kernels themselves. A consequence is that it can give an estimation of causality relationships between nodes (or users), based on their activity timestamps (on a social network for instance), without …
We present a new framework for learning Granger causality networks for multivariate categorical time series, based on the mixture transition distribution (MTD) model. Traditionally, MTD is plagued by a nonconvex objective, non-identifiability, and presence of many local optima. To circumvent these problems, we recast inference in the MTD as a convex problem. The new formulation facilitates the application of MTD to high-dimensional multivariate time series. As a baseline, we also formulate a multi-output logistic autoregressive model (mLTD), which while a straightforward extension of autoregre…
To gain insight into complex systems it is a key challenge to infer nonlinear causal directional relations from observational time-series data. Specifically, estimating causal relationships between interacting components in large systems with only short recordings over few temporal observations remains an important, yet unresolved problem. Here, we introduce a large-scale Nonlinear Granger Causality (lsNGC) approach for inferring directional, nonlinear, multivariate causal interactions between system components from short high-dimensional time-series recordings. By modeling interactions with n…
Causality-Inspired Safe Residual Correction for Multivariate Time Series
University of New South Wales（新南威尔士大学）；University of Science and Technology of China（中国科学技术大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Dissecting Spectral Granger Causality through Partial Information Decomposition
University of Palermo（巴勒莫大学）；Faculty of Technical Sciences, University of Novi Sad（诺维萨德大学技术科学学院）；University of Ghent（根特大学）；University of Bari Aldo Moro（巴里阿尔多·莫罗大学）
We investigate how efficiently a known underlying sparse causality structure of a simulated multivariate linear process can be retrieved from the analysis of time-series of short lengths. Causality is quantified from conditional transfer entropy and the network is constructed by retaining only the statistically validated contributions. We compare results from three methodologies: two commonly used regularization methods, Glasso and ridge, and a newly introduced technique, LoGo, based on the combination of information filtering network and graphical modelling. For these three methodologies we e…
Identifying causal relations among simultaneously acquired signals is an important problem in multivariate time series analysis. For linear stochastic systems Granger proposed a simple procedure called the Granger causality to detect such relations. In this work we consider nonlinear extensions of Granger's idea and refer to the result as Extended Granger Causality. A simple approach implementing the Extended Granger Causality is presented and applied to multiple chaotic time series and other types of nonlinear signals. In addition, for situations with three or more time series we propose a co…
Granger causal modeling is an emerging topic that can uncover Granger causal relationship behind multivariate time series data. In many real-world systems, it is common to encounter a large amount of multivariate time series data collected from different individuals with sharing commonalities. However, there are ongoing concerns regarding Granger causality's applicability in such large scale complex scenarios, presenting both challenges and opportunities for Granger causal structure reconstruction. Existing methods usually train a distinct model for each individual, suffering from inefficiency…
Identifying risk spillovers in financial markets is of great importance for assessing systemic risk and portfolio management. Granger causality in tail (or in risk) tests whether past extreme events of a time series help predicting future extreme events of another time series. The topology and connectedness of networks built with Granger causality in tail can be used to measure systemic risk and to identify risk transmitters. Here we introduce a novel test of Granger causality in tail which adopts the likelihood ratio statistic and is based on the multivariate generalization of a discrete auto…
This article investigates the causality structure of financial time series. We concentrate on three main approaches to measuring causality: linear Granger causality, kernel generalisations of Granger causality (based on ridge regression and the Hilbert--Schmidt norm of the cross-covariance operator) and transfer entropy, examining each method and comparing their theoretical properties, with special attention given to the ability to capture nonlinear causality. We also present the theoretical benefits of applying non-symmetrical measures rather than symmetrical measures of dependence. We apply …
The problems of selecting partial correlation and causality graphs for count data are considered. A parameter driven generalized linear model is used to describe the observed multivariate time series of counts. Partial correlation and causality graphs corresponding to this model explain the dependencies between each time series of the multivariate count data. In order to estimate these graphs with tunable sparsity, an appropriate likelihood function maximization is regularized with an l1-type constraint. A novel MCEM algorithm is proposed to iteratively solve this regularized MLE. Asymptotic c…
This paper proves the consistency property for the regularized maximum likelihood estimators (MLEs) of multivariate Hawkes processes (MHPs). It also develops an alternating minimization type algorithm (AA-iPALM) to compute the MLEs with guaranteed global convergence to the set of stationary points. The performance of this AA-iPALM algorithm on both synthetic and real-world data shows that AA-iPALM consistently improves over iPALM and PALM. Moreover, AA-iPALM is able to identify the causality structure in rental listings on craigslist and herd behavior in the NASDAQ ITCH dataset.
Hawkes (1971) introduced a powerful multivariate point process model of mutually exciting processes to explain causal structure in data. In this paper it is shown that the Granger causality structure of such processes is fully encoded in the corresponding link functions of the model. A new nonparametric estimator of the link functions based on a time-discretized version of the point process is introduced by using an infinite order autoregression. Consistency of the new estimator is derived. The estimator is applied to simulated data and to neural spike train data from the spinal dorsal horn of…
The dependencies of the lagged (Pearson) correlation function on the coefficients of multivariate autoregressive models are interpreted in the framework of time series graphs. Time series graphs are related to the concept of Granger causality and encode the conditional independence structure of a multivariate process. The authors show that the complex dependencies of the Pearson correlation coefficient complicate an interpretation and propose a novel partial correlation measure with a straightforward graph-theoretical interpretation. The novel measure has the additional advantage that its samp…
In modeling multivariate time series for either forecast or policy analysis, it would be beneficial to have figured out the cause-effect relations within the data. Regression analysis, however, is generally for correlation relation, and very few researches have focused on variance analysis for causality discovery. We first set up an equilibrium for the cause-effect relations using a fictitious vector autoregressive model. In the equilibrium, long-run relations are identified from noise, and spurious ones are negligibly close to zero. The solution, called causality distribution, measures the re…
We introduce graphical time series models for the analysis of dynamic relationships among variables in multivariate time series. The modelling approach is based on the notion of strong Granger causality and can be applied to time series with non-linear dependencies. The models are derived from ordinary time series models by imposing constraints that are encoded by mixed graphs. In these graphs each component series is represented by a single vertex and directed edges indicate possible Granger-causal relationships between variables while undirected edges are used to map the contemporaneous depe…
Inferring linear dependence between time series is central to our understanding of natural and artificial systems. Unfortunately, the hypothesis tests that are used to determine statistically significant directed or multivariate relationships from time-series data often yield spurious associations (Type I errors) or omit causal relationships (Type II errors). This is due to the autocorrelation present in the analysed time series -- a property that is ubiquitous across diverse applications, from brain dynamics to climate change. Here we show that, for limited data, this issue cannot be mediated…
Identifying directed interactions between species from time series of their population densities has many uses in ecology. This key statistical task is equivalent to causal time series inference, which connects to the Granger causality (GC) concept: $x$ causes $y$ if $x$ improves the prediction of $y$ in a dynamic model. However, the entangled nature of nonlinear ecological systems has led to question the appropriateness of Granger causality, especially in its classical linear Multivariate AutoRegressive (MAR) model form. Convergent-cross mapping (CCM), a nonparametric method developed for det…
Learning the influence structure of multiple time series data is of great interest to many disciplines. This paper studies the problem of recovering the causal structure in network of multivariate linear Hawkes processes. In such processes, the occurrence of an event in one process affects the probability of occurrence of new events in some other processes. Thus, a natural notion of causality exists between such processes captured by the support of the excitation matrix. We show that the resulting causal influence network is equivalent to the Directed Information graph (DIG) of the processes, …
Identifying causal relationships is a challenging yet crucial problem in many fields of science like epidemiology, climatology, ecology, genomics, economics and neuroscience, to mention only a few. Recent studies have demonstrated that ordinal partition transition networks (OPTNs) allow inferring the coupling direction between two dynamical systems. In this work, we generalize this concept to the study of the interactions among multiple dynamical systems and we propose a new method to detect causality in multivariate observational data. By applying this method to numerical simulations of coupl…
We show that univariate and symmetric multivariate Hawkes processes are only weakly causal: the true log-likelihoods of real and reversed event time vectors are almost equal, thus parameter estimation via maximum likelihood only weakly depends on the direction of the arrow of time. In ideal (synthetic) conditions, tests of goodness of parametric fit unambiguously reject backward event times, which implies that inferring kernels from time-symmetric quantities, such as the autocovariance of the event rate, only rarely produce statistically significant fits. Finally, we find that fitting financia…
We propose a new measure to estimate the direction of information flux in multivariate time series from complex systems. This measure, based on the slope of the phase spectrum (Phase Slope Index) has invariance properties that are important for applications in real physical or biological systems: (a) it is strictly insensitive to mixtures of arbitrary independent sources, (b) it gives meaningful results even if the phase spectrum is not linear, and (c) it properly weights contributions from different frequencies. Simulations of a class of coupled multivariate random data show that for truly un…
University of Southern California（南加州大学）；IBM T.J. Watson Research Center（IBM T.J. Watson研究中心）
In many applications of time series models, such as climate analysis and social media analysis, we are often interested in extreme events, such as heatwave, wind gust, and burst of topics. These time series data usually exhibit a heavy-tailed distribution rather than a Gaussian distribution. This poses great challenges to existing approaches due to the significantly different assumptions on the data distributions and the lack of sufficient past data on extreme events. In this paper, we propose the Sparse-GEV model, a latent state model based on the theory of extreme value modeling to automatic…
Causality graphs are routinely estimated in social sciences, natural sciences, and engineering due to their capacity to efficiently represent the spatiotemporal structure of multivariate data sets in a format amenable for human interpretation, forecasting, and anomaly detection. A popular approach to mathematically formalize causality is based on vector autoregressive (VAR) models and constitutes an alternative to the well-known, yet usually intractable, Granger causality. Relying on such a VAR causality notion, this paper develops two algorithms with complementary benefits to track time-varyi…
In the study of complex physical and biological systems represented by multivariate stochastic processes, an issue of great relevance is the description of the system dynamics spanning multiple temporal scales. While methods to assess the dynamic complexity of individual processes at different time scales are well-established, multiscale analysis of directed interactions has never been formalized theoretically, and empirical evaluations are complicated by practical issues such as filtering and downsampling. Here we extend the very popular measure of Granger causality (GC), a prominent tool for…
Granger causality analysis is a popular method for inference on directed interactions in complex systems of many variables. A shortcoming of the standard framework for Granger causality is that it only allows for examination of interactions between single (univariate) variables within a system, perhaps conditioned on other variables. However, interactions do not necessarily take place between single variables, but may occur among groups, or "ensembles", of variables. In this study we establish a principled framework for Granger causality in the context of causal interactions among two or more …
Causal and Local Correlations Based Network for Multivariate Time Series Classification
School of Information Science and Engineering, Shandong Normal University（山东师范大学信息科学与工程学院）；Shandong Provincial Key Laboratory for Distributed Computer Software Novel Technology（山东省分布式计算机软件新技术重点实验室）
The multivariate nonlinear Granger causality developed by Bai et al. (2010) plays an important role in detecting the dynamic interrelationships between two groups of variables. Following the idea of Hiemstra-Jones (HJ) test proposed by Hiemstra and Jones (1994), they attempt to establish a central limit theorem (CLT) of their test statistic by applying the asymptotical property of multivariate $U$-statistic. However, Bai et al. (2016) revisit the HJ test and find that the test statistic given by HJ is NOT a function of $U$-statistics which implies that the CLT neither proposed by Hiemstra and …
At least two recent developments have put the spotlight on some significant gaps in the theory of multivariate time series. The recent interest in the dynamics of networks; and the advent, across a range of applications, of measuring modalities that operate on different temporal scales. Fundamental to the description of network dynamics is the direction of interaction between nodes, accompanied by a measure of the strength of such interactions. Granger causality (GC) and its associated frequency domain strength measures (GEMs) (due to Geweke) provide a framework for the formulation and analysi…
Information Technologies Institute, Centre of Research and Technology Hellas（希腊研究与技术中心信息技术研究所）
Multivariate time series forecasting is of great importance to many scientific disciplines and industrial sectors. The evolution of a multivariate time series depends on the dynamics of its variables and the connectivity network of causal interrelationships among them. Most of the existing time series models do not account for the causal effects among the system's variables and even if they do they rely just on determining the between-variables causality network. Knowing the structure of such a complex network and even more specifically knowing the exact lagged variables that contribute to the…
Multivariate functional data arise in a wide range of applications. One fundamental task is to understand the causal relationships among these functional objects of interest, which has not yet been fully explored. In this article, we develop a novel Bayesian network model for multivariate functional data where the conditional independence and causal structure are both encoded by a directed acyclic graph. Specifically, we allow the functional objects to deviate from Gaussian process, which is adopted by most existing functional data analysis models. The more reasonable non-Gaussian assumption i…
University of Maastricht（马斯特里赫特大学）；University College London（伦敦大学学院）
We propose a definition of causality for time series in terms of the effect of an intervention in one component of a multivariate time series on another component at some later point in time. Conditions for identifiability, comparable to the back-door and front-door criteria, are presented and can also be verified graphically. Computation of the causal effect is derived and illustrated for the linear case.
The current literature in probabilistic forecasting is focused on quantifying the uncertainty of each random variable individually. This leads to the failure in informing about interdependence structure of uncertainty at different locations and/or different lead times. When there is a positive or negative association between a number of random variables, the prediction regions for them should be reflected by multivariate or joint uncertainty sets. The existing literature is very primitive in the area of multivariate uncertainty sets modeling. In this paper, uncertainty regions are generated in…
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Graduate School, Tsinghua University（清华大学研究生院）
IBM Thomas J. Watson Research Center（IBM托马斯·J·沃森研究中心）
Discovering temporal lagged and inter-dependencies in multivariate time series data is an important task. However, in many real-world applications, such as commercial cloud management, manufacturing predictive maintenance, and portfolios performance analysis, such dependencies can be non-linear and time-variant, which makes it more challenging to extract such dependencies through traditional methods such as Granger causality or clustering. In this work, we present a novel deep learning model that uses multiple layers of customized gated recurrent units (GRUs) for discovering both time lagged b…
Wiener and Granger have introduced an intuitive concept of causality between two variables which is based on the idea that an effect never occurs before its cause. Later, Geweke has generalized this concept to a multivariate Granger causality, i.e., n variables Granger-cause another variable. Although Granger causality is not "effective causality", this concept is useful to infer directionality and information flow in observational data. Granger causality is usually identified by using VAR models due to their simplicity. In the last few years, several VAR-based models were presented in order t…
Our goal is to estimate causal interactions in multivariate time series. Using vector autoregressive (VAR) models, these can be defined based on non-vanishing coefficients belonging to respective time-lagged instances. As in most cases a parsimonious causality structure is assumed, a promising approach to causal discovery consists in fitting VAR models with an additional sparsity-promoting regularization. Along this line we here propose that sparsity should be enforced for the subgroups of coefficients that belong to each pair of time series, as the absence of a causal relation requires the co…
Wiener-Granger causality is a widely used framework of causal analysis for temporally resolved events. We introduce a new measure of Wiener-Granger causality based on kernelization of partial canonical correlation analysis with specific advantages in the context of large high-dimensional data. The introduced measure is able to detect non-linear and non-monotonous signals, is designed to be immune to noise, and offers tunability in terms of computational complexity in its estimations. Furthermore, we show that, under specified conditions, the introduced measure can be regarded as an estimate of…
The Y-test is a useful tool for detecting missing confounders in the context of a multivariate regression.However, it is rarely used in practice since it requires identifying multiple conditionally independent instruments, which is often impossible. We propose a heuristic test which relaxes the independence requirement. We then show how to apply this heuristic test on a price-demand and a firm loan-productivity problem. We conclude that the test is informative when the variables are linearly related with Gaussian additive noise, but it can be misleading in other contexts. Still, we believe tha…
The stochastic system approach to causality is applied to situations where the risk of death is not negligible. This approach grounds causality on physical laws, distinguishes system and observation and represents the system by multivariate stochastic processes. The particular role of death is highlighted, and it is shown that local influences must be defined on the random horizon of time of death. We particularly study the problem of estimating the effect of a factor $V$ on a process of interest $Y$, taking death into account. We unify the cases where $Y$ is a counting process (describing an …
