Variable-lag Granger Causality and Transfer Entropy for Time Series Analysis
National Electronics and Computer Technology Center（国家电子和计算机技术中心）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；The Ohio State University（俄亥俄州立大学）
Chainarong Amornbunchornvej, Elena Zheleva, Tanya Berger-Wolf
备注：
This preprint is the extension of the work [arXiv:1912.10829] entitled "Variable-lag Granger Causality for Time Series Analysis" by the same authors. The revision was made based on reviewers' suggestions. The R package is available at https://github.com/DarkEyes/VLTimeSeriesCausality
发表信息：
ACM Transactions on Knowledge Discovery from Data (TKDD), 15(4), 67 (2021)
Granger causality is a fundamental technique for causal inference in time series data, commonly used in the social and biological sciences. Typical operationalizations of Granger causality make a strong assumption that every time point of the effect time series is influenced by a combination of other time series with a fixed time delay. The assumption of fixed time delay also exists in Transfer Entropy, which is considered to be a non-linear version of Granger causality. However, the assumption of the fixed time delay does not hold in many applications, such as collective behavior, financial markets, and many natural phenomena. To address this issue, we develop Variable-lag Granger causality and Variable-lag Transfer Entropy, generalizations of both Granger causality and Transfer Entropy that relax the assumption of the fixed time delay and allow causes to influence effects with arbitrary time delays. In addition, we propose methods for inferring both variable-lag Granger causality and Transfer Entropy relations. In our approaches, we utilize an optimal warping path of Dynamic Time Warping (DTW) to infer variable-lag causal relations. We demonstrate our approaches on an application for studying coordinated collective behavior and other real-world casual-inference datasets and show that our proposed approaches perform better than several existing methods in both simulated and real-world datasets. Our approaches can be applied in any domain of time series analysis. The software of this work is available in the R-CRAN package: VLTimeCausality.
Beijing University of Posts and Telecommunications（北京邮电大学）；Tsinghua University（清华大学）
Granger causality method analyzes the time series causalities without building a complex causality graph. However, the traditional Granger causality method assumes that the causalities lie between time series channels and remain constant, which cannot model the real-world time series data with dynamic causalities along the time series channels. In this paper, we present the dynamic window-level Granger causality method (DWGC) for multi-channel time series data. We build the causality model on the window-level by doing the F-test with the forecasting errors on the sliding windows. We propose th…
National Electronics and Computer Technology Center（国家电子和计算机技术中心）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
Granger causality is a fundamental technique for causal inference in time series data, commonly used in the social and biological sciences. Typical operationalizations of Granger causality make a strong assumption that every time point of the effect time series is influenced by a combination of other time series with a fixed time delay. However, the assumption of the fixed time delay does not hold in many applications, such as collective behavior, financial markets, and many natural phenomena. To address this issue, we develop variable-lag Granger causality, a generalization of Granger causali…
Learning Granger causality for general point processes is a very challenging task. In this paper, we propose an effective method, learning Granger causality, for a special but significant type of point processes --- Hawkes process. We reveal the relationship between Hawkes process's impact function and its Granger causality graph. Specifically, our model represents impact functions using a series of basis functions and recovers the Granger causality graph via group sparsity of the impact functions' coefficients. We propose an effective learning algorithm combining a maximum likelihood estimato…
Introduced more than a half century ago, Granger causality has become a popular tool for analyzing time series data in many application domains, from economics and finance to genomics and neuroscience. Despite this popularity, the validity of this notion for inferring causal relationships among time series has remained the topic of continuous debate. Moreover, while the original definition was general, limitations in computational tools have primarily limited the applications of Granger causality to simple bivariate vector auto-regressive processes or pairwise relationships among a set of vari…
Faculty of Business and Economics, University of Lausanne（洛桑大学商学院）；Department of Statistics, UC Berkeley（伯克利大学统计学系）；Research Institute for Statistics and Information Science, University of Geneva（日内瓦大学统计与信息科学研究所）；Department of Industrial Engineering and Operations Research, Columbia University（哥伦比亚大学工业工程与运筹学系）
Inferring causal relationships in observational time series data is an important task when interventions cannot be performed. Granger causality is a popular framework to infer potential causal mechanisms between different time series. The original definition of Granger causality is restricted to linear processes and leads to spurious conclusions in the presence of a latent confounder. In this work, we harness the expressive power of recurrent neural networks and propose a deep learning-based approach to model non-linear Granger causality by directly accounting for latent confounders. Our appro…
Identifying causal relations among simultaneously acquired signals is an important problem in multivariate time series analysis. For linear stochastic systems Granger proposed a simple procedure called the Granger causality to detect such relations. In this work we consider nonlinear extensions of Granger's idea and refer to the result as Extended Granger Causality. A simple approach implementing the Extended Granger Causality is presented and applied to multiple chaotic time series and other types of nonlinear signals. In addition, for situations with three or more time series we propose a co…
In this paper, we propose a new Granger causality measure which is robust against the confounding influence of latent common inputs. This measure is inspired by partial Granger causality in the literature, and its variant. Using numerical experiments we first show that the test statistics for detecting directed interactions between time series approximately obey the $F$-distributions when there are no interactions. Then, we propose a practical procedure for inferring directed interactions, which is based on the idea of multiple statistical test in situations where the confounding influence of …
We show for the first time that, under the null hypothesis of vanishing Granger causality, the single-regression Granger-Geweke estimator converges to a generalised $χ^2$ distribution, which may be well approximated by a $Γ$ distribution. We show that this holds too for Geweke's spectral causality averaged over a given frequency band, and derive explicit expressions for the generalised $χ^2$ and $Γ$-approximation parameters in both cases. We present an asymptotically valid Neyman-Pearson test based on the single-regression estimators, and discuss in detail how it may be usefully employed in re…
While most classical approaches to Granger causality detection repose upon linear time series assumptions, many interactions in neuroscience and economics applications are nonlinear. We develop an approach to nonlinear Granger causality detection using multilayer perceptrons where the input to the network is the past time lags of all series and the output is the future value of a single series. A sufficient condition for Granger non-causality in this setting is that all of the outgoing weights of the input data, the past lags of a series, to the first hidden layer are zero. For estimation, we …
Physicists are starting to work in areas where noisy signal analysis is required. In these fields, such as Economics, Neuroscience, and Physics, the notion of causality should be interpreted as a statistical measure. We introduce to the lay reader the Granger causality between two time series and illustrate ways of calculating it: a signal $X$ ``Granger-causes'' a signal $Y$ if the observation of the past of $X$ increases the predictability of the future of $Y$ when compared to the same prediction done with the past of $Y$ alone. In other words, for Granger causality between two quantities it …
We study causality between bivariate curve time series using the Granger causality generalized measures of correlation. With this measure, we can investigate which curve time series Granger-causes the other; in turn, it helps determine the predictability of any two curve time series. Illustrated by a climatology example, we find that the sea surface temperature Granger-causes the sea-level atmospheric pressure. Motivated by a portfolio management application in finance, we single out those stocks that lead or lag behind Dow-Jones industrial averages. Given a close relationship between S&P 500 …
This paper proposes a novel method (GLS Granger test) to determine causal relationships between time series based on the estimation of the autocovariance matrix and generalized least squares. We show the effectiveness of proposed autocovariance matrix estimator (the sliding autocovariance matrix) and we compare the proposed method with the classical Granger F-test with via a synthetic dataset and a real dataset composed by cryptocurrencies. The simulations show that the proposed GLS Granger test captures causality more accurately than Granger F-tests in the cases of heteroskedastic or structur…
Granger causality is a widely-used criterion for analyzing interactions in large-scale networks. As most physical interactions are inherently nonlinear, we consider the problem of inferring the existence of pairwise Granger causality between nonlinearly interacting stochastic processes from their time series measurements. Our proposed approach relies on modeling the embedded nonlinearities in the measurements using a component-wise time series prediction model based on Statistical Recurrent Units (SRUs). We make a case that the network topology of Granger causal relations is directly inferrabl…
That physiological oscillations of various frequencies are present in fMRI signals is the rule, not the exception. Herein, we propose a novel theoretical framework, spatio-temporal Granger causality, which allows us to more reliably and precisely estimate the Granger causality from experimental datasets possessing time-varying properties caused by physiological oscillations. Within this framework, Granger causality is redefined as a global index measuring the directed information flow between two time series with time-varying properties. Both theoretical analyses and numerical examples demonst…
Granger causality and variants of this concept allow the study of complex dynamical systems as networks constructed from multivariate time series. In this work, a large number of Granger causality measures used to form causality networks from multivariate time series are assessed. These measures are in the time domain, such as model-based and information measures, the frequency domain and the phase domain. The study aims also to compare bivariate and multivariate measures, linear and nonlinear measures, as well as the use of dimension reduction in linear model-based measures and information me…
Granger causality is well established within the neurosciences for inference of directed functional connectivity from neurophysiological data. These data usually consist of time series which subsample a continuous-time biophysiological process. While it is well-known that subsampling can lead to imputation of spurious causal connections where none exist, here we address the equally important issue of the effects of subsampling on the ability to reliably detect causal connections which do exist. Neurophysiological processes typically feature signal propagation delays on multiple time scales; ac…
Granger causality has been used for the investigation of the inter-dependence structure of the underlying systems of multi-variate time series. In particular, the direct causal effects are commonly estimated by the conditional Granger causality index (CGCI). In the presence of many observed variables and relatively short time series, CGCI may fail because it is based on vector autoregressive models (VAR) involving a large number of coefficients to be estimated. In this work, the VAR is restricted by a scheme that modifies the recently developed method of backward-in-time selection (BTS) of the…
This paper develops a deep learning-based econometric methodology to determine the causality of the financial time series. This method is applied to the imbalances in daily transactions in individual stocks, as well as the ETFs reported to SEC with a nanosecond time stamp. Based on our method, we conclude that transaction imbalances of ETFs alone are more informative than the transaction imbalances in the entire market. Characteristically, a sheer number of imbalance messages related to the individual stocks dominates the imbalance messages due to the ETF in the proportion of 8:1.
Identifying directed interactions between species from time series of their population densities has many uses in ecology. This key statistical task is equivalent to causal time series inference, which connects to the Granger causality (GC) concept: $x$ causes $y$ if $x$ improves the prediction of $y$ in a dynamic model. However, the entangled nature of nonlinear ecological systems has led to question the appropriateness of Granger causality, especially in its classical linear Multivariate AutoRegressive (MAR) model form. Convergent-cross mapping (CCM), a nonparametric method developed for det…
Dissecting Spectral Granger Causality through Partial Information Decomposition
University of Palermo（巴勒莫大学）；Faculty of Technical Sciences, University of Novi Sad（诺维萨德大学技术科学学院）；University of Ghent（根特大学）；University of Bari Aldo Moro（巴里阿尔多·莫罗大学）
Granger causal modeling is an emerging topic that can uncover Granger causal relationship behind multivariate time series data. In many real-world systems, it is common to encounter a large amount of multivariate time series data collected from different individuals with sharing commonalities. However, there are ongoing concerns regarding Granger causality's applicability in such large scale complex scenarios, presenting both challenges and opportunities for Granger causal structure reconstruction. Existing methods usually train a distinct model for each individual, suffering from inefficiency…
We present a new framework for learning Granger causality networks for multivariate categorical time series, based on the mixture transition distribution (MTD) model. Traditionally, MTD is plagued by a nonconvex objective, non-identifiability, and presence of many local optima. To circumvent these problems, we recast inference in the MTD as a convex problem. The new formulation facilitates the application of MTD to high-dimensional multivariate time series. As a baseline, we also formulate a multi-output logistic autoregressive model (mLTD), which while a straightforward extension of autoregre…
Identifying risk spillovers in financial markets is of great importance for assessing systemic risk and portfolio management. Granger causality in tail (or in risk) tests whether past extreme events of a time series help predicting future extreme events of another time series. The topology and connectedness of networks built with Granger causality in tail can be used to measure systemic risk and to identify risk transmitters. Here we introduce a novel test of Granger causality in tail which adopts the likelihood ratio statistic and is based on the multivariate generalization of a discrete auto…
Linear regression model (LRM) based on mean square error (MSE) criterion is widely used in Granger causality analysis (GCA), which is the most commonly used method to detect the causality between a pair of time series. However, when signals are seriously contaminated by non-Gaussian noises, the LRM coefficients will be inaccurately identified. This may cause the GCA to detect a wrong causal relationship. Minimum error entropy (MEE) criterion can be used to replace the MSE criterion to deal with the non-Gaussian noises. But its calculation requires a double summation operation, which brings com…
We study Granger causality in the context of wide-sense stationary time series, where our focus is on the topological aspects of the underlying causality graph. We establish sufficient conditions (in particular, we develop the notion of a "strongly causal" graph topology) under which the true causality graph can be recovered via pairwise causality testing alone, and provide examples from the gene regulatory network literature suggesting that our concept of a strongly causal graph may be applicable to this field. We implement and detail finite-sample heuristics derived from our theory, and esta…
To gain insight into complex systems it is a key challenge to infer nonlinear causal directional relations from observational time-series data. Specifically, estimating causal relationships between interacting components in large systems with only short recordings over few temporal observations remains an important, yet unresolved problem. Here, we introduce a large-scale Nonlinear Granger Causality (lsNGC) approach for inferring directional, nonlinear, multivariate causal interactions between system components from short high-dimensional time-series recordings. By modeling interactions with n…
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Graduate School, Tsinghua University（清华大学研究生院）
In the study of complex physical and biological systems represented by multivariate stochastic processes, an issue of great relevance is the description of the system dynamics spanning multiple temporal scales. While methods to assess the dynamic complexity of individual processes at different time scales are well-established, multiscale analysis of directed interactions has never been formalized theoretically, and empirical evaluations are complicated by practical issues such as filtering and downsampling. Here we extend the very popular measure of Granger causality (GC), a prominent tool for…
At least two recent developments have put the spotlight on some significant gaps in the theory of multivariate time series. The recent interest in the dynamics of networks; and the advent, across a range of applications, of measuring modalities that operate on different temporal scales. Fundamental to the description of network dynamics is the direction of interaction between nodes, accompanied by a measure of the strength of such interactions. Granger causality (GC) and its associated frequency domain strength measures (GEMs) (due to Geweke) provide a framework for the formulation and analysi…
The concept of Granger causality is increasingly being applied for the characterization of directional interactions in different applications. A multivariate framework for estimating Granger causality is essential in order to account for all the available information from multivariate time series. However, the inclusion of non-informative or non-significant variables creates estimation problems related to the 'curse of dimensionality'. To deal with this issue, direct causality measures using variable selection and dimension reduction techniques have been introduced. In this comparative work, t…
This paper develops a method for estimating parameters of a vector autoregression (VAR) observed in white noise. The estimation method assumes the noise variance matrix is known and does not require any iterative process. This study provides consistent estimators and shows the asymptotic distribution of the parameters required for conducting tests of Granger causality. Methods in the existing statistical literature cannot be used for testing Granger causality, since under the null hypothesis the model becomes unidentifiable. Measurement error effects on parameter estimates were evaluated by us…
University of Applied Sciences and Arts of Western Switzerland（瑞士西部应用科学与艺术大学）；University of Geneva（日内瓦大学）
We consider the problem of learning models for forecasting multiple time-series systems together with discovering the leading indicators that serve as good predictors for the system. We model the systems by linear vector autoregressive models (VAR) and link the discovery of leading indicators to inferring sparse graphs of Granger-causality. We propose new problem formulations and develop two new methods to learn such models, gradually increasing the complexity of assumptions and approaches. While the first method assumes common structures across the whole system, our second method uncovers mod…
This article investigates the causality structure of financial time series. We concentrate on three main approaches to measuring causality: linear Granger causality, kernel generalisations of Granger causality (based on ridge regression and the Hilbert--Schmidt norm of the cross-covariance operator) and transfer entropy, examining each method and comparing their theoretical properties, with special attention given to the ability to capture nonlinear causality. We also present the theoretical benefits of applying non-symmetrical measures rather than symmetrical measures of dependence. We apply …
Linear Vector AutoRegressive (VAR) models where the innovations could be unconditionally heteroscedastic and serially dependent are considered. The volatility structure is deterministic and quite general, including breaks or trending variances as special cases. In this framework we propose Ordinary Least Squares (OLS), Generalized Least Squares (GLS) and Adaptive Least Squares (ALS) procedures. The GLS estimator requires the knowledge of the time-varying variance structure while in the ALS approach the unknown variance is estimated by kernel smoothing with the outer product of the OLS residual…
Exploratory analysis of time series data can yield a better understanding of complex dynamical systems. Granger causality is a practical framework for analysing interactions in sequential data, applied in a wide range of domains. In this paper, we propose a novel framework for inferring multivariate Granger causality under nonlinear dynamics based on an extension of self-explaining neural networks. This framework is more interpretable than other neural-network-based techniques for inferring Granger causality, since in addition to relational inference, it also allows detecting signs of Granger-…
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Aalborg University（奥尔堡大学）；XaiPient（赛派恩特）；Duke University（杜克大学）
We study the problem of learning Granger causality between event types from asynchronous, interdependent, multi-type event sequences. Existing work suffers from either limited model flexibility or poor model explainability and thus fails to uncover Granger causality across a wide variety of event sequences with diverse event interdependency. To address these weaknesses, we propose CAUSE (Causality from AttribUtions on Sequence of Events), a novel framework for the studied task. The key idea of CAUSE is to first implicitly capture the underlying event interdependency by fitting a neural point p…
We introduce graphical time series models for the analysis of dynamic relationships among variables in multivariate time series. The modelling approach is based on the notion of strong Granger causality and can be applied to time series with non-linear dependencies. The models are derived from ordinary time series models by imposing constraints that are encoded by mixed graphs. In these graphs each component series is represented by a single vertex and directed edges indicate possible Granger-causal relationships between variables while undirected edges are used to map the contemporaneous depe…
Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所）；RIKEN AIP（理化学研究所人工智能项目）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）
Granger causality analysis, as one of the most popular time series causality methods, has been widely used in the economics, neuroscience. However, unobserved confounders is a fundamental problem in the observational studies, which is still not solved for the non-linear Granger causality. The application works often deal with this problem in virtue of the proxy variables, who can be treated as a measure of the confounder with noise. But the proxy variables has been proved to be unreliable, because of the bias it may induce. In this paper, we try to "recover" the unobserved confounders for the …
We generalize a previously proposed approach for nonlinear Granger causality of time series, based on radial basis function. The proposed model is not constrained to be additive in variables from the two time series and can approximate any function of these variables, still being suitable to evaluate causality. Usefulness of this measure of causality is shown in a physiological example and in the study of the feed-back loop in a model of excitatory and inhibitory neurons.
In this foundational expository article on the application of Causality Analysis in IoT, we establish the basic theory and algorithms for estimating Structural and Granger causality factors from measured multichannel sensor data (vector timeseries). Vector timeseries is modeled as a Structural Vector Autoregressive (SVAR) model; utilizing Kalman Filter and Independent Component Analysis (ICA) methods, Structural and generalized Granger causality factors are estimated. The estimated causal factors are presented as a Fence graph which we call Causal Digital Twin. Practical applications of Causal…
