Inferring linear dependence between time series is central to our understanding of natural and artificial systems. Unfortunately, the hypothesis tests that are used to determine statistically significant directed or multivariate relationships from time-series data often yield spurious associations (Type I errors) or omit causal relationships (Type II errors). This is due to the autocorrelation present in the analysed time series -- a property that is ubiquitous across diverse applications, from brain dynamics to climate change. Here we show that, for limited data, this issue cannot be mediated by fitting a time-series model alone (e.g., in Granger causality or prewhitening approaches), and instead that the degrees of freedom in statistical tests should be altered to account for the effective sample size induced by cross-correlations in the observations. This insight enabled us to derive modified hypothesis tests for any multivariate correlation-based measures of linear dependence between covariance-stationary time series, including Granger causality and mutual information with Gaussian marginals. We use both numerical simulations (generated by autoregressive models and digital filtering) as well as recorded fMRI-neuroimaging data to show that our tests are unbiased for a variety of stationary time series. Our experiments demonstrate that the commonly used $F$- and $χ^2$-tests can induce significant false-positive rates of up to $100\%$ for both measures, with and without prewhitening of the signals. These findings suggest that many dependencies reported in the scientific literature may have been, and may continue to be, spuriously reported or missed if modified hypothesis tests are not used when analysing time series.
We introduce supervised feature ranking and feature subset selection algorithms for multivariate time series (MTS) classification. Unlike most existing supervised/unsupervised feature selection algorithms for MTS our techniques do not require a feature extraction step to generate a one-dimensional feature vector from the time series. Instead it is based on directly computing similarity between individual time series and assessing how well the resulting cluster structure matches the labels. The techniques are amenable to heterogeneous MTS data, where the time series measurements may have differ…
Integrating Statistical Significance and Discriminative Power in Pattern Discovery
INESC-ID and Instituto Superior Técnico, Universidade de Lisboa（INESC-ID 和里斯本大学高等技术学院）；LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, Universidade NOVA de Lisboa（LAQV-REQUIMTE，新里斯本大学科学与技术学院化学系）
We propose new tests for assessing whether covariates in a treatment group and matched control group are balanced in observational studies. The tests exhibit high power under a wide range of multivariate alternatives, some of which existing tests have little power for. The asymptotic permutation null distributions of the proposed tests are studied and the p-values calculated through the asymptotic results work well in finite samples, facilitating the application of the test to large data sets. The tests are illustrated in a study of the effect of smoking on blood lead levels. The proposed test…
This chapter discusses correlation analysis of stationary multivariate Gaussian time series in the spectral or Fourier domain. The goal is to identify the hub time series, i.e., those that are highly correlated with a specified number of other time series. We show that Fourier components of the time series at different frequencies are asymptotically statistically independent. This property permits independent correlation analysis at each frequency, alleviating the computational and statistical challenges of high-dimensional time series. To detect correlation hubs at each frequency, an existing…
To gain insight into complex systems it is a key challenge to infer nonlinear causal directional relations from observational time-series data. Specifically, estimating causal relationships between interacting components in large systems with only short recordings over few temporal observations remains an important, yet unresolved problem. Here, we introduce a large-scale Nonlinear Granger Causality (lsNGC) approach for inferring directional, nonlinear, multivariate causal interactions between system components from short high-dimensional time-series recordings. By modeling interactions with n…
Coherence and phase synchronization between time series corresponding to different spatial locations are usually interpreted as indicators of the connectivity between locations. In neurophysiology, time series of electric neuronal activity are essential for studying brain interconnectivity. Such signals can either be invasively measured from depth electrodes, or computed from very high time resolution, non-invasive, extracranial recordings of scalp electric potential differences (EEG: electroencephalogram) and magnetic fields (MEG: magnetoencephalogram) by means of a tomography such as sLORETA…
Longitudinal imaging studies have moved to the forefront of medical research due to their ability to characterize spatio-temporal features of biological structures across the lifespan. Credible models of the correlations in longitudinal imaging require two or more pattern components. Valid inference requires enough flexibility of the correlation model to allow reasonable fidelity to the true pattern. On the other hand, the existence of computable estimates demands a parsimonious parameterization of the correlation structure. For many one-dimensional spatial or temporal arrays, the linear expon…
Understanding which phenotypic traits are consistently correlated throughout evolution is a highly pertinent problem in modern evolutionary biology. Here, we propose a multivariate phylogenetic latent liability model for assessing the correlation between multiple types of data, while simultaneously controlling for their unknown shared evolutionary history informed through molecular sequences. The latent formulation enables us to consider in a single model combinations of continuous traits, discrete binary traits and discrete traits with multiple ordered and unordered states. Previous approache…
In many psychometric applications, the relationship between the mean of an outcome and a quantitative covariate is too complex to be described by simple parametric functions; instead, flexible nonlinear relationships can be incorporated using penalized splines. Penalized splines can be conveniently represented as a linear mixed effects model (LMM), where the coefficients of the spline basis functions are random effects. The LMM representation of penalized splines makes the extension to multivariate outcomes relatively straightforward. In the LMM, no effect of the quantitative covariate on the …
Any decision making process that relies on a probabilistic forecast of future events necessarily requires a calibrated forecast. This paper proposes new methods for empirically assessing forecast calibration in a multivariate setting where the probabilistic forecast is given by an ensemble of equally probable forecast scenarios. Multivariate properties are mapped to a single dimension through a pre-rank function and the calibration is subsequently assessed visually through a histogram of the ranks of the observation's pre-ranks. Average ranking assigns a pre-rank based on the average univariat…
Multivariate categorical data are common in many fields. We are motivated by election polls studies assessing evidence of changes in voters opinions with their candidates preferences in the 2016 United States Presidential primaries or caucuses. Similar goals arise routinely in several applications, but current literature lacks a general methodology which combines flexibility, efficiency, and tractability in testing for group differences in multivariate categorical data at different---potentially complex---scales. We address this goal by leveraging a Bayesian representation which factorizes the…
A Formal Graphical Inference Framework for Combining Effect Size with Statistical Significance: Application to Multivariate and Functional Linear Models
Faculty of Agriculture and Technology, University of South Bohemia（南波希米亚大学农业与技术学院）；Natural Resources Institute Finland (Luke)（芬兰自然资源研究所（Luke））；University of Helsinki（赫尔辛基大学）
TriSig: Assessing the statistical significance of triclusters
INESC-ID（INESC-ID（葡萄牙国家系统与计算机工程研究所））；Instituto Superior Técnico, University of Lisbon（里斯本大学高等技术学院）；LAQV-REQUIMTE（LAQV-REQUIMTE（化学技术研究所-国家化学与生物化学研究所））；NOVA School of Science and Technology, Universidade NOVA de Lisboa（新里斯本大学新科学与技术学院）
A framework for assessing the matrix variate normality of three-way data is developed. The framework comprises a visual method and a goodness of fit test based on the Mahalanobis squared distance (MSD). The MSD of multivariate and matrix variate normal estimators, respectively, are used as an assessment tool for matrix variate normality. Specifically, these are used in the form of a distance-distance (DD) plot as a graphical method for visualizing matrix variate normality. In addition, we employ the popular Kolmogorov-Smirnov goodness of fit test in the context of assessing matrix variate norm…
Statistical disclosure limitation (SDL) methods aim to provide analysts general access to a data set while limiting the risk of disclosure of individual records. Many methods in the existing literature are aimed only at the case of univariate distributions, but the multivariate case is crucial, since most statistical analyses are multivariate in nature. Yet preserving the multivariate structure of the data can be challenging, especially when both continuous and categorical variables are present. Here we present a new SDL method that automatically attains the correct multivariate structure, reg…
Confidence intervals are central to statistical inference as a tool to evaluate the type I error risk at a given significance level. We devise a method to construct confidence intervals using a single run of a permutation test. This methodology is extended to a multivariate setting, where we are able to handle multiple testing under arbitrary dependence. We demonstrate the method on a weather data set and in a simulation example.
In this paper we provide new methodology for inference of the geometric features of a multivariate density in deconvolution. Our approach is based on multiscale tests to detect significant directional derivatives of the unknown density at arbitrary points in arbitrary directions. The multiscale method is used to identify regions of monotonicity and to construct a general procedure for the detection of modes of the multivariate density. Moreover, as an important application a significance test for the presence of a local maximum at a pre-specified point is proposed. The performance of the new m…
We present a new statistical test to detect that a presented state of a reversible Markov chain was not chosen from a stationary distribution. In particular, given a value function for the states of the Markov chain, we would like to demonstrate rigorously that the presented state is an outlier with respect to the values, by establishing a $p$-value for observations we make about the state under the null hypothesis that it was chosen uniformly at random. A simple heuristic used in practice is to sample ranks of states from long random trajectories on the Markov chain, and compare these to the …
We consider the problem of testing significance of predictors in multivariate nonparametric quantile regression. A stochastic process is proposed, which is based on a comparison of the responses with a nonparametric quantile regression estimate under the null hypothesis. It is demonstrated that under the null hypothesis this process converges weakly to a centered Gaussian process and the asymptotic properties of the test under fixed and local alternatives are also discussed. In particular we show, that - in contrast to the nonparametric approach based on estimation of $L^2$-distances - the new…
A Bayesian lattice filtering and smoothing approach is proposed for fast and accurate modeling and inference in multivariate non-stationary time series. This approach offers computational feasibility and interpretable time-frequency analysis in the multivariate context. The proposed framework allows us to obtain posterior estimates of the time-varying spectral densities of individual time series components, as well as posterior measurements of the time-frequency relationships across multiple components, such as time-varying coherence and partial coherence. The proposed formulation considers mu…
Clustering methods have led to a number of important discoveries in bioinformatics and beyond. A major challenge in their use is determining which clusters represent important underlying structure, as opposed to spurious sampling artifacts. This challenge is especially serious, and very few methods are available, when the data are very high in dimension. Statistical Significance of Clustering (SigClust) is a recently developed cluster evaluation tool for high dimensional low sample size data. An important component of the SigClust approach is the very definition of a single cluster as a subset…
We derive tests of stationarity for univariate time series by combining change-point tests sensitive to changes in the contemporary distribution with tests sensitive to changes in the serial dependence. The proposed approach relies on a general procedure for combining dependent tests based on resampling. After proving the asymptotic validity of the combining procedure under the conjunction of null hypotheses and investigating its consistency, we study rank-based tests of stationarity by combining cumulative sum change-point tests based on the contemporary empirical distribution function and on…
Scuola Normale Superiore（意大利高等师范学院）；IMT School for Advanced Studies（IMT高等研究院）；Institute for Advanced Study, University of Amsterdam（阿姆斯特丹大学高级研究所）；Korteweg-de Vries Institute for Mathematics, University of Amsterdam（阿姆斯特丹大学科特韦格-德弗里斯数学研究所）；Istituto Nazionale di Alta Matematica ‘Francesco Severi’ (INdAM-GNAMPA)（弗朗切斯科·塞韦里高等数学国家研究所）
Many data producers seek to provide users access to confidential data without unduly compromising data subjects' privacy and confidentiality. One general strategy is to require users to do analyses without seeing the confidential data; for example, analysts only get access to synthetic data or query systems that provide disclosure-protected outputs of statistical models. With synthetic data or redacted outputs, the analyst never really knows how much to trust the resulting findings. In particular, if the user did the same analysis on the confidential data, would regression coefficients of inte…
A new non-parametric test for multivariate paired data from pair matching or paired designs
School of Data Science, Fudan University（复旦大学数据科学学院）；Department of Statistics, University of California, Davis（加州大学戴维斯分校统计系）；Beijing International Center for Mathematical Research and School of Public Health, Peking University（北京大学北京国际数学研究中心与公共卫生学院）
Assessing variability according to distinct factors in data is a fundamental technique of statistics. The method commonly regarded to as analysis of variance (ANOVA) is, however, typically confined to the case where all levels of a factor are present in the data (i.e. the population of factor levels has been exhausted). Random and mixed effects models are used for more elaborate cases, but require distinct nomenclature, concepts and theory, as well as distinct inferential procedures. Following a hierarchical Bayesian approach, a comprehensive ANOVA framework is shown, which unifies the above s…
Time series datasets often contain heterogeneous signals, composed of both continuously changing quantities and discretely occurring events. The coupling between these measurements may provide insights into key underlying mechanisms of the systems under study. To better extract this information, we investigate the asymptotic statistical properties of coupling measures between continuous signals and point processes. We first introduce martingale stochastic integration theory as a mathematical model for a family of statistical quantities that include the Phase Locking Value, a classical coupling…
We consider the testing and estimation of change-points, locations where the distribution abruptly changes, in a sequence of multivariate or non-Euclidean observations. We study a nonparametric framework that utilizes similarity information among observations, which can be applied to various data types as long as an informative similarity measure on the sample space can be defined. The existing approach along this line has low power and/or biased estimates for change-points under some common scenarios. We address these problems by considering new tests based on similarity information. Simulati…
Linear functions of the site frequency spectrum (SFS) play a major role for understanding and investigating genetic diversity. Estimators of the mutation rate (e.g. based on the total number of segregating sites or average of the pairwise differences) and tests for neutrality (e.g. Tajima's D) are perhaps the most well-known examples. The distribution of linear functions of the SFS is important for constructing confidence intervals for the estimators, and to determine significance thresholds for neutrality tests. These distributions are often approximated using simulation procedures. In this p…
There have been increased concerns that the use of statins, one of the most commonly prescribed drugs for treating coronary artery disease, is potentially associated with the increased risk of new-onset type II diabetes (T2D). Nevertheless, to date, there is no robust evidence supporting as to whether and what kind of populations are indeed vulnerable for developing T2D after taking statins. In this case study, leveraging the biobank and electronic health record data in the Partner Health System, we introduce a new data analysis pipeline and a novel statistical methodology that address existin…
Accurate knowledge of the null distribution of hypothesis tests is important for valid application of the tests. In previous papers and software, the asymptotic null distribution of likelihood ratio tests for detecting genetic linkage in multivariate variance components models has been stated to be a mixture of chi-square distributions with binomial mixing probabilities. Here we show, by simulation and by theoretical arguments based on the geometry of the parameter space, that all aspects of the previously stated asymptotic null distribution are incorrect--both the binomial mixing probabilitie…
In genetic association studies, detecting phenotype-genotype association is a primary goal. We assume that the relationship between the data -phenotype, genetic markers and environmental covariates - can be modelled by a generalized linear model (GLM). The inclusion of environmental covariates makes it possible to account for important confounding factors, such as sex and population substructure. A multivariate score statistic, which under the complete null hypothesis of no phenotype-genotype association asymptotically has a multivariate normal distribution with a covariance matrix that can be…
Spatial Principal Component Analysis and Moran Statistics for Multivariate Functional Areal Data
Institute of Mathematical Sciences, Faculty of Science, Universiti Malaya（马来亚大学理学院数学科学研究所）；Universiti Malaya Centre for Data Analytics, Universiti Malaya（马来亚大学数据分析中心）；Center of Research for Statistical Modelling and Methodology, Faculty of Science, Universiti Malaya（马来亚大学理学院统计建模与方法研究中心）；Institut de Mathématiques de Bordeaux, UMR 5251, CNRS, Université de Bordeaux, Bordeaux INP（波尔多大学波尔多数学研究所）；Mathematics, Division of Science, New York University Abu Dhabi（纽约大学阿布扎比分校理学院数学系）；CNRS, UMR 8524–Laboratoire Paul Painlevé, Inria-Datavers, Université de Lille（里尔大学保罗·潘勒瓦数学研究中心）；Independent Researcher（独立研究者）
The so-called gut-brain axis has stimulated extensive research on microbiomes. One focus is to assess the association between certain clinical outcomes and the relative abundances of gut microbes, which can be presented as sub-compositional data in conformity with the taxonomic hierarchy of bacteria. Motivated by a study for identifying the microbes in the gut microbiome of preterm infants that impact their later neurobehavioral outcomes, we formulate a constrained integrative multi-view regression, where the neurobehavioral scores form multivariate response, the sub-compositional microbiome d…
Mediation analysis seeks to identify and quantify the paths by which an exposure affects an outcome. Intermediate variables which are effected by the exposure and which effect the outcome are known as mediators. There exists extensive work on mediation analysis in the context of models with a single mediator and continuous and binary outcomes. However these methods are often not suitable for multi-omic data that include highly interconnected variables measuring biological mechanisms and various types of outcome variables such as censored survival responses. In this article, we develop a genera…
Experimental research on behavior and cognition frequently rests on stimulus or subject selection where not all characteristics can be fully controlled, even when attempting strict matching. For example, when contrasting patients to controls, variables such as intelligence or socioeconomic status are often correlated with patient status. Similarly, when presenting word stimuli, variables such as word frequency are often correlated with primary variables of interest. One procedure very commonly employed to control for such nuisance effects is conducting inferential tests on confounding stimulus…
1. Sample size estimation through power analysis is a fundamental tool in planning an ecological study, yet there are currently no well-established procedures for when multivariate abundances are to be collected. A power analysis procedure would need to address three challenges: designing a parsimonious simulation model that captures key community data properties; measuring effect size in a realistic yet interpretable fashion; and ensuring computational feasibility when simulation is used both for power estimation and significance testing. 2. Here we propose a power analysis procedure that add…
