Time series are an interesting frontier for kernel-based methods, for the simple reason that there is no kernel designed to represent them and their unique characteristics in full generality. Existing sequential kernels ignore the time indices, with many assuming that the series must be regularly-spaced; some such kernels are not even psd. In this manuscript, we show that a "series kernel" that is general enough to represent irregularly-spaced multivariate time series may be built out of well-known "vector kernels". We also show that all series kernels constructed using our methodology are psd, and are thus widely applicable. We demonstrate this point by formulating a Gaussian process-based strategy - with our series kernel at its heart - to make predictions about test series when given a training set. We validate the strategy experimentally by estimating its generalisation error on multiple datasets and comparing it to relevant baselines. We also demonstrate that our series kernel may be used for the more traditional setting of time series classification, where its performance is broadly in line with alternative methods.
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；IBM Almaden Research Center（IBM阿尔马登研究中心）
Sparse and irregularly sampled multivariate time series are common in clinical, climate, financial and many other domains. Most recent approaches focus on classification, regression or forecasting tasks on such data. In forecasting, it is necessary to not only forecast the right value but also to forecast when that value will occur in the irregular time series. In this work, we present an approach to forecast not only the values but also the time at which they are expected to occur.
College of Information and Computer Sciences（信息与计算机科学学院）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
In this paper, we present a new deep learning architecture for addressing the problem of supervised learning with sparse and irregularly sampled multivariate time series. The architecture is based on the use of a semi-parametric interpolation network followed by the application of a prediction network. The interpolation network allows for information to be shared across multiple dimensions of a multivariate time series during the interpolation stage, while any standard deep learning model can be used for the prediction network. This work is motivated by the analysis of physiological time serie…
This paper develops empirical likelihood methodology for irregularly spaced spatial data in the frequency domain. Unlike the frequency domain empirical likelihood (FDEL) methodology for time series (on a regular grid), the formulation of the spatial FDEL needs special care due to lack of the usual orthogonality properties of the discrete Fourier transform for irregularly spaced data and due to presence of nontrivial bias in the periodogram under different spatial asymptotic structures. A spatial FDEL is formulated in the paper taking into account the effects of these factors. The main results …
A novel first-order moving-average model for analyzing time series observed at irregularly spaced intervals is introduced. Two definitions are presented, which are equivalent under Gaussianity. The first one relies on normally distributed data and the specification of second-order moments. The second definition provided is more flexible in the sense that it allows for considering other distributional assumptions. The statistical properties are investigated along with the one-step linear predictors and their mean squared errors. It is established that the process is strictly stationary under no…
Irregularly-sampled time series occur in many domains including healthcare. They can be challenging to model because they do not naturally yield a fixed-dimensional representation as required by many standard machine learning models. In this paper, we consider irregular sampling from the perspective of missing data. We model observed irregularly-sampled time series data as a sequence of index-value pairs sampled from a continuous but unobserved function. We introduce an encoder-decoder framework for learning from such generic indexed sequences. We propose learning methods for this framework ba…
Tongji University（同济大学）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；IBM Research（IBM研究院）；NEC Laboratories America（美国电气股份有限公司美洲实验室）
Multivariate time series (MTS) data are becoming increasingly ubiquitous in diverse domains, e.g., IoT systems, health informatics, and 5G networks. To obtain an effective representation of MTS data, it is not only essential to consider unpredictable dynamics and highly variable lengths of these data but also important to address the irregularities in the sampling rates of MTS. Existing parametric approaches rely on manual hyperparameter tuning and may cost a huge amount of labor effort. Therefore, it is desirable to learn the representation automatically and efficiently. To this end, we propo…
The discrete-time GARCH methodology which has had such a profound influence on the modelling of heteroscedasticity in time series is intuitively well motivated in capturing many `stylized facts' concerning financial series, and is now almost routinely used in a wide range of situations, often including some where the data are not observed at equally spaced intervals of time. However, such data is more appropriately analyzed with a continuous-time model which preserves the essential features of the successful GARCH paradigm. One possible such extension is the diffusion limit of Nelson, but this…
College of Information and Computer Sciences（信息与计算机科学学院）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
Irregularly sampled time series data arise naturally in many application domains including biology, ecology, climate science, astronomy, and health. Such data represent fundamental challenges to many classical models from machine learning and statistics due to the presence of non-uniform intervals between observations. However, there has been significant progress within the machine learning community over the last decade on developing specialized models and architectures for learning from irregularly sampled univariate and multivariate time series data. In this survey, we first describe severa…
College of Information and Computer Sciences（信息与计算机科学学院）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
While the volume of electronic health records (EHR) data continues to grow, it remains rare for hospital systems to capture dense physiological data streams, even in the data-rich intensive care unit setting. Instead, typical EHR records consist of sparse and irregularly observed multivariate time series, which are well understood to present particularly challenging problems for machine learning methods. In this paper, we present a new deep learning architecture for addressing this problem based on the use of a semi-parametric interpolation network followed by the application of a prediction n…
A novel first-order autoregressive moving average model for analyzing discrete-time series observed at irregularly spaced times is introduced. Under Gaussianity, it is established that the model is strictly stationary and ergodic. In the general case, it is shown that the model is weakly stationary. The lowest dimension of the state-space representation is given along with the one-step linear predictors and their mean squared errors. The maximum likelihood estimation procedure is discussed, and their finite-sample behavior is assessed through Monte Carlo experiments. These experiments show tha…
We present a novel approach for nonparametric regression using wavelet basis functions. Our proposal, $\texttt{waveMesh}$, can be applied to non-equispaced data with sample size not necessarily a power of 2. We develop an efficient proximal gradient descent algorithm for computing the estimator and establish adaptive minimax convergence rates. The main appeal of our approach is that it naturally extends to additive and sparse additive models for a potentially large number of covariates. We prove minimax optimal convergence rates under a weak compatibility condition for sparse additive models. …
For temporal regularly spaced datasets, a lot of methods are available and the properties of these methods are extensively investigated. Less research has been performed on irregular temporal datasets subject to measurement error with complex dependence structures, while this type of datasets is widely available. In this paper, the performance of kernel smoother for trend and its derivatives is considered under dependent measurement errors and irregularly spaced sampling scheme. The error processes are assumed to be subordinated Gaussian long memory processes and have varying marginal distribu…
In several disciplines it is common to find time series measured at irregular observational times. In particular, in astronomy there are a large number of surveys that gather information over irregular time gaps and in more than one passband. Some examples are Pan-STARRS, ZTF and also the LSST. However, current commonly used time series models that estimate the time dependency in astronomical light curves consider the information of each band separately (e.g, CIAR, IAR and CARMA models) disregarding the dependency that might exist between different passbands. In this paper we propose a novel b…
In this paper, we construct a sequence of discrete time stochastic processes that converges in probability and in the Skorokhod metric to a COGARCH(p,q) model. The result is useful for the estimation of the continuous model defined for irregularly spaced time series data. The estimation procedure is based on the maximization of a pseudo log-likelihood function and is implemented in the yuima package.
University of Oxford（牛津大学）；The Alan Turing Institute（艾伦·图灵研究所）
The shapelet transform is a form of feature extraction for time series, in which a time series is described by its similarity to each of a collection of `shapelets'. However it has previously suffered from a number of limitations, such as being limited to regularly-spaced fully-observed time series, and having to choose between efficient training and interpretability. Here, we extend the method to continuous time, and in doing so handle the general case of irregularly-sampled partially-observed multivariate time series. Furthermore, we show that a simple regularisation penalty may be used to t…
Earth and space observations meet complex algebras: from complex to octonions for multivariate autoregressive time series analysis
Faculty of Engineering and Science, Universidad Adolfo Ibáñez（阿道夫·伊巴涅斯大学工程与科学学院）；Data Observatory Foundation, ANID Technology Center No. DO210001（数据观测基金会，ANID技术中心No. DO210001）；Millennium Institute of Astrophysics, ICM-ANID ICN12_009（天体物理学千年研究所，ICM-ANID ICN12_009）；Department of Mathematics and Computer Science, Faculty of ScienceUniversidad de Santiago（圣地亚哥大学理学院数学与计算机科学系）
Mathematical Institute, University of Oxford（牛津大学数学研究所）；The Alan Turing Institute（艾伦·图灵研究所）
Neural ordinary differential equations are an attractive option for modelling temporal dynamics. However, a fundamental issue is that the solution to an ordinary differential equation is determined by its initial condition, and there is no mechanism for adjusting the trajectory based on subsequent observations. Here, we demonstrate how this may be resolved through the well-understood mathematics of \emph{controlled differential equations}. The resulting \emph{neural controlled differential equation} model is directly applicable to the general setting of partially-observed irregularly-sampled m…
University of California, San Diego（加利福尼亚大学圣迭戈分校）；USC Information Sciences Institute（南加州大学信息科学研究所）；Children’s Hospital LA（洛杉矶儿童医院）
We demonstrate a simple strategy to cope with missing data in sequential inputs, addressing the task of multilabel classification of diagnoses given clinical time series. Collected from the pediatric intensive care unit (PICU) at Children's Hospital Los Angeles, our data consists of multivariate time series of observations. The measurements are irregularly spaced, leading to missingness patterns in temporally discretized sequences. While these artifacts are typically handled by imputation, we achieve superior predictive performance by treating the artifacts as features. Unlike linear models, r…
Feedzai（Feedzai公司）；Instituto Superior Técnico, Universidade de Lisboa（里斯本大学高等技术学院）；DCC, Faculdade de Ciêncidas da Universidade do Porto（波尔图大学理学院数据与计算机科学系）；Instituto de Telecomunicações（电信研究所）
Electronic records contain sequences of events, some of which take place all at once in a single visit, and others that are dispersed over multiple visits, each with a different timestamp. We postulate that fine temporal detail, e.g., whether a series of blood tests are completed at once or in rapid succession should not alter predictions based on this data. Motivated by this intuition, we propose models for analyzing sequences of multivariate clinical time series data that are invariant to this temporal clustering. We propose an efficient data augmentation technique that exploits the postulat…
High-dimensional multivariate spatial-temporal data arise frequently in a wide range of applications; however, there are relatively few statistical methods that can simultaneously deal with spatial, temporal and variable-wise dependencies in large data sets. In this paper, we propose a new approach to utilize the correlations in variable, space and time to achieve dimension reduction and to facilitate spatial/temporal predictions in the high-dimensional settings. The multivariate spatial-temporal process is represented as a linear transformation of a lower-dimensional latent factor process. Th…
In many environmental applications, time series are either incomplete or irregularly spaced. We investigate the application of the Spartan random process to missing data prediction. We employ a novel modified method of moments (MMoM) for parameter inference. The CPU time of MMoM is shown to be much faster than that of maximum likelihood estimation and almost independent of the data size. We formulate an explicit Spartan interpolator for estimating missing data. The model validation is performed on both synthetic data and real time series of atmospheric aerosol concentrations. The prediction pe…
Missing data and noisy observations pose significant challenges for reliably predicting events from irregularly sampled multivariate time series (longitudinal) data. Imputation methods, which are typically used for completing the data prior to event prediction, lack a principled mechanism to account for the uncertainty due to missingness. Alternatively, state-of-the-art joint modeling techniques can be used for jointly modeling the longitudinal and event data and compute event probabilities conditioned on the longitudinal observations. These approaches, however, make strong parametric assumpti…
University of Florida（佛罗里达大学）；Sepsis and Critical Illness Research Center（脓毒症与危重病研究中心）
Background: During the early stages of hospital admission, clinicians must use limited information to make diagnostic and treatment decisions as patient acuity evolves. However, it is common that the time series vital sign information from patients to be both sparse and irregularly collected, which poses a significant challenge for machine / deep learning techniques to analyze and facilitate the clinicians to improve the human health outcome. To deal with this problem, We propose a novel deep interpolation network to extract latent representations from sparse and irregularly sampled time-serie…
We propose a continuous-time Markov-switching generalized autoregressive conditional heteroskedasticity (COMS-GARCH) process for handling irregularly spaced time series (TS) with multiple volatilities states. We employ a Gibbs sampler in the Bayesian framework to estimate the COMS-GARCH model parameters, the latent state path and volatilities. To improve the inferential robustness and computational efficiency for obtaining the maximum a posteriori estimates for the state path and volatilities, we suggest a multi-path sampling scheme and incorporate the Bernoulli noise injection in the computat…
SELDON: Supernova Explosions Learned by Deep ODE Networks
NSF-Simons AI Institute for the Sky (SkAI)（NSF-模拟人工智能天空研究所（SkAI））；Department of Industrial Engineering and Management Sciences（工业工程与管理科学系）；Department of Astronomy（天文学系）；National Center for Supercomputing Applications (NCSA)（国家超级计算应用中心（NCSA））；Department of Physics and Astronomy（物理与天文学系）；Center for Interdisciplinary Exploration and Research in Astrophysics (CIERA)（天文学跨学科探索与研究中心（CIERA））；Department of Statistics and Data Science（统计学与数据科学系）
The Atmospheric Radiation Measurement program is a U.S. Department of Energy project that collects meteorological observations at several locations around the world in order to study how weather processes affect global climate change. As one of its initiatives, it operates a set of fixed but irregularly-spaced monitoring facilities in the Southern Great Plains region of the U.S. We describe methods for interpolating temperature records from these fixed facilities to locations at which no observations were made, which can be useful when values are required on a spatial grid. We interpolate by c…
VISTA-SSM: Varying and Irregular Sampling Time-series Analysis via State Space Models
Johns Hopkins University（约翰斯·霍普金斯大学）；Institute for Psycholinguistics and Digital Health, United States of America（美国心理语言学和健康数字研究所）；New York University School of Medicine（纽约大学医学院）；University of Houston（休斯顿大学）
In many fields observations are performed irregularly along time, due to either measurement limitations or lack of a constant immanent rate. While discrete-time Markov models (as Dynamic Bayesian Networks) introduce either inefficient computation or an information loss to reasoning about such processes, continuous-time Markov models assume either a discrete state space (as Continuous-Time Bayesian Networks), or a flat continuous state space (as stochastic differential equations). To address these problems, we present a new modeling class called Irregular-Time Bayesian Networks (ITBNs), general…
University of Cambridge（剑桥大学）；The Alan Turing Institute（艾伦·图灵研究所）；University of California Los Angeles（加州大学洛杉矶分校）
Counterfactual estimation using synthetic controls is one of the most successful recent methodological developments in causal inference. Despite its popularity, the current description only considers time series aligned across units and synthetic controls expressed as linear combinations of observed control units. We propose a continuous-time alternative that models the latent counterfactual path explicitly using the formalism of controlled differential equations. This model is directly applicable to the general setting of irregularly-aligned multivariate time series and may be optimized in ri…
