The signature transform is a 'universal nonlinearity' on the space of continuous vector-valued paths, and has received attention for use in machine learning on time series. However, real-world temporal data is typically observed at discrete points in time, and must first be transformed into a continuous path before signature techniques can be applied. We make this step explicit by characterising it as an imputation problem, and empirically assess the impact of various imputation strategies when applying signature-based neural nets to irregular time series data. For one of these strategies, Gaussian process (GP) adapters, we propose an extension~(GP-PoM) that makes uncertainty information directly available to the subsequent classifier while at the same time preventing costly Monte-Carlo (MC) sampling. In our experiments, we find that the choice of imputation drastically affects shallow signature models, whereas deeper architectures are more robust. Next, we observe that uncertainty-aware predictions (based on GP-PoM or indicator imputations) are beneficial for predictive performance, even compared to the uncertainty-aware training of conventional GP adapters. In conclusion, we have demonstrated that the path construction is indeed crucial for signature models and that our proposed strategy leads to competitive performance in general, while improving robustness of signature models in particular.
Electronic health records (EHR) consist of longitudinal clinical observations portrayed with sparsity, irregularity, and high-dimensionality, which become major obstacles in drawing reliable downstream clinical outcomes. Although there exist great numbers of imputation methods to tackle these issues, most of them ignore correlated features, temporal dynamics and entirely set aside the uncertainty. Since the missing value estimates involve the risk of being inaccurate, it is appropriate for the method to handle the less certain information differently than the reliable data. In that regard, we …
Multivariate time-series data are used in many classification and regression predictive tasks, and recurrent models have been widely used for such tasks. Most common recurrent models assume that time-series data elements are of equal length and the ordered observations are recorded at regular intervals. However, real-world time-series data have neither a similar length nor a same number of observations. They also have missing entries, which hinders the performance of predictive tasks. In this paper, we approach these issues by presenting a model for the combined task of imputing and predicting…
Key Laboratory of Machine Perception (Ministry of Education), Peking University（北京大学教育部机器感知重点实验室）；School of Electronics Engineering and Computer Science, Peking University（北京大学电子工程与计算机科学学院）；National Institute of Health Data Science, Peking University（北京大学国家健康数据科学研究院）；Institute of Medical Technology, Health Science Center, Peking University（北京大学医学部医学技术研究所）
Irregularly sampled time series (ISTS) data has irregular temporal intervals between observations and different sampling rates between sequences. ISTS commonly appears in healthcare, economics, and geoscience. Especially in the medical environment, the widely used Electronic Health Records (EHRs) have abundant typical irregularly sampled medical time series (ISMTS) data. Developing deep learning methods on EHRs data is critical for personalized treatment, precise diagnosis and medical management. However, it is challenging to directly use deep learning models for ISMTS data. On the one hand, I…
College of Information and Computer Sciences（信息与计算机科学学院）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
Missing values, irregularly collected samples, and multi-resolution signals commonly occur in multivariate time series data, making predictive tasks difficult. These challenges are especially prevalent in the healthcare domain, where patients' vital signs and electronic records are collected at different frequencies and have occasionally missing information due to the imperfections in equipment or patient circumstances. Researchers have handled each of these issues differently, often handling missing data through mean value imputation and then using sequence models over the multivariate signal…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Time series imputation is a fundamental task for understanding time series with missing data. Existing methods either do not directly handle irregularly-sampled data or degrade severely with sparsely observed data. In this work, we reformulate time series as permutation-equivariant sets and propose a novel imputation model NRTSI that does not impose any recurrent structures. Taking advantage of the permutation equivariant formulation, we design a principled and efficient hierarchical imputation procedure. In addition, NRTSI can directly handle irregularly-sampled time series, perform multiple-…
We present a simple yet novel time series imputation technique with the goal of constructing an irregular time series that is uniform across every sample in a data set. Specifically, we fix a grid defined by the midpoints of non-overlapping bins (dubbed "slices") of observation times and ensure that each sample has values for all of the features at that given time. This allows one to both impute fully missing observations to allow uniform time series classification across the entire data and, in special cases, to impute individually missing features. To do so, we slightly generalize the well-k…
Stanford School of Medicine（斯坦福大学医学院）；Stanford University（斯坦福大学）
There is an emerging trend in the reinforcement learning for healthcare literature. In order to prepare longitudinal, irregularly sampled, clinical datasets for reinforcement learning algorithms, many researchers will resample the time series data to short, regular intervals and use last-observation-carried-forward (LOCF) imputation to fill in these gaps. Typically, they will not maintain any explicit information about which values were imputed. In this work, we (1) call attention to this practice and discuss its potential implications; (2) propose an alternative representation of the patient …
University of California, San Diego（加利福尼亚大学圣迭戈分校）；USC Information Sciences Institute（南加州大学信息科学研究所）；Children’s Hospital LA（洛杉矶儿童医院）
We demonstrate a simple strategy to cope with missing data in sequential inputs, addressing the task of multilabel classification of diagnoses given clinical time series. Collected from the pediatric intensive care unit (PICU) at Children's Hospital Los Angeles, our data consists of multivariate time series of observations. The measurements are irregularly spaced, leading to missingness patterns in temporally discretized sequences. While these artifacts are typically handled by imputation, we achieve superior predictive performance by treating the artifacts as features. Unlike linear models, r…
Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patching
Oxford-Man Institute, University of Oxford（牛津大学牛津-曼研究所）；Machine Learning Research Group, University of Oxford（牛津大学机器学习研究组）；Mathematical Institute, University of Oxford（牛津大学数学研究所）
University of Warwick（华威大学）；Alan Turing Institute（艾伦·图灵研究所）；University of Oxford（牛津大学）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织数据61中心）
Distribution regression refers to the supervised learning problem where labels are only available for groups of inputs instead of individual inputs. In this paper, we develop a rigorous mathematical framework for distribution regression where inputs are complex data streams. Leveraging properties of the expected signature and a recent signature kernel trick for sequential data from stochastic analysis, we introduce two new learning techniques, one feature-based and the other kernel-based. Each is suited to a different data regime in terms of the number of data streams and the dimensionality of…
Collaborative Imputation of Urban Time Series through Cross-city Meta-learning
Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University（香港理工大学土木与环境工程系）；Department of Traffic Engineering, Tongji University（同济大学交通工程系）；Centre for Learning, Teaching, and Technology, The Education University of Hong Kong（香港教育大学学习、教学与科技中心）；Research Institute for Artificial Intelligence of Things, Department of Computing, The Hong Kong Polytechnic University（香港理工大学人工智能物联网研究所）
Missing data and noisy observations pose significant challenges for reliably predicting events from irregularly sampled multivariate time series (longitudinal) data. Imputation methods, which are typically used for completing the data prior to event prediction, lack a principled mechanism to account for the uncertainty due to missingness. Alternatively, state-of-the-art joint modeling techniques can be used for jointly modeling the longitudinal and event data and compute event probabilities conditioned on the longitudinal observations. These approaches, however, make strong parametric assumpti…
Deep Survival Analysis for Competing Risk Modeling with Functional Covariates and Missing Data Imputation
Department of Quantitative Health Science, Cleveland Clinic, Cleveland OH, USA（定量健康科学系，克利夫兰诊所）；Department of Pulmonary and Critical Care Medicine, Cleveland Clinic, Cleveland OH, USA（呼吸与重症医学科，克利夫兰诊所）；Department of Statistics, Purdue University, Lafayette IN, USA（统计系，普渡大学）
Temporal Gaussian Copula For Clinical Multivariate Time Series Data Imputation
Chongqing Institute of Green and Intelligent Technology, CAS（中国科学院重庆绿色智能技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Singapore Management University（新加坡管理大学）；Southwest University（西南大学）
École de Technologie Supérieure, Université du Québec（魁北克大学高等技术学院）
Data acquisition and recording in the form of databases are routine operations. The process of collecting data, however, may experience irregularities, resulting in databases with missing data. Missing entries might alter analysis efficiency and, consequently, the associated decision-making process. This paper focuses on databases collecting information related to the natural environment. Given the broad spectrum of recorded activities, these databases typically are of mixed nature. It is therefore relevant to evaluate the performance of missing data processing methods considering this charact…
Irregularly-sampled time series (ITS) are native to high-impact domains like healthcare, where measurements are collected over time at uneven intervals. However, for many classification problems, only small portions of long time series are often relevant to the class label. In this case, existing ITS models often fail to classify long series since they rely on careful imputation, which easily over- or under-samples the relevant regions. Using this insight, we then propose CAT, a model that classifies multivariate ITS by explicitly seeking highly-relevant portions of an input series' timeline. …
Electronic health record (EHR) data is sparse and irregular as it is recorded at irregular time intervals, and different clinical variables are measured at each observation point. In this work, we propose a multi-view features integration learning from irregular multivariate time series data by self-attention mechanism in an imputation-free manner. Specifically, we devise a novel multi-integration attention module (MIAM) to extract complex information inherent in irregular time series data. In particular, we explicitly learn the relationships among the observed values, missing indicators, and …
Department of Computer Science, Saarland University（萨尔兰大学计算机科学系）；Department of Biology, University of Copenhagen（哥本哈根大学生物学系）；Novo Nordisk A/S（诺华北欧制药有限公司）；Department of Applied Mathematics and Computer Science, Technical University of Denmark（丹麦技术大学应用数学与计算机科学系）
University of Washington（华盛顿大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
In wearable sensing applications, data is inevitable to be irregularly sampled or partially missing, which pose challenges for any downstream application. An unique aspect of wearable data is that it is time-series data and each channel can be correlated to another one, such as x, y, z axis of accelerometer. We argue that traditional methods have rarely made use of both times-series dynamics of the data as well as the relatedness of the features from different sensors. We propose a model, termed as DynImp, to handle different time point's missingness with nearest neighbors along feature axis a…
Spatiotemporal System Forecasting with Irregular Time Steps via Masked Autoencoder
University College London（伦敦大学学院）；Imperial College London（帝国理工学院）；University of Liverpool（利物浦大学）；ENPC, EDF R&D, Institut Polytechnique de Paris（巴黎高科国立路桥学校，法国电力集团研发部，巴黎综合理工学院）
North Carolina State University（北卡罗来纳州立大学）；Intuit Inc.（财捷公司）
Real-world Electronic Health Records (EHRs) are often plagued by a high rate of missing data. In our EHRs, for example, the missing rates can be as high as 90% for some features, with an average missing rate of around 70% across all features. We propose a Time-Aware Dual-Cross-Visit missing value imputation method, named TA-DualCV, which spontaneously leverages multivariate dependencies across features and longitudinal dependencies both within- and cross-visit to maximize the information extracted from limited observable records in EHRs. Specifically, TA-DualCV captures the latent structure of…
