Missing values are prevalent in multivariate time series, compromising the integrity of analyses and degrading the performance of downstream tasks. Consequently, research has focused on multivariate time series imputation, aiming to accurately impute the missing values based on available observations. A key research question is how to ensure imputation consistency, i.e., intra-consistency between observed and imputed values, and inter-consistency between adjacent windows after imputation. However, previous methods rely solely on the inductive bias of the imputation targets to guide the learning process, ignoring imputation consistency and ultimately resulting in poor performance. Diffusion models, known for their powerful generative abilities, prefer to generate consistent results based on available observations. Therefore, we propose a conditional diffusion model for Multivariate Time Series Consistent Imputation (MTSCI). Specifically, MTSCI employs a contrastive complementary mask to generate dual views during the forward noising process. Then, the intra contrastive loss is calculated to ensure intra-consistency between the imputed and observed values. Meanwhile, MTSCI utilizes a mixup mechanism to incorporate conditional information from adjacent windows during the denoising process, facilitating the inter-consistency between imputed samples. Extensive experiments on multiple real-world datasets demonstrate that our method achieves the state-of-the-art performance on multivariate time series imputation task under different missing scenarios. Code is available at https://github.com/JeremyChou28/MTSCI.
Classification of large multivariate time series with strong class imbalance is an important task in real-world applications. Standard methods of class weights, oversampling, or parametric data augmentation do not always yield significant improvements for predicting minority classes of interest. Non-parametric data augmentation with Generative Adversarial Networks (GANs) offers a promising solution. We propose Imputation Balanced GAN (IB-GAN), a novel method that joins data augmentation and classification in a one-step process via an imputation-balancing approach. IB-GAN uses imputation and re…
Columbia University（哥伦比亚大学）；Mitsubishi Electric（三菱电机）
Many real-world applications involve multivariate, geo-tagged time series data: at each location, multiple sensors record corresponding measurements. For example, air quality monitoring system records PM2.5, CO, etc. The resulting time-series data often has missing values due to device outages or communication errors. In order to impute the missing values, state-of-the-art methods are built on Recurrent Neural Networks (RNN), which process each time stamp sequentially, prohibiting the direct modeling of the relationship between distant time stamps. Recently, the self-attention mechanism has be…
Time series data are ubiquitous in real-world applications. However, one of the most common problems is that the time series data could have missing values by the inherent nature of the data collection process. So imputing missing values from multivariate (correlated) time series data is imperative to improve a prediction performance while making an accurate data-driven decision. Conventional works for imputation simply delete missing values or fill them based on mean/zero. Although recent works based on deep neural networks have shown remarkable results, they still have a limitation to captur…
IST, ULisbon（里斯本大学高等理工学院）；Instituto de Telecomunicações（葡萄牙电信研究所）
Multivariate time series is a very active topic in the research community and many machine learning tasks are being used in order to extract information from this type of data. However, in real-world problems data has missing values, which may difficult the application of machine learning techniques to extract information. In this paper we focus on the task of imputation of time series. Many imputation methods for time series are based on regression methods. Unfortunately, these methods perform poorly when the variables are categorical. To address this case, we propose a new imputation method …
Probabilistic Computing Project Massachusetts Institute of Technology（麻省理工学院概率计算项目）
This article proposes a Bayesian nonparametric method for forecasting, imputation, and clustering in sparsely observed, multivariate time series data. The method is appropriate for jointly modeling hundreds of time series with widely varying, non-stationary dynamics. Given a collection of $N$ time series, the Bayesian model first partitions them into independent clusters using a Chinese restaurant process prior. Within a cluster, all time series are modeled jointly using a novel "temporally-reweighted" extension of the Chinese restaurant process mixture. Markov chain Monte Carlo techniques are…
Score-CDM: Score-Weighted Convolutional Diffusion Model for Multivariate Time Series Imputation
Central South University（中南大学）；Aalborg University（奥尔堡大学）；The Hong Kong Polytechnic University（香港理工大学）；National University of Defense Technology（国防科技大学）
ETH Zürich（苏黎世联邦理工学院）；Yandex（俄罗斯搜索引擎公司）；UC Irvine（加州大学欧文分校）
Multivariate time series with missing values are common in areas such as healthcare and finance, and have grown in number and complexity over the years. This raises the question whether deep learning methodologies can outperform classical data imputation methods in this domain. However, naive applications of deep learning fall short in giving reliable confidence estimates and lack interpretability. We propose a new deep sequential latent variable model for dimensionality reduction and data imputation. Our modeling assumption is simple and interpretable: the high dimensional time series has a l…
Sloan School of Management and Operations Research Center（斯隆管理学院与运筹学研究中心）；Massachusetts Institute of Technology（麻省理工学院）
Missing data is a common problem in real-world settings and particularly relevant in healthcare applications where researchers use Electronic Health Records (EHR) and results of observational studies to apply analytics methods. This issue becomes even more prominent for longitudinal data sets, where multiple instances of the same individual correspond to different observations in time. Standard imputation methods do not take into account patient specific information incorporated in multivariate panel data. We introduce the novel imputation algorithm MedImpute that addresses this problem, exten…
We propose computationally efficient methods for estimating stationary multivariate spatial and spatial-temporal spectra from incomplete gridded data. The methods are iterative and rely on successive imputation of data and updating of model estimates. Imputations are done according to a periodic model on an expanded domain. The periodicity of the imputations is a key feature that reduces edge effects in the periodogram and is facilitated by efficient circulant embedding techniques. In addition, we describe efficient methods for decomposing the estimated cross spectral density function into a l…
Rochester Data Science Consortium（罗切斯特数据科学联盟）；University of Rochester（罗切斯特大学）
We consider the topic of data imputation, a foundational task in machine learning that addresses issues with missing data. To that end, we propose MCFlow, a deep framework for imputation that leverages normalizing flow generative models and Monte Carlo sampling. We address the causality dilemma that arises when training models with incomplete data by introducing an iterative learning scheme which alternately updates the density estimate and the values of the missing entries in the training data. We provide extensive empirical validation of the effectiveness of the proposed method on standard m…
Time series prediction has been a long-standing research topic and an essential application in many domains. Modern time series collected from sensor networks (e.g., energy consumption and traffic flow) are often large-scale and incomplete with considerable corruption and missing values, making it difficult to perform accurate predictions. In this paper, we propose a low-rank autoregressive tensor completion (LATC) framework to model multivariate time series data. The key of LATC is to transform the original multivariate time series matrix (e.g., sensor$\times$time point) to a third-order tens…
Synthetic data for ratemaking: imputation-based methods vs adversarial networks and autoencoders
Department of Actuarial Science, Faculty of Business and Economics, University of Lausanne（卢塞恩大学商学院精算科学系）；Institute of Mathematics and Statistics, Faculty of Science and Technology, University of Tartu（塔尔图大学科学与技术学院数学与统计学研究所）
Exposure to air pollution is associated with increased morbidity and mortality. Recent technological advancements permit the collection of time-resolved personal exposure data. Such data are often incomplete with missing observations and exposures below the limit of detection, which limit their use in health effects studies. In this paper we develop an infinite hidden Markov model for multiple asynchronous multivariate time series with missing data. Our model is designed to include covariates that can inform transitions among hidden states. We implement beam sampling, a combination of slice sa…
The presence of missing values within high-dimensional data is an ubiquitous problem for many applied sciences. A serious limitation of many available data mining and machine learning methods is their inability to handle partially missing values and so an integrated approach that combines imputation and model estimation is vital for down-stream analysis. A computationally fast algorithm, called EMFlow, is introduced that performs imputation in a latent space via an online version of Expectation-Maximization (EM) algorithm by using a normalizing flow (NF) model which maps the data space to a la…
UiT The Arctic University of Norway（挪威北极大学）；Universidad Rey Juan Carlos（胡安·卡洛斯国王大学）；University Hospital of North Norway (UNN)（挪威北部大学医院）
A large fraction of the electronic health records consists of clinical measurements collected over time, such as blood tests, which provide important information about the health status of a patient. These sequences of clinical measurements are naturally represented as time series, characterized by multiple variables and the presence of missing data, which complicate analysis. In this work, we propose a surgical site infection detection framework for patients undergoing colorectal cancer surgery that is completely unsupervised, hence alleviating the problem of getting access to labelled traini…
An End-to-End Model for Time Series Classification In the Presence of Missing Values
School of Computer Science and Engineering, Tianjin University of Technology（天津理工大学计算机科学与工程学院）；Technical University of Denmark（丹麦技术大学）；Aalborg University（奥尔堡大学）
CoIFNet: A Unified Framework for Multivariate Time Series Forecasting with Missing Values
School of Computing and Artificial Intelligence, Southwest Jiaotong University（计算机与人工智能学院，西南交通大学）；School of Mathematics, Southwest Jiaotong University（数学学院，西南交通大学）；Institute Carbon Neutrality, Tsinghua University（碳中和研究院，清华大学）；White Rose Grid e-Science Centre, University of Leeds（白玫瑰网格e科学中心，利兹大学）
Multivariate time-series data are used in many classification and regression predictive tasks, and recurrent models have been widely used for such tasks. Most common recurrent models assume that time-series data elements are of equal length and the ordered observations are recorded at regular intervals. However, real-world time-series data have neither a similar length nor a same number of observations. They also have missing entries, which hinders the performance of predictive tasks. In this paper, we approach these issues by presenting a model for the combined task of imputing and predicting…
Boundary-enhanced time series data imputation with long-term dependency diffusion models
Henan University（河南大学）；Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China（电子科技大学长三角研究院（衢州））；University College London（伦敦大学学院）
UiT The Arctic University of Norway（挪威北极大学）；University Hospital of North-Norway（挪威北部大学医院）；Rey Juan Carlos University（雷·胡安·卡洛斯大学）
A large fraction of the electronic health records (EHRs) consists of clinical measurements collected over time, such as lab tests and vital signs, which provide important information about a patient's health status. These sequences of clinical measurements are naturally represented as time series, characterized by multiple variables and large amounts of missing data, which complicate the analysis. In this work, we propose a novel kernel which is capable of exploiting both the information from the observed values as well the information hidden in the missing patterns in multivariate time series…
Informative missingness is unavoidable in the digital processing of continuous time series, where the value for one or more observations at different time points are missing. Such missing observations are one of the major limitations of time series processing using deep learning. Practical applications, e.g., sensor data, healthcare, weather, generates data that is in truth continuous in time, and informative missingness is a common phenomenon in these datasets. These datasets often consist of multiple variables, and often there are missing values for one or many of these variables. This chara…
University of Southern California（南加州大学）；New York University（纽约大学）
Multivariate time series data in practical applications, such as health care, geoscience, and biology, are characterized by a variety of missing values. In time series prediction and other related tasks, it has been noted that missing values and their missing patterns are often correlated with the target labels, a.k.a., informative missingness. There is very limited work on exploiting the missing patterns for effective imputation and improving prediction performance. In this paper, we develop novel deep learning models, namely GRU-D, as one of the early attempts. GRU-D is based on Gated Recurr…
Teikametrics；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；International Institute for Applied Systems Analysis（国际应用系统分析研究所）
We present an approach that uses a deep learning model, in particular, a MultiLayer Perceptron (MLP), for estimating the missing values of a variable in multivariate time series data. We focus on filling a long continuous gap (e.g., multiple months of missing daily observations) rather than on individual randomly missing observations. Our proposed gap filling algorithm uses an automated method for determining the optimal MLP model architecture, thus allowing for optimal prediction performance for the given time series. We tested our approach by filling gaps of various lengths (three months to …
Carnegie Mellon University（卡内基梅隆大学）；AWS AI Labs（AWS人工智能实验室）
In this work, we tackle two widespread challenges in real applications for time-series forecasting that have been largely understudied: distribution shifts and missing data. We propose SpectraNet, a novel multivariate time-series forecasting model that dynamically infers a latent space spectral decomposition to capture current temporal dynamics and correlations on the recent observed history. A Convolution Neural Network maps the learned representation by sequentially mixing its components and refining the output. Our proposed approach can simultaneously produce forecasts and interpolate past …
The Swiss AI Lab IDSIA, Università della Svizzera italiana（瑞士人工智能实验室IDSIA，瑞士意大利语区大学）；Politecnico di Milano（米兰理工大学）
Dealing with missing values and incomplete time series is a labor-intensive, tedious, inevitable task when handling data coming from real-world applications. Effective spatio-temporal representations would allow imputation methods to reconstruct missing temporal data by exploiting information coming from sensors at different locations. However, standard methods fall short in capturing the nonlinear time and space dependencies existing within networks of interconnected sensors and do not take full advantage of the available - and often strong - relational information. Notably, most state-of-the…
Deep Learning for Multivariate Time Series Imputation: A Survey
PyPOTS Research（PyPOTS研究机构）；Hong Kong University of Science and Technology（香港理工大学）；University of Oxford（牛津大学）；King’s College London（伦敦大学学院）；Huawei Noah’s Ark Lab（华为诺亚实验室）；Hong Kong University of Science and Technology (Guangzhou)（香港理工大学（广州））；Squirrel Ai Learning（Squirrel Ai学习机构）
Temporal Gaussian Copula For Clinical Multivariate Time Series Data Imputation
Chongqing Institute of Green and Intelligent Technology, CAS（中国科学院重庆绿色智能技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Singapore Management University（新加坡管理大学）；Southwest University（西南大学）
SEGAN: semi-supervised learning approach for missing data imputation
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Software Technology, Zhejiang University（浙江大学软件学院）；Binjiang Institute of Zhejiang University（浙江大学滨江研究院）；China Academy of Space Technology（中国空间技术研究院）；Beijing Life Science Academy（北京生命科学研究院）；Zhengzhou Tobacco Research Institute of China National Tobacco Corporation（中国烟草总公司郑州烟草研究院）；Purdue University（普渡大学）；COSMOPlat IoT Technology Co.,Ltd.（卡奥斯物联科技有限公司）
We present a multivariate hierarchical space-time model to describe the joint series of monthly extreme temperatures and amounts of rainfall. Data are available for 360 monitoring stations over 60 years, with missing data affecting almost all series. Model components account for spatio-temporal dependence with annual cycles, dependence on covariates and between responses. The very large amount of data is tackled modeling the spatio-temporal dependence by the nearest neighbor Gaussian process. Response multivariate dependencies are described using the linear model of coregionalization, while an…
EMFusion: Uncertainty-Aware Conditional Diffusion Model for Multivariate Narrow-band Exposure Forecasting
department of Electrical Engineering and Computer Science, York University（电气工程与计算机科学系，约克大学）；School of Electrical and Electronic Engineering, Huazhong University of Science and Technology（电子与电气工程学院，华中科技大学）；Central China Branch of State Grid Corporation of China（国家电网公司中部分部）；Department of Electronic Engineering, University of Rome Tor Vergata（罗马大学Tor Vergata电子工程系）；Consorzio Nazionale Interuniversitario per le Telecomunicazioni (CNIT)（国家大学间电信研究会（CNIT））
In this paper, we predict severity of extreme weather events (tropical storms, hurricanes, etc.) using buoy data time series variables such as wind speed and air temperature. The prediction/forecasting method is based on various forecasting and machine learning models. The following steps are used. Data sources for the buoys and weather events are identified, aggregated and merged. For missing data imputation, we use Kalman filters as well as splines for multivariate time series. Then, statistical tests are run to ascertain increasing trends in weather event severity. Next, we use machine lear…
