The problem of missing values in multivariable time series is a key challenge in many applications such as clinical data mining. Although many imputation methods show their effectiveness in many applications, few of them are designed to accommodate clinical multivariable time series. In this work, we propose a multiple imputation model that capture both cross-sectional information and temporal correlations. We integrate Gaussian processes with mixture models and introduce individualized mixing weights to handle the variance of predictive confidence of Gaussian process models. The proposed model is compared with several state-of-the-art imputation algorithms on both real-world and synthetic datasets. Experiments show that our best model can provide more accurate imputation than the benchmarks on all of our datasets.
Imputation is a popular technique for handling item nonresponse in survey sampling. Parametric imputation is based on a parametric model for imputation and is less robust against the failure of the imputation model. Nonparametric imputation is fully robust but is not applicable when the dimension of covariates is large due to the curse of dimensionality. Semiparametric imputation is another robust imputation based on a flexible model where the number of model parameters can increase with the sample size. In this paper, we propose another semiparametric imputation based on a more flexible model…
Item nonresponse is frequently encountered in practice. Ignoring missing data can lose efficiency and lead to misleading inference. Fractional imputation is a frequentist approach of imputation for handling missing data. However, the parametric fractional imputation of \cite{kim2011parametric} may be subject to bias under model misspecification. In this paper, we propose a novel semiparametric fractional imputation method using Gaussian mixture models. The proposed method is computationally efficient and leads to robust estimation. The proposed method is further extended to incorporate the cat…
This paper proposes a flexible Bayesian approach to multiple imputation using conditional Gaussian mixtures. We introduce novel shrinkage priors for covariate-dependent mixing proportions in the mixture models to automatically select the suitable number of components used in the imputation step. We develop an efficient sampling algorithm for posterior computation and multiple imputation via Markov Chain Monte Carlo methods. The proposed method can be easily extended to the situation where the data contains not only continuous variables but also discrete variables such as binary and count value…
Copula-based methods provide a flexible approach to build missing data imputation models of multivariate data of mixed types. However, the choice of copula function is an open question. We consider a Bayesian nonparametric approach by using an infinite mixture of elliptical copulas induced by a Dirichlet process mixture to build a flexible copula function. A slice sampling algorithm is used to sample from the infinite dimensional parameter space. We extend the work on prior parallel tempering used in finite mixture models to the Dirichlet process mixture model to overcome the mixing issue in m…
In clinical trials, mixed effects models for repeated measures (MMRM) and pattern mixture models (PMM) are often used to analyze longitudinal continuous outcomes. We describe a simple missing data imputation algorithm for the MMRM that can be easily implemented in standard statistical software packages such as SAS PROC MI. We explore the relationship of the missing data distribution in the control-based and delta-adjusted PMMs with that in the MMRM, and suggest an efficient imputation algorithm for these PMMs. The unobserved values in PMMs can be imputed by subtracting the mean difference in t…
University of Oxford（牛津大学）；CEREMADE, CNRS Université Paris-Dauphine（巴黎多芬大学CNRS-CEREMADE实验室）
We propose a variational autoencoder architecture to model both ignorable and nonignorable missing data using pattern-set mixtures as proposed by Little (1993). Our model explicitly learns to cluster the missing data into missingness pattern sets based on the observed data and missingness masks. Underpinning our approach is the assumption that the data distribution under missingness is probabilistically semi-supervised by samples from the observed data distribution. Our setup trades off the characteristics of ignorable and nonignorable missingness and can thus be applied to data of both types.…
TerraNIS；University of Toulouse（图卢兹大学）；IRIT-INP-ENSEEIHT（图卢兹国立综合理工学院-ENSEEIHT-图卢兹计算机研究所）；TéSA（航空航天电信实验室）；IRIT-UPS（图卢兹第三大学-图卢兹计算机研究所）
Missing data is a recurrent problem in remote sensing, mainly due to cloud coverage for multispectral images and acquisition problems. This can be a critical issue for crop monitoring, especially for applications relying on machine learning techniques, which generally assume that the feature matrix does not have missing values. This paper proposes a Gaussian Mixture Model (GMM) for the reconstruction of parcel-level features extracted from multispectral images. A robust version of the GMM is also investigated, since datasets can be contaminated by inaccurate samples or features (e.g., wrong cr…
We present a nonparametric Bayesian joint model for multivariate continuous and categorical variables, with the intention of developing a flexible engine for multiple imputation of missing values. The model fuses Dirichlet process mixtures of multinomial distributions for categorical variables with Dirichlet process mixtures of multivariate normal distributions for continuous variables. We incorporate dependence between the continuous and categorical variables by (i) modeling the means of the normal distributions as component-specific functions of the categorical variables and (ii) forming dis…
Multiple imputation is a common approach for dealing with missing values in statistical databases. The imputer fills in missing values with draws from predictive models estimated from the observed data, resulting in multiple, completed versions of the database. Researchers have developed a variety of default routines to implement multiple imputation; however, there has been limited research comparing the performance of these methods, particularly for categorical data. We use simulation studies to compare repeated sampling properties of three default multiple imputation methods for categorical …
Electronic health records (EHRs) are increasingly used for clinical and comparative effectiveness research, but suffer from missing data. Motivated by health services research on diabetes care, we seek to increase the quality of EHRs by focusing on missing values of longitudinal glycosylated hemoglobin (A1c), a key risk factor for diabetes complications and adverse events. Under the framework of multiple imputation (MI), we propose an individualized Bayesian latent profiling approach to capture A1c measurement trajectories subject to missingness. The proposed method is applied to EHRs of adult…
tBayes-MICE: A Bayesian Approach to Multiple Imputation for Time Series Data
Transitional Artificial Intelligence Research Group, School of Mathematics and Statistics, UNSW, Sydney, Australia（过渡人工智能研究组，数学与统计学院，新南威尔士大学，悉尼，澳大利亚）；School of Mathematics and Statistics, UNSW, Sydney, Australia（数学与统计学院，新南威尔士大学，悉尼，澳大利亚）
Joint data imputation and mechanistic modelling for simulating heart-brain interactions in incomplete datasets
Inria（法国国家信息与自动化技术研究所）；Université Côte d’Azur（蔚蓝海岸大学）；PhySense（PhySense公司）；Department of Information and Communication Technologies（信息与通信技术系）；Universitat Pompeu Fabra（庞培法布拉大学）
Beyond Random Missingness: Clinically Rethinking for Healthcare Time Series Imputation
King’s College London（伦敦国王学院）；PyPOTS Research（PyPOTS研究中心）；University of Oxford（牛津大学）；Hong Kong University of Science and Technology（香港科技大学）；University College London（伦敦大学学院）
Multiple imputation (MI) is a popular method for dealing with missing values. One main advantage of MI is to separate the imputation phase and the analysis one. However, both are related since they are based on distribution assumptions that have to be consistent. This point is well known as congeniality. In this paper, we discuss congeniality for clustering on continuous data. First, we theoretically highlight how two joint modeling (JM) MI methods (JM-GL and JM-DP) are congenial with various clustering methods. Then, we propose a new fully conditional specification (FCS) MI method with the sa…
Radboud University Medical Centre（拉德堡德大学医学中心）；LIS data solutions（LIS数据解决方案公司）；University of Zurich（苏黎世大学）；Karakter Child and Adolescent Psychiatry University Centre（Karakter儿童与青少年精神病学大学中心）；University of Heidelberg（海德堡大学）；SRH University（SRH大学）；Goethe University（法兰克福大学）；University of Cambridge（剑桥大学）；University Medical Center Utrecht（乌得勒支大学医学中心）；University of Oxford（牛津大学）
An increasing number of large-scale multi-modal research initiatives has been conducted in the typically developing population, as well as in psychiatric cohorts. Missing data is a common problem in such datasets due to the difficulty of assessing multiple measures on a large number of participants. The consequences of missing data accumulate when researchers aim to explore relationships between multiple measures. Here we aim to evaluate different imputation strategies to fill in missing values in clinical data from a large (total N=764) and deeply characterised (i.e. range of clinical and cog…
Institute for Infocomm Research（资讯通信研究院）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局（A*STAR））
Real-world clinical time series data sets exhibit a high prevalence of missing values. Hence, there is an increasing interest in missing data imputation. Traditional statistical approaches impose constraints on the data-generating process and decouple imputation from prediction. Recent works propose recurrent neural network based approaches for missing data imputation and prediction with time series data. However, they generate deterministic outputs and neglect the inherent uncertainty. In this work, we introduce a unified Bayesian recurrent framework for simultaneous imputation and prediction…
University of California, Los Angeles（加州大学洛杉矶分校）；David Geffen School of Medicine, University of California, Los Angeles（加州大学洛杉矶分校大卫·格芬医学院）
We develop a personalized real time risk scoring algorithm that provides timely and granular assessments for the clinical acuity of ward patients based on their (temporal) lab tests and vital signs. Heterogeneity of the patients population is captured via a hierarchical latent class model. The proposed algorithm aims to discover the number of latent classes in the patients population, and train a mixture of Gaussian Process (GP) experts, where each expert models the physiological data streams associated with a specific class. Self-taught transfer learning is used to transfer the knowledge of l…
Inference problems with incomplete observations often aim at estimating population properties of unobserved quantities. One simple way to accomplish this estimation is to impute the unobserved quantities of interest at the individual level and then take an empirical average of the imputed values. We show that this simple imputation estimator can provide partial protection against model misspecification. We illustrate imputation estimators' robustness to model specification on three examples: mixture model-based clustering, estimation of genotype frequencies in population genetics, and estimati…
Electronic health records (EHR) consist of longitudinal clinical observations portrayed with sparsity, irregularity, and high-dimensionality, which become major obstacles in drawing reliable downstream clinical outcomes. Although there exist great numbers of imputation methods to tackle these issues, most of them ignore correlated features, temporal dynamics and entirely set aside the uncertainty. Since the missing value estimates involve the risk of being inaccurate, it is appropriate for the method to handle the less certain information differently than the reliable data. In that regard, we …
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Deep Learning (DL) methods have dramatically increased in popularity in recent years. While its initial success was demonstrated in the classification and manipulation of image data, there has been significant growth in the application of DL methods to problems in the biomedical sciences. However, the greater prevalence and complexity of missing data in biomedical datasets present significant challenges for DL methods. Here, we provide a formal treatment of missing data in the context of Variational Autoencoders (VAEs), a popular unsupervised DL architecture commonly utilized for dimension red…
Biomedical Research and Innovation Institute of Cádiz (INiBICA)（加的斯生物医学研究与创新研究所（INiBICA））；Puerta del Mar University（马门大学）；Universidad Autónoma de Madrid（马德里自治大学）；Puerta del Mar University Hospital（马门大学医院）；University of Cádiz（加的斯大学）
Missing values are common in many real-life datasets. However, most of the current machine learning methods can not handle missing values. This means that they should be imputed beforehand. Gaussian Processes (GPs) are non-parametric models with accurate uncertainty estimates that combined with sparse approximations and stochastic variational inference scale to large data sets. Sparse GPs can be used to compute a predictive distribution for missing data. Here, we present a hierarchical composition of sparse GPs that is used to predict missing values at each dimension using all the variables fr…
Sloan School of Management and Operations Research Center（斯隆管理学院与运筹学研究中心）；Massachusetts Institute of Technology（麻省理工学院）
Missing data is a common problem in real-world settings and particularly relevant in healthcare applications where researchers use Electronic Health Records (EHR) and results of observational studies to apply analytics methods. This issue becomes even more prominent for longitudinal data sets, where multiple instances of the same individual correspond to different observations in time. Standard imputation methods do not take into account patient specific information incorporated in multivariate panel data. We introduce the novel imputation algorithm MedImpute that addresses this problem, exten…
Deep latent variable models (DLVMs) combine the approximation abilities of deep neural networks and the statistical foundations of generative models. Variational methods are commonly used for inference; however, the exact likelihood of these models has been largely overlooked. The purpose of this work is to study the general properties of this quantity and to show how they can be leveraged in practice. We focus on important inferential problems that rely on the likelihood: estimation and missing data imputation. First, we investigate maximum likelihood estimation for DLVMs: in particular, we s…
Missing data theory deals with the statistical methods in the occurrence of missing data. Missing data occurs when some values are not stored or observed for variables of interest. However, most of the statistical theory assumes that data is fully observed. An alternative to deal with incomplete databases is to fill in the spaces corresponding to the missing information based on some criteria, this technique is called imputation. We introduce a new imputation methodology for databases with univariate missing patterns based on additional information from fully-observed auxiliary variables. We a…
MGE-LDM: Joint Latent Diffusion for Simultaneous Music Generation and Source Extraction
Music and Audio Research Group (MARG)（音乐与音频研究组）；Interdisciplinary Program in Artificial Intelligence (IPAI)（人工智能跨学科项目）；AIIS（人工智能研究所）；Department of Intelligence and Information（智能与信息系）
The Current Population Survey is the gold-standard data source for studying who turns out to vote in elections. However, it suffers from potentially nonignorable unit and item nonresponse. Fortunately, after elections, the total number of voters is known from administrative sources and can be used to adjust for potential nonresponse bias. We present a model-based approach to utilize this known voter turnout rate, as well as other population marginal distributions of demographic variables, in multiple imputation for unit and item nonresponse. In doing so, we ensure that the imputations produce …
Multivariate categorical data nested within households often include reported values that fail edit constraints---for example, a participating household reports a child's age as older than his biological parent's age---as well as missing values. Generally, agencies prefer datasets to be free from erroneous or missing values before analyzing them or disseminating them to secondary data users. We present a model-based engine for editing and imputation of household data based on a Bayesian hierarchical model that includes (i) a nested data Dirichlet process mixture of products of multinomial dist…
Understanding of the pathophysiology of obstructive lung disease (OLD) is limited by available methods to examine the relationship between multi-omic molecular phenomena and clinical outcomes. Integrative factorization methods for multi-omic data can reveal latent patterns of variation describing important biological signal. However, most methods do not provide a framework for inference on the estimated factorization, simultaneously predict important disease phenotypes or clinical outcomes, nor accommodate multiple imputation. To address these gaps, we propose Bayesian Simultaneous Factorizati…
Gaussian mixtures are commonly used for modeling heavy-tailed error distributions in robust linear regression. Combining the likelihood of a multivariate robust linear regression model with a standard improper prior distribution yields an analytically intractable posterior distribution that can be sampled using a data augmentation algorithm. When the response matrix has missing entries, there are unique challenges to the application and analysis of the convergence properties of the algorithm. Conditions for geometric ergodicity are provided when the incomplete data have a "monotone" structure.…
Partially recorded data are frequently encountered in many applications and usually clustered by first removing incomplete cases or features with missing values, or by imputing missing values, followed by application of a clustering algorithm to the resulting altered dataset. Here, we develop clustering methodology through a model-based approach using the marginal density for the observed values, assuming a finite mixture model of multivariate $t$ distributions. We compare our approximate algorithm to the corresponding full expectation-maximization (EM) approach that considers the missing valu…
The Gaussian graphical model is a widely used tool for learning gene regulatory networks with high-dimensional gene expression data. Most existing methods for Gaussian graphical models assume that the data are homogeneous, i.e., all samples are drawn from a single Gaussian distribution. However, for many real problems, the data are heterogeneous, which may contain some subgroups or come from different resources. This paper proposes to model the heterogeneous data using a mixture Gaussian graphical model, and apply the imputation-consistency algorithm, combining with the $ψ$-learning algorithm,…
Learnable Prompt as Pseudo-Imputation: Rethinking the Necessity of Traditional EHR Data Imputation in Downstream Clinical Prediction
Peking University（北京大学）；School of Computer Science（计算机学院）；School of Software and Microelectronics（软件与微电子学院）；Center on Frontiers of Computing Studies（计算前沿研究中心）；National Engineering Research Center for Software Engineering（国家软件工程工程研究中心）；Peking University Information Technology Institute (Tianjin Binhai)（北京大学信息技术学院（天津滨海））
本文提出可学习提示作为伪插补（PAI）的新训练协议，以可学习提示建模下游模型对缺失值的隐式偏好，替代传统 EHR 插补模型，在多个临床预测任务和数据集上显著提升现有模型性能并具有更强鲁棒性。
Flow cytometry is a technology that rapidly measures antigen-based markers associated to cells in a cell population. Although analysis of flow cytometry data has traditionally considered one or two markers at a time, there has been increasing interest in multidimensional analysis. However, flow cytometers are limited in the number of markers they can jointly observe, which is typically a fraction of the number of markers of interest. For this reason, practitioners often perform multiple assays based on different, overlapping combinations of markers. In this paper, we address the challenge of i…
Technical University of Munich（慕尼黑工业大学）；Ludwig-Maximilians University（慕尼黑大学）
Large-scale population-based studies in medicine are a key resource towards better diagnosis, monitoring, and treatment of diseases. They also serve as enablers of clinical decision support systems, in particular Computer Aided Diagnosis (CADx) using machine learning (ML). Numerous ML approaches for CADx have been proposed in literature. However, these approaches assume full data availability, which is not always feasible in clinical data. To account for missing data, incomplete data samples are either removed or imputed, which could lead to data bias and may negatively affect classification p…
Identification of charged particles in a multilayer detector by the energy loss technique may also be achieved by the use of a neural network. The performance of the network becomes worse when a large fraction of information is missing, for instance due to detector inefficiencies. Algorithms which provide a way to impute missing information have been developed over the past years. Among the various approaches, we focused on normal mixtures models in comparison with standard mean imputation and multiple imputation methods. Further, to account for the intrinsic asymmetry of the energy loss data,…
National Remote Sensing Centre (ISRO), India（印度国家遥感中心（ISRO））
Decision making from data involves identifying a set of attributes that contribute to effective decision making through computational intelligence. The presence of missing values greatly influences the selection of right set of attributes and this renders degradation in classification accuracies of the classifiers. As missing values are quite common in data collection phase during field experiments or clinical trails appropriate handling would improve the classifier performance. In this paper we present a review of recently developed missing value imputation algorithms and compare their perfor…
Astronomical data often suffer from noise and incompleteness. We extend the common mixtures-of-Gaussians density estimation approach to account for situations with a known sample incompleteness by simultaneous imputation from the current model. The method, called GMMis, generalizes existing Expectation-Maximization techniques for truncated data to arbitrary truncation geometries and probabilistic rejection processes, as long as they can be specified and do not depend on the density itself. The method accounts for independent multivariate normal measurement errors for each of the observed sampl…
Hainan University（海南大学）；Beijing University of Technology（北京工业大学）
Missing value is a very common and unavoidable problem in sensors, and researchers have made numerous attempts for missing value imputation, particularly in deep learning models. However, for real sensor data, the specific data distribution and data periods are rarely considered, making it difficult to choose the appropriate evaluation indexes and models for different sensors. To address this issue, this study proposes a multistage imputation framework based on deep learning with adaptability for missing value imputation. The model presents a mixture measurement index of low- and higher-order …
Temporal Gaussian Copula For Clinical Multivariate Time Series Data Imputation
Chongqing Institute of Green and Intelligent Technology, CAS（中国科学院重庆绿色智能技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Singapore Management University（新加坡管理大学）；Southwest University（西南大学）
Imputation-free transformer learning enables robust Alzheimer's disease prediction and calibrated uncertainty quantification across heterogeneous clinical cohorts
Platform of Bioinformatics（生物信息平台）；Lausanne University Hospital（洛桑大学医院）；Faculty of Biology and Medicine（生物学与医学学院）；University of Lausanne（洛桑大学）；Leenaards Memory Centre（Leenaards记忆中心）；Department of Clinical Neurosciences（临床神经科学系）
Clinical decision support using data mining techniques offers more intelligent way to reduce the decision error in the last few years. However, clinical datasets often suffer from high missingness, which adversely impacts the quality of modelling if handled improperly. Imputing missing values provides an opportunity to resolve the issue. Conventional imputation methods adopt simple statistical analysis, such as mean imputation or discarding missing cases, which have many limitations and thus degrade the performance of learning. This study examines a series of machine learning based imputation …
