Due to the cost or interference of measurement, we need to control measurement system. Assuming that each variable can be measured sequentially, there exists optimal policy choosing next measurement for the former observations. Though optimal measurement policy is actually dependent on the goal of measurement, we mainly focus on retrieving complete data, so called as imputation. Also, we adapt the imputation method to missingness varying with measurement policy. However, learning measurement policy and imputation requires complete data which is impossible to be observed, unfortunately. To tackle this problem, we propose a data generation method and joint learning algorithm. The main idea is that 1) the data generation method is inherited by imputation method, and 2) the adaptation of imputation encourages measurement policy to learn more than individual learning. We implemented some variations of proposed algorithm for two different datasets and various missing rates. From the experimental results, we demonstrate that our algorithm is generally applicable and outperforms baseline methods.
Toronto Rehabilitation Institute（多伦多康复研究所）；United Arab Emirates University（阿联酋大学）；University of Toronto（多伦多大学）
Presence of missing values in a dataset can adversely affect the performance of a classifier. Single and Multiple Imputation are normally performed to fill in the missing values. In this paper, we present several variants of combining single and multiple imputation with bootstrapping to create ensembles that can model uncertainty and diversity in the data, and that are robust to high missingness in the data. We present three ensemble strategies: bootstrapping on incomplete data followed by (i) single imputation and (ii) multiple imputation, and (iii) multiple imputation ensemble without bootst…
Tennessee State University（田纳西州立大学）；Old Dominion University（老道明大学）
Missing values in tabular data restrict the use and performance of machine learning, requiring the imputation of missing values. The most popular imputation algorithm is arguably multiple imputations using chains of equations (MICE), which estimates missing values from linear conditioning on observed values. This paper proposes methods to improve both the imputation accuracy of MICE and the classification accuracy of imputed data by replacing MICE's linear conditioning with ensemble learning and deep neural networks (DNN). The imputation accuracy is further improved by characterizing individua…
Beyond Random Missingness: Clinically Rethinking for Healthcare Time Series Imputation
King’s College London（伦敦国王学院）；PyPOTS Research（PyPOTS研究中心）；University of Oxford（牛津大学）；Hong Kong University of Science and Technology（香港科技大学）；University College London（伦敦大学学院）
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；University of Cambridge（剑桥大学）；University of Oxford（牛津大学）；The Alan Turing Institute（艾伦·图灵研究所）
Missing data is an important problem in machine learning practice. Starting from the premise that imputation methods should preserve the causal structure of the data, we develop a regularization scheme that encourages any baseline imputation method to be causally consistent with the underlying data generating mechanism. Our proposal is a causally-aware imputation algorithm (MIRACLE). MIRACLE iteratively refines the imputation of a baseline by simultaneously modeling the missingness generating mechanism, encouraging imputation to be consistent with the causal structure of the data. We conduct e…
Missing data are a concern in many real world data sets and imputation methods are often needed to estimate the values of missing data, but data sets with excessive missingness and high dimensionality challenge most approaches to imputation. Here we show that appropriate feature selection can be an effective preprocessing step for imputation, allowing for more accurate imputation and subsequent model predictions. The key feature of this preprocessing is that it incorporates uncertainty: by accounting for uncertainty due to missingness when selecting features we can reduce the degree of missing…
Department of Computing Science Simon Fraser University（西蒙弗雷泽大学计算机科学系）
Missing data is a significant problem impacting all domains. State-of-the-art framework for minimizing missing data bias is multiple imputation, for which the choice of an imputation model remains nontrivial. We propose a multiple imputation model based on overcomplete deep denoising autoencoders. Our proposed model is capable of handling different data types, missingness patterns, missingness proportions and distributions. Evaluation on several real life datasets show our proposed model significantly outperforms current state-of-the-art methods under varying conditions while simultaneously im…
University of Cambridge（剑桥大学）；UCLA（加州大学洛杉矶分校）；Meta；Alan Turing Institute（艾伦·图灵研究所）
Missing data is a systemic problem in practical scenarios that causes noise and bias when estimating treatment effects. This makes treatment effect estimation from data with missingness a particularly tricky endeavour. A key reason for this is that standard assumptions on missingness are rendered insufficient due to the presence of an additional variable, treatment, besides the input (e.g. an individual) and the label (e.g. an outcome). The treatment variable introduces additional complexity with respect to why some variables are missing that is not fully explored by previous work. In our work…
Fed-ReMasker: Federated Tabular Imputation under Feature-Level Missingness
ARTORG Center for Biomedical Engineering Research University of Bern（伯尔尼大学生物医学工程研究中心）；Graduate School for Cellular and Biomedical Sciences University of Bern（伯尔尼大学细胞与生物医学科学研究生院）
University of Oxford（牛津大学）；CEREMADE, CNRS Université Paris-Dauphine（巴黎多芬大学CNRS-CEREMADE实验室）
We propose a variational autoencoder architecture to model both ignorable and nonignorable missing data using pattern-set mixtures as proposed by Little (1993). Our model explicitly learns to cluster the missing data into missingness pattern sets based on the observed data and missingness masks. Underpinning our approach is the assumption that the data distribution under missingness is probabilistically semi-supervised by samples from the observed data distribution. Our setup trades off the characteristics of ignorable and nonignorable missingness and can thus be applied to data of both types.…
Most practical data science problems encounter missing data. A wide variety of solutions exist, each with strengths and weaknesses that depend upon the missingness-generating process. Here we develop a theoretical framework for training and inference using only observed variables enabling modeling of incomplete datasets without imputation. Using an information and measure-theoretic argument we construct models with latent space representations that regularize against the potential bias introduced by missing data. The theoretical properties of this approach are demonstrated empirically using a …
One Pipeline, Many Transformers: Pattern-Specific Imputation Specialists for Tabular Missing Data
Industrial Engineering \& Operations Research, Columbia University, New York, USA；Department of Statistics, Columbia University, New York, USA；Operations Research \& Information Engineering, Cornell Tech, New York, USA
M$^3$-Impute: Mask-guided Representation Learning for Missing Value Imputation
BNU-HKBU United International College（北师大-浸会大学联合国际学院）；University College Dublin（都柏林大学学院）；The Hong Kong University of Science and Technology（香港科技大学）；Texas State University（德克萨斯州立大学）
Gary and Mary West Health Institute（加里与玛丽·韦斯特健康研究所）
Although data may be abundant, complete data is less so, due to missing columns or rows. This missingness undermines the performance of downstream data products that either omit incomplete cases or create derived completed data for subsequent processing. Appropriately managing missing data is required in order to fully exploit and correctly use data. We propose a Multiple Imputation model using Denoising Autoencoders to learn the internal representation of data. Furthermore, we use the novel mechanisms of Metamorphic Truth and Imputation Feedback to maintain statistical integrity of attributes…
GAUDI: Geometry-Aware Diffusion for Calibrated Air-Quality Time-Series Imputation
Columbia University（哥伦比亚大学）；University of California, Davis（加州大学戴维斯分校）；University of California, San Diego（加州大学圣迭戈分校）；Georgia Institute of Technology（佐治亚理工学院）；Lehigh University（理海大学）；Stony Brook University（纽约州立大学石溪分校）；San Francisco State University（旧金山州立大学）；Carnegie Mellon University（卡内基梅隆大学）；University of Southern California（南加州大学）
University of Washington（华盛顿大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
In wearable sensing applications, data is inevitable to be irregularly sampled or partially missing, which pose challenges for any downstream application. An unique aspect of wearable data is that it is time-series data and each channel can be correlated to another one, such as x, y, z axis of accelerometer. We argue that traditional methods have rarely made use of both times-series dynamics of the data as well as the relatedness of the features from different sensors. We propose a model, termed as DynImp, to handle different time point's missingness with nearest neighbors along feature axis a…
DeepSea Technologies；University of Oxford（牛津大学）；University of Cambridge（剑桥大学）
The problem of missing data, usually absent incurated and competition-standard datasets, is an unfortunate reality for most machine learning models used in industry applications. Recent work has focused on understanding the nature and the negative effects of such phenomena, while devising solutions for optimal imputation of the missing data, using both discriminative and generative approaches. We propose a novel mechanism based on multi-head attention which can be applied effortlessly in any model and achieves better downstream performance without the introduction of the full dataset in any pa…
Increasing Missingness to Reduce Bias: Richardson-SGD with Missing Data
Sorbonne Université and Université Paris Cité, CNRS, Laboratoire de Probabilités, Statistique et Modélisation, LPSM（索邦大学和巴黎cité大学，CNRS，概率、统计与建模实验室，LPSM）
MoCap-Impute: A Comprehensive Benchmark and Comparative Analysis of Imputation Methods for IMU-based Motion Capture Data
Peter Faber Business School, Australian Catholic University（澳大利亚天主教大学彼得·法伯商学院）；University of Technology Sydney（技术与应用科学大学悉尼分校）；Department of Information Technology, College of Computing and Information Sciences, University of Technology and Applied Sciences（技术与应用科学大学信息科技系）；Faculty of Computers and Informatics, Zagazig University（扎基尔大学计算机与信息学院）；AI Applications Research Chair, University of Nizwa（努孜瓦大学人工智能应用研究 chair）；Department of Data Science, Wakeb（数据科学系，Wakeb）；Department of Computer Science, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University（计算机科学系，普林斯·萨塔姆·本·阿卜杜勒阿齐兹大学）；Higher Future Institute for Specialized Technological Studies, Cairo, Egypt（埃及未来高级专门技术研究学院）；Department of Computer Science, Faculty of Computers & Informatics, Suez Canal University（计算机科学系，苏伊士运河大学）；Department of Mathematics, Faculty of Science, Suez Canal University（数学系，苏伊士运河大学）
University of the Witwatersrand（金山大学）；University of Johannesburg（约翰内斯堡大学）
Missing data is a common concern in health datasets, and its impact on good decision-making processes is well documented. Our study's contribution is a methodology for tackling missing data problems using a combination of synthetic dataset generation, missing data imputation and deep learning methods to resolve missing data challenges. Specifically, we conducted a series of experiments with these objectives; $a)$ generating a realistic synthetic dataset, $b)$ simulating data missingness, $c)$ recovering the missing data, and $d)$ analyzing imputation performance. Our methodology used a gaussia…
Missing Data Multiple Imputation for Tabular Q-Learning in Online RL
Department of Statistics, Harvard University, Cambridge, United States（哈佛大学统计学系）；Department of Statistics, Stanford University, Stanford, United States（斯坦福大学统计学系）；Department of Computer Science, Harvard University, Cambridge, United States（哈佛大学计算机科学系）
PAMF: Prior-Aware Multimodal Fusion for Incomplete Time Series Data
Department of Computer Science, University of Central Florida（中央佛罗里达大学计算机科学系）；Department of Computer Science, University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校计算机科学系）
University of Cambridge（剑桥大学）；Microsoft Research Cambridge（微软剑桥研究院）
Real-world datasets often have missing values associated with complex generative processes, where the cause of the missingness may not be fully observed. This is known as missing not at random (MNAR) data. However, many imputation methods do not take into account the missingness mechanism, resulting in biased imputation values when MNAR data is present. Although there are a few methods that have considered the MNAR scenario, their model's identifiability under MNAR is generally not guaranteed. That is, model parameters can not be uniquely determined even with infinite data samples, hence the i…
Imputation of missing data is a common application in various classification problems where the feature training matrix has missingness. A widely used solution to this imputation problem is based on the lazy learning technique, $k$-nearest neighbor (kNN) approach. However, most of the previous work on missing data does not take into account the presence of the class label in the classification problem. Also, existing kNN imputation methods use variants of Minkowski distance as a measure of distance, which does not work well with heterogeneous data. In this paper, we propose a novel iterative k…
Imputation Strategies Under Clinical Presence: Impact on Algorithmic Fairness
MRC Biostatistics Unit（MRC生物统计学单元）；University of Cambridge（剑桥大学）；The Alan Turing Institute（阿兰·图灵研究所）；McCombs School of Business（麦库姆斯商学院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Rady School of Management（雷迪管理学院）；University of California, San Diego（加利福尼亚大学圣迭戈分校）
Generative Modeling under Non-Monotone MAR Missingness via Approximate Wasserstein Gradient Flows
Department of Mathematics（数学系）；Research Institute for Statistics and Information Science（统计与信息科学研究所）；University of Geneva（日内瓦大学）；University of Hamburg（汉堡大学）
Stanford School of Medicine（斯坦福大学医学院）；Stanford University（斯坦福大学）
There is an emerging trend in the reinforcement learning for healthcare literature. In order to prepare longitudinal, irregularly sampled, clinical datasets for reinforcement learning algorithms, many researchers will resample the time series data to short, regular intervals and use last-observation-carried-forward (LOCF) imputation to fill in these gaps. Typically, they will not maintain any explicit information about which values were imputed. In this work, we (1) call attention to this practice and discuss its potential implications; (2) propose an alternative representation of the patient …
Clinical decision support using data mining techniques offers more intelligent way to reduce the decision error in the last few years. However, clinical datasets often suffer from high missingness, which adversely impacts the quality of modelling if handled improperly. Imputing missing values provides an opportunity to resolve the issue. Conventional imputation methods adopt simple statistical analysis, such as mean imputation or discarding missing cases, which have many limitations and thus degrade the performance of learning. This study examines a series of machine learning based imputation …
The Wharton School of the University of Pennsylvania（宾夕法尼亚大学沃顿商学院）
We present a method for incorporating missing data in non-parametric statistical learning without the need for imputation. We focus on a tree-based method, Bayesian Additive Regression Trees (BART), enhanced with "Missingness Incorporated in Attributes," an approach recently proposed incorporating missingness into decision trees (Twala, 2008). This procedure takes advantage of the partitioning mechanisms found in tree-based models. Simulations on generated models and real data indicate that our proposed method can forecast well on complicated missing-at-random and not-missing-at-random models …
