有效的数据插补需要从“普通”表格数据中挖掘丰富潜在“结构”的能力。近期基于图神经网络的数据插补方案通过将表格数据直接转化为二部图,展现出强大的结构学习潜力。然而,由于缺乏样本间的关联,这些方案对所有样本一视同仁,这与“相似样本应能提供更多缺失值相关信息”的重要观察相悖。\n本文提出一种用于缺失数据插补的新型迭代图生成与重构框架IGRM(Iterative graph Generation and Reconstruction framework for Missing data imputation)。我们并未对所有样本同等对待,而是引入“朋友网络”的概念来表征样本间的不同关联。为在存在缺失数据的情况下生成准确的朋友网络,我们设计了一种端到端的朋友网络重构方案,可在插补学习过程中持续优化朋友网络。优化后的朋友网络表征反过来又通过差异化消息传递进一步优化数据插补过程。\n在8个基准数据集上的实验结果表明,与9种基线方法相比,IGRM的平均绝对误差降低了39.13%,较次优方法降低9.04%。我们的代码可在https://github.com/G-AILab/IGRM获取。
英文摘要:
Effective data imputation demands rich latent ``structure" discovery capabilities from ``plain" tabular data. Recent advances in graph neural networks-based data imputation solutions show their strong structure learning potential by directly translating tabular data as bipartite graphs. However, due to a lack of relations between samples, those solutions treat all samples equally which is against one important observation: ``similar sample should give more information about missing values." This paper presents a novel Iterative graph Generation and Reconstruction framework for Missing data imputation(IGRM). Instead of treating all samples equally, we introduce the concept: ``friend networks" to represent different relations among samples. To generate an accurate friend network with missing data, an end-to-end friend network reconstruction solution is designed to allow for continuous friend network optimization during imputation learning. The representation of the optimized friend network, in turn, is used to further optimize the data imputation process with differentiated message passing. Experiment results on eight benchmark datasets show that IGRM yields 39.13% lower mean absolute error compared with nine baselines and 9.04% lower than the second-best. Our code is available at https://github.com/G-AILab/IGRM.
School of Artificial Intelligence, OPtics and ElectroNics (iOPEN)(人工智能学院(iOPEN));Northwestern Polytechnical University(西北工业大学);Institute of Artificial Intelligence (TeleAI)(人工智能研究所(TeleAI))
Nonparametric End-to-End Probabilistic Forecasting of Distributed Generation Outputs Considering Missing Data Imputation
Guangzhou Power Supply Bureau Guangdong Power Grid Co., Ltd.(广东电网有限责任公司广州供电局);Tsinghua University(清华大学);Tsinghua-Berkeley Shenzhen Institute (TBSI)(清华-伯克利深圳学院)
missForestPredict -- Missing data imputation for prediction settings
KU Leuven(KU Leuven(荷语天主教鲁汶大学));Leuven Unit for Health Technology Assessment Research (LUHTAR)(鲁汶卫生技术评估研究单元(LUHTAR));School for Public Health and Primary Care, Maastricht University(马斯特里赫特大学公共卫生与基层保健学院);Leiden University Medical Center(莱顿大学医学中心)
DiffPuter: Empowering Diffusion Models for Missing Data Imputation
Computer Science Department, University of Illinois at Chicago(伊利诺伊大学芝加哥分校计算机科学系);Eric and Wendy Schmidt Center, Broad Institute of MIT and Harvard(MIT和哈佛大学Broad研究所埃里克和文迪·施密特中心)
Handling Missing Data in Downstream Tasks With Distribution-Preserving Guarantees
Institute of Computer Science University of Rostock(计算机科学研究所罗斯托克大学);Institute of Computer Science / Soda University of Rostock / Inria Saclay(计算机科学研究所/ Soda罗斯托克大学/ Inria萨克雷);Department of Data Science IISER Thiruvananthapuram(数据科学系IISER特里凡纳塔普拉姆)
针对分类等下游任务中的缺失数据问题,提出可与分类器联合训练的 F3I 迭代 K 近邻插补方法,并给出分布保持理论保证与实验验证。
Reconstruction of Sentinel-2 Time Series Using Robust Gaussian Mixture Models -- Application to the Detection of Anomalous Crop Development in wheat and rapeseed crops
TerraNIS;University of Toulouse(图卢兹大学);IRIT-INP-ENSEEIHT(图卢兹国立综合理工学院-ENSEEIHT-图卢兹计算机研究所);TéSA(航空航天电信实验室);IRIT-UPS(图卢兹第三大学-图卢兹计算机研究所)
Joint Learning of Unsupervised Multi-view Feature and Instance Co-selection with Cross-view Imputation
Joint Laboratory of Data Science and Business Intelligence, School of Statistics and Data Science, Southwestern University of Finance and Economics(数据科学与商务智能联合实验室,统计与数据科学学院,西南财经大学);Department of Electrical Engineering and Computer Science, University of Kansas(电气工程与计算机科学系,堪萨斯大学);School of Computing and Artificial Intelligence, Southwest Jiaotong University(计算与人工智能学院,西南交通大学);Faculty of Mathematics and Computer Science, University of Münster(数学与计算机科学学院,穆斯堡大学)
Learnable Prompt as Pseudo-Imputation: Rethinking the Necessity of Traditional EHR Data Imputation in Downstream Clinical Prediction
Peking University(北京大学);School of Computer Science(计算机学院);School of Software and Microelectronics(软件与微电子学院);Center on Frontiers of Computing Studies(计算前沿研究中心);National Engineering Research Center for Software Engineering(国家软件工程工程研究中心);Peking University Information Technology Institute (Tianjin Binhai)(北京大学信息技术学院(天津滨海))
本文提出可学习提示作为伪插补(PAI)的新训练协议,以可学习提示建模下游模型对缺失值的隐式偏好,替代传统 EHR 插补模型,在多个临床预测任务和数据集上显著提升现有模型性能并具有更强鲁棒性。
MoCap-Impute: A Comprehensive Benchmark and Comparative Analysis of Imputation Methods for IMU-based Motion Capture Data
Peter Faber Business School, Australian Catholic University(澳大利亚天主教大学彼得·法伯商学院);University of Technology Sydney(技术与应用科学大学悉尼分校);Department of Information Technology, College of Computing and Information Sciences, University of Technology and Applied Sciences(技术与应用科学大学信息科技系);Faculty of Computers and Informatics, Zagazig University(扎基尔大学计算机与信息学院);AI Applications Research Chair, University of Nizwa(努孜瓦大学人工智能应用研究 chair);Department of Data Science, Wakeb(数据科学系,Wakeb);Department of Computer Science, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University(计算机科学系,普林斯·萨塔姆·本·阿卜杜勒阿齐兹大学);Higher Future Institute for Specialized Technological Studies, Cairo, Egypt(埃及未来高级专门技术研究学院);Department of Computer Science, Faculty of Computers & Informatics, Suez Canal University(计算机科学系,苏伊士运河大学);Department of Mathematics, Faculty of Science, Suez Canal University(数学系,苏伊士运河大学)
Efficient data augmentation using graph imputation neural networks
Department of Information Engineering, Electronics;Telecommunications (DIET), ``Sapienza'' University of Rome, Via Eudossiana 18, 00184, Rome. \ : \ simone.scardapane
Traffic Data Imputation using Deep Convolutional Neural Networks
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Korea Advanced Institute of Science and Technology(韩国科学技术院);New York University Abu Dhabi(纽约大学阿布扎比分校);New York University Tandon School of Engineering(纽约大学坦登工程学院)