arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.10656cs.LGcs.AIstat.ML
收藏

虚拟人生成模型:用于学习人类特征的掩码建模方法

Virtual Human Generative Model: Masked Modeling Approach for Learning Human Characteristics

Preferred Networks, Inc.(Preferred Networks公司);Kao Corporation(Kao公司);The University of Tokyo(东京大学)
Kenta Oono, Nontawat Charoenphakdee, Kotatsu Bito, Zhengyan Gao, Hideyoshi Igata, Masashi Yoshikawa, Yoshiaki Ota, Hiroki Okui, Kei Akita, Shoichiro Yamaguchi, … 展开作者

Kenta Oono, Nontawat Charoenphakdee, Kotatsu Bito, Zhengyan Gao, Hideyoshi Igata, Masashi Yoshikawa, Yoshiaki Ota, Hiroki Okui, Kei Akita, Shoichiro Yamaguchi, Yohei Sugawara, Shin-ichi Maeda, Kunihiko Miyoshi, Yuki Saito, Koki Tsuda, Hiroshi Maruyama, Kohei Hayashi

AI总结:

本文提出面向高维稀疏医疗数据的VHGM-MAE算法,通过似然建模、Transformer掩码自编码器与适配缺失模式的训练方案,解决医疗数据建模四大核心挑战,在缺失值插补和合成数据生成上优于现有方法。

摘要:

虚拟人生成模型(Virtual Human Generative Model, VHGM)是一种对2000余个人类医疗相关属性的联合概率进行近似的生成模型。本文提出核心算法VHGM-MAE,这是一款专为处理高维稀疏医疗数据定制的掩码自编码器(masked autoencoder, MAE)。VHGM-MAE解决了四项关键技术挑战:(1)医疗数据类型的异质性;(2)概率分布建模;(3)多数据源导致的训练数据集系统性缺失问题;(4)高维小n大p问题。为应对上述挑战,VHGM-MAE采用基于似然的方法对异质类型分布进行建模,使用基于Transformer的MAE捕捉观测属性与缺失属性间的复杂依赖关系,并设计了全新训练方案,可有效利用具备多样缺失模式的可用样本以缓解小n大p问题。实验结果表明,VHGM-MAE在缺失值插补与合成数据生成两项任务上的表现均优于现有方法。

英文摘要:

Virtual Human Generative Model (VHGM) is a generative model that approximates the joint probability over more than 2000 human healthcare-related attributes. This paper presents the core algorithm, VHGM-MAE, a masked autoencoder (MAE) tailored for handling high-dimensional, sparse healthcare data. VHGM-MAE tackles four key technical challenges: (1) heterogeneity of healthcare data types, (2) probability distribution modeling, (3) systematic missingness in the training dataset arising from multiple data sources, and (4) the high-dimensional, small-$n$-large-$p$ problem. To address these challenges, VHGM-MAE employs a likelihood-based approach to model distributions with heterogeneous types, a transformer-based MAE to capture complex dependencies among observed and missing attributes, and a novel training scheme that effectively leverages available samples with diverse missingness patterns to mitigate the small-n-large-p problem. Experimental results demonstrate that VHGM-MAE outperforms existing methods in both missing value imputation and synthetic data generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑