基于深度度量变分自编码器的多模态数据生成
Multi-modal data generation with a deep metric variational autoencoder
AI总结:
本研究提出一种在潜在空间采用三元组损失的深度度量变分自编码器,通过类别簇内潜在空间采样实现多模态条件数据生成,在鼓膜耳镜图像与宽频鼓室测量的多模态数据集上验证有效,可用于多模态数据的高效增强。
摘要:
我们提出一种用于多模态数据生成的深度度量变分自编码器。该变分自编码器在潜在空间中采用三元组损失,可通过在每个类别簇的潜在空间内采样实现条件数据生成。我们在一个多模态数据集上评估了该方法,该数据集包含鼓膜耳镜图像及对应的宽频鼓室测量数据。数据集中的模态因代表中耳状态的不同方面而具有相关性,但不存在直接的像素级对应关系。该方法在图像与鼓室图对的条件生成上展现出良好前景,可实现多模态来源数据的高效数据增强。
英文摘要:
We present a deep metric variational autoencoder for multi-modal data generation. The variational autoencoder employs triplet loss in the latent space, which allows for conditional data generation by sampling in the latent space within each class cluster. The approach is evaluated on a multi-modal dataset consisting of otoscopy images of the tympanic membrane with corresponding wideband tympanometry measurements. The modalities in this dataset are correlated, as they represent different aspects of the state of the middle ear, but they do not present a direct pixel-to-pixel correlation. The approach shows promising results for the conditional generation of pairs of images and tympanograms, and will allow for efficient data augmentation of data from multi-modal sources.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。