arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.03434eess.IVcs.CV

基于深度度量变分自编码器的多模态数据生成

Multi-modal data generation with a deep metric variational autoencoder

Technical University of Denmark(丹麦技术大学);Interacoustics Research Unit(国际听力研究单元);Interacoustics A/S(国际听力公司);Kamide ENT clinic(上田耳鼻咽喉科诊所);Diatec Japan(迪亚泰克日本公司)

Josefine Vilsbøll Sundgaard, Morten Rieger Hannemose, Søren Laugesen, Peter Bray, James Harte, Yosuke Kamide, Chiemi Tanaka, Rasmus R. Paulsen, Anders Nymark Christensen

AI总结:

本研究提出一种在潜在空间采用三元组损失的深度度量变分自编码器,通过类别簇内潜在空间采样实现多模态条件数据生成,在鼓膜耳镜图像与宽频鼓室测量的多模态数据集上验证有效,可用于多模态数据的高效增强。

摘要:

我们提出一种用于多模态数据生成的深度度量变分自编码器。该变分自编码器在潜在空间中采用三元组损失,可通过在每个类别簇的潜在空间内采样实现条件数据生成。我们在一个多模态数据集上评估了该方法,该数据集包含鼓膜耳镜图像及对应的宽频鼓室测量数据。数据集中的模态因代表中耳状态的不同方面而具有相关性,但不存在直接的像素级对应关系。该方法在图像与鼓室图对的条件生成上展现出良好前景,可实现多模态来源数据的高效数据增强。

英文摘要:

We present a deep metric variational autoencoder for multi-modal data generation. The variational autoencoder employs triplet loss in the latent space, which allows for conditional data generation by sampling in the latent space within each class cluster. The approach is evaluated on a multi-modal dataset consisting of otoscopy images of the tympanic membrane with corresponding wideband tympanometry measurements. The modalities in this dataset are correlated, as they represent different aspects of the state of the middle ear, but they do not present a direct pixel-to-pixel correlation. The approach shows promising results for the conditional generation of pairs of images and tympanograms, and will allow for efficient data augmentation of data from multi-modal sources.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑