arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.05534cs.CVcs.LGeess.IV
收藏

随机缺失输入数据的多域图像补齐

Multi-Domain Image Completion for Random Missing Input Data

Stanford University(斯坦福大学);NVIDIA(英伟达);National Institutes of Health(美国国立卫生研究院)
Liyue Shen, Wentao Zhu, Xiaosong Wang, Lei Xing, John M. Pauly, Baris Turkbey, Stephanie Anne Harmon, Thomas Hogue Sanford, Sherif Mehralivand, Peter Choyke, Br… 展开作者

Liyue Shen, Wentao Zhu, Xiaosong Wang, Lei Xing, John M. Pauly, Baris Turkbey, Stephanie Anne Harmon, Thomas Hogue Sanford, Sherif Mehralivand, Peter Choyke, Bradford Wood, Daguang Xu

摘要:

多域数据在视觉应用中被广泛利用,利用来自不同模态的互补信息,例如,从多参数磁共振成像(MRI)分割脑肿瘤。然而,由于可能的数据损坏和不同的成像协议,每个域的图像的可用性在实践中可能在多个数据源之间变化,这使得利用不同的输入数据集合构建通用模型具有挑战性。为了解决这个问题,我们提出了一种通用的方法来完成随机缺失域数据在实际应用中。具体来说,我们开发了一种新的多域图像完成方法,该方法利用生成对抗网络(GAN)与代表性的解纠缠方案来提取共享的骨架编码和分离的肉体编码跨多个域。我们进一步说明了多域图像完成中的学习表示可以用于高级任务,例如,分割,通过引入一个统一的框架,包括图像完成和分割与共享的内容编码器。实验表明,一致的性能提高三个数据集的脑肿瘤分割,前列腺分割,面部表情图像完成。

英文摘要:

Multi-domain data are widely leveraged in vision applications taking advantage of complementary information from different modalities, e.g., brain tumor segmentation from multi-parametric magnetic resonance imaging (MRI). However, due to possible data corruption and different imaging protocols, the availability of images for each domain could vary amongst multiple data sources in practice, which makes it challenging to build a universal model with a varied set of input data. To tackle this problem, we propose a general approach to complete the random missing domain(s) data in real applications. Specifically, we develop a novel multi-domain image completion method that utilizes a generative adversarial network (GAN) with a representational disentanglement scheme to extract shared skeleton encoding and separate flesh encoding across multiple domains. We further illustrate that the learned representation in multi-domain image completion could be leveraged for high-level tasks, e.g., segmentation, by introducing a unified framework consisting of image completion and segmentation with a shared content encoder. The experiments demonstrate consistent performance improvement on three datasets for brain tumor segmentation, prostate segmentation, and facial expression image completion respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑