arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1805.08400cs.CV
收藏

深度学习与电影渲染：使用真实感医学图像微调深度神经网络

Deep Learning with Cinematic Rendering: Fine-Tuning Deep Neural Networks Using Photorealistic Medical Images

Faisal Mahmood, Richard Chen, Sandra Sudarsky, Daphne Yu, Nicholas J. Durr

备注：
14 Pages, 4 Figures, 3 Tables, Physics in Medicine and Biology (2018)
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

摘要：

深度学习已经成为一种强大的人工智能工具，用于解释越来越多的各种应用的医学图像。然而，缺乏高质量的注释，这是必要的训练这些方法的医学成像数据最终限制了他们的表现。由于隐私问题、可用于注释的专家短缺、罕见疾病的有限代表性和成本，医疗数据的获取具有挑战性。这个问题以前已经通过使用合成生成的数据来解决。然而，在合成数据上训练的网络往往无法推广到真实数据。电影渲染模拟光通过从CT数据重建的组织模型的传播和相互作用，从而能够生成逼真的图像。在本文中，我们提出了电影渲染在深度学习中的首批应用之一，其中我们提出使用电影渲染的CT数据微调合成数据驱动网络，用于内窥镜检查中的单目深度估计任务。我们的实验表明：（a）在合成数据上训练并在真实感电影渲染数据上微调的卷积神经网络（CNN）与没有微调的网络相比更好地适应真实医学图像，并表现出更鲁棒的性能，（b）这些微调的网络需要更少的训练数据来收敛到最优解，以及（c）利用来自同一场景的各种真实感绘制条件的数据进行微调防止了网络学习患者特异性信息并有助于模型的泛化。我们的实证评估表明，网络微调与电影渲染数据预测深度与渲染内窥镜图像的误差减少56.87%，27.49%的误差减少真正的猪结肠内窥镜图像。

英文摘要：

Deep learning has emerged as a powerful artificial intelligence tool to interpret medical images for a growing variety of applications. However, the paucity of medical imaging data with high-quality annotations that is necessary for training such methods ultimately limits their performance. Medical data is challenging to acquire due to privacy issues, shortage of experts available for annotation, limited representation of rare conditions and cost. This problem has previously been addressed by using synthetically generated data. However, networks trained on synthetic data often fail to generalize to real data. Cinematic rendering simulates the propagation and interaction of light passing through tissue models reconstructed from CT data, enabling the generation of photorealistic images. In this paper, we present one of the first applications of cinematic rendering in deep learning, in which we propose to fine-tune synthetic data-driven networks using cinematically rendered CT data for the task of monocular depth estimation in endoscopy. Our experiments demonstrate that: (a) Convolutional Neural Networks (CNNs) trained on synthetic data and fine-tuned on photorealistic cinematically rendered data adapt better to real medical images and demonstrate more robust performance when compared to networks with no fine-tuning, (b) these fine-tuned networks require less training data to converge to an optimal solution, and (c) fine-tuning with data from a variety of photorealistic rendering conditions of the same scene prevents the network from learning patient-specific information and aids in generalizability of the model. Our empirical evaluation demonstrates that networks fine-tuned with cinematically rendered data predict depth with 56.87% less error for rendered endoscopy images and 27.49% less error for real porcine colon endoscopy images.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑