arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2008.09884cs.CV
收藏

用于肺水肿评估的胸片和放射学报告的联合建模

Joint Modeling of Chest Radiographs and Radiology Reports for Pulmonary Edema Assessment

Massachusetts Institute of Technology(麻省理工学院);Philips Research North America(飞利浦北美研究院);Beth Israel Deaconess Medical Center, Harvard Medical School(哈佛医学院贝斯以色列女执事医疗中心)

Geeticka Chauhan, Ruizhi Liao, William Wells, Jacob Andreas, Xin Wang, Seth Berkowitz, Steven Horng, Peter Szolovits, Polina Golland

摘要:

我们提出并证明了一种新的机器学习算法,评估肺水肿的严重程度,从胸片。虽然存在大型公开可用的胸片数据集和自由文本放射学报告,但只能从放射学报告中提取有限的数字水肿严重程度标签。这是学习用于图像分类的这种模型的重大挑战。为了利用放射学报告中的丰富信息,我们开发了一个神经网络模型,该模型在图像和自由文本上进行训练,以在推理时间从胸片评估肺水肿的严重程度。我们的实验结果表明,联合图像-文本表示学习提高了肺水肿评估的性能相比,监督模型训练的图像。我们还显示了使用的文本解释的联合模型的图像分类。据我们所知,我们的方法是第一个利用自由文本放射学报告来提高该应用程序中的图像模型性能。我们的代码可在https://github.com/RayRuizhiLiao/joint_chestxray上获得。

英文摘要:

We propose and demonstrate a novel machine learning algorithm that assesses pulmonary edema severity from chest radiographs. While large publicly available datasets of chest radiographs and free-text radiology reports exist, only limited numerical edema severity labels can be extracted from radiology reports. This is a significant challenge in learning such models for image classification. To take advantage of the rich information present in the radiology reports, we develop a neural network model that is trained on both images and free-text to assess pulmonary edema severity from chest radiographs at inference time. Our experimental results suggest that the joint image-text representation learning improves the performance of pulmonary edema assessment compared to a supervised model trained on images only. We also show the use of the text for explaining the image classification by the joint model. To the best of our knowledge, our approach is the first to leverage free-text radiology reports for improving the image model performance in this application. Our code is available at https://github.com/RayRuizhiLiao/joint_chestxray.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑