arXiv 2306.067802023-06-11eess.IVcs.CVq-bio.QM

H&E切片与多重免疫荧光图像之间的多模态病理图像搜索

Multimodal Pathology Image Search Between H&E Slides and Multiplexed Immunofluorescent Images

The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Ronald Reagan University of California Los Angeles Medical Center（罗纳德·里根加州大学洛杉矶分校医疗中心）

Amir Hajighasemi, MD Jillur Rahman Saurav, Mohammad S Nasr, Jai Prakash Veerla, Aarti Darji, Parisa Boodaghi Malidarreh, Michael Robben, Helen H Shang, Jacob M Luber

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究提出一种基于VAE潜空间DTW与排序投票的多模态病理图像搜索方法，可对齐匹配H&E切片与mIF图像，经样本与消融实验验证具备鲁棒性，能辅助精准诊断与个性化治疗。

摘要：

我们提出了一种多模态病理图像搜索方法：在变分自编码器（Variational Autoencoder, VAE）潜空间上使用动态时间规整（dynamic time warping, DTW），并将结果输入排序选择投票方案，以检索与查询H&E切片最相似的多重免疫荧光成像（multiplexed immunofluorescent imaging, mIF）。通过训练VAE并应用DTW，我们对mIF和H&E切片进行对齐与比较。我们的方法将形态学H&E数据与来自mIF的免疫表型分析相结合，改进了鉴别诊断和治疗决策，为临床医生提供了疾病状态的丰富视角。这有助于理解组织样本中的空间关系，或可革新诊断流程，提升精准度并支持个性化治疗方案选择。我们的技术在结直肠癌和健康扁桃体样本上验证了可行性。我们对一款旨在探索多重免疫荧光（mIF）与苏木精-伊红（Hematoxylin and Eosin, H&E）染色之间相关性的搜索引擎开展了详尽的消融研究，以验证其将这些不同模态映射到统一向量空间的能力。尽管存在极端的类别不平衡，该系统仍展现出鲁棒性和实用性，在不同数据特征下均能返回相似结果，表明其未来可应用于多模态组织病理学数据分析。

英文摘要：

We present an approach for multimodal pathology image search, using dynamic time warping (DTW) on Variational Autoencoder (VAE) latent space that is fed into a ranked choice voting scheme to retrieve multiplexed immunofluorescent imaging (mIF) that is most similar to a query H&E slide. Through training the VAE and applying DTW, we align and compare mIF and H&E slides. Our method improves differential diagnosis and therapeutic decisions by integrating morphological H&E data with immunophenotyping from mIF, providing clinicians a rich perspective of disease states. This facilitates an understanding of the spatial relationships in tissue samples and could revolutionize the diagnostic process, enhancing precision and enabling personalized therapy selection. Our technique demonstrates feasibility using colorectal cancer and healthy tonsil samples. An exhaustive ablation study was conducted on a search engine designed to explore the correlation between multiplexed Immunofluorescence (mIF) and Hematoxylin and Eosin (H&E) staining, in order to validate its ability to map these distinct modalities into a unified vector space. Despite extreme class imbalance, the system demonstrated robustness and utility by returning similar results across various data features, which suggests potential for future use in multimodal histopathology data analysis.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。