基于语义引导医学文本基础模型的临床级多器官病理报告生成：面向多尺度全切片图像
Clinical-grade Multi-Organ Pathology Report Generation for Multi-scale Whole Slide Images via a Semantically Guided Medical Text Foundation Model
AI总结：
针对多尺度WSI尺寸大和标注成本高导致病理报告生成受限的问题，提出患者级多器官病理报告生成（PMPRG）模型，利用多尺度区域视觉Transformer（MR-ViT）提取特征指导VLM训练，在多器官数据集上达到0.68的METEOR分数，有效提升病理报告生成效率。
摘要：
视觉语言模型（VLM）在自然语言理解和图像识别任务中均取得了成功。然而，由于多尺度WSI的巨大尺寸和WSI标注的高昂成本，其在全切片图像（WSI）病理报告生成中的使用仍然有限。此外，在现有的大多数病理报告生成研究中，尚未对临床疗效进行充分验证。在此，我们提出了一种新颖的患者级多器官病理报告生成（PMPRG）模型，该模型利用我们提出的多尺度区域视觉Transformer（MR-ViT）模型提取的多尺度WSI特征及其真实病理报告，来指导VLM训练以生成准确的病理报告。随后，该模型基于所提供的关注关键区域的特征自动生成报告。我们使用包含结肠和肾脏等多个器官的WSI数据集评估了我们的模型。我们的模型达到了0.68的METEOR分数，证明了该方法的有效性。该模型使病理学家能够高效地为患者生成病理报告，无论涉及的WSI数量多少。
英文摘要：
Vision language models (VLM) have achieved success in both natural language comprehension and image recognition tasks. However, their use in pathology report generation for whole slide images (WSIs) is still limited due to the huge size of multi-scale WSIs and the high cost of WSI annotation. Moreover, in most of the existing research on pathology report generation, sufficient validation regarding clinical efficacy has not been conducted. Herein, we propose a novel Patient-level Multi-organ Pathology Report Generation (PMPRG) model, which utilizes the multi-scale WSI features from our proposed multi-scale regional vision transformer (MR-ViT) model and their real pathology reports to guide VLM training for accurate pathology report generation. The model then automatically generates a report based on the provided key features attended regional features. We assessed our model using a WSI dataset consisting of multiple organs, including the colon and kidney. Our model achieved a METEOR score of 0.68, demonstrating the effectiveness of our approach. This model allows pathologists to efficiently generate pathology reports for patients, regardless of the number of WSIs involved.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
