arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2502.10536cs.CVcs.AIcs.LG

PolyPath:为多切片病理报告生成适配大型多模态模型

PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generation

Google Research(谷歌研究院)
Faruk Ahmed, Lin Yang, Tiam Jaroensri, Andrew Sellergren, Yossi Matias, Avinatan Hassidim, Greg S. Corrado, Dale R. Webster, Shravya Shetty, Shruthi Prabhakara,… 展开作者

Faruk Ahmed, Lin Yang, Tiam Jaroensri, Andrew Sellergren, Yossi Matias, Avinatan Hassidim, Greg S. Corrado, Dale R. Webster, Shravya Shetty, Shruthi Prabhakara, Yun Liu, Daniel Golden, Ellery Wulczyn, David F. Steiner

AI总结:

针对计算病理学中现有视觉语言能力局限于单张全切片图像等问题,本研究利用具有100万token上下文窗口的大型多模态模型Gemini 1.5 Flash,实现从多张全切片图像生成病理报告,68%案例报告准确性等同或优于原始报告,展现长上下文LMM在该任务的潜力。

摘要:

组织病理学病例的解读是医学中许多重要诊断和治疗决策的基础。值得注意的是,这一过程通常需要病理学家整合并总结每个病例的多张切片的发现。计算病理学中现有的视觉语言能力目前在很大程度上局限于小的感兴趣区域、低放大倍数下的较大区域或单张全切片图像(WSIs)。这限制了对跨多张WSIs的多个高放大倍数区域的发现的解读。通过使用Gemini 1.5 Flash(一个具有100万token上下文窗口的大型多模态模型(LMM)),我们展示了从多达40,000个768x768像素的图像补丁(来自多张10倍放大倍数的WSIs)生成底线诊断的能力,这相当于1 fps下长达11小时的视频。专家病理学家评估表明,对于最多5张切片的多切片案例,生成的报告文本在临床准确性上等同于或优于原始报告的比例为68%(95% CI:[60%,76%])。虽然对于6张或更多切片的案例性能有所下降,但本研究证明了利用现代LMM的长上下文能力解决医学报告生成这一独特挑战性任务的潜力,该任务中每个案例可能包含数千个图像补丁。

英文摘要:

The interpretation of histopathology cases underlies many important diagnostic and treatment decisions in medicine. Notably, this process typically requires pathologists to integrate and summarize findings across multiple slides per case. Existing vision-language capabilities in computational pathology have so far been largely limited to small regions of interest, larger regions at low magnification, or single whole-slide images (WSIs). This limits interpretation of findings that span multiple high-magnification regions across multiple WSIs. By making use of Gemini 1.5 Flash, a large multimodal model (LMM) with a 1-million token context window, we demonstrate the ability to generate bottom-line diagnoses from up to 40,000 768x768 pixel image patches from multiple WSIs at 10X magnification. This is the equivalent of up to 11 hours of video at 1 fps. Expert pathologist evaluations demonstrate that the generated report text is clinically accurate and equivalent to or preferred over the original reporting for 68% (95% CI: [60%, 76%]) of multi-slide examples with up to 5 slides. While performance decreased for examples with 6 or more slides, this study demonstrates the promise of leveraging the long-context capabilities of modern LMMs for the uniquely challenging task of medical report generation where each case can contain thousands of image patches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑