利用大型多模态模型导航千兆像素病理图像
Navigating Gigapixel Pathology Images with Large Multimodal Models
AI总结：
提出GIANT方法，无需训练即可让通用多模态模型自主导航WSI，通过迭代选择多放大倍数裁剪并聚合证据，在MultiPathQA基准上实现SOTA。
摘要：
近期大型多模态模型的进展使得开发能够对话和推理病理全切片图像（WSI）的交互式聊天模型成为可能。然而，现有的切片级聊天系统通常高度专业化，通常将WSI压缩为固定的切片级嵌入或依赖多组件流水线，这可能会丢失多尺度细节并限制目标任务之外的泛化能力。我们提出GIANT（千兆像素图像组织导航代理），一种简单、无需训练的方法，让通用多模态模型自主导航WSI，迭代选择多放大倍数裁剪并随时间聚合证据。为了评估WSI问答中的泛化能力并促进可重复性，我们引入了MultiPathQA，一个涵盖五个临床挑战和934个问题（涉及868个独特WSI）的基准套件。其中包括128道由病理学家编写的多项选择题，旨在模拟真实的诊断搜索和多尺度推理。使用GPT-5，GIANT在五个基准中的四个上取得了最先进的性能，优于专门用于病理问答的模型。
英文摘要：
Recent advances in large multimodal models have allowed for the development of interactive chat models that can converse and reason about pathology whole-slide images (WSIs). However, existing slide-level chat systems are often highly specialized, typically compressing WSIs into fixed slide-level embeddings or relying on multi-component pipelines, which can lose multi-scale detail and limit generalizability beyond the target task. We present GIANT (Gigapixel Image Agent for Navigating Tissue), a simple, training-free approach that lets general-purpose multimodal models navigate WSIs on their own, iteratively selecting multi-magnification crops and aggregating evidence over time. To evaluate generalizability in WSI question answering and to promote reproducibility, we introduce MultiPathQA, a benchmark suite spanning five clinical challenges and 934 questions over 868 unique WSIs. This includes a new set of 128 pathologist-authored multiple-choice questions designed to mirror real diagnostic search and multi-scale reasoning. Using GPT-5, GIANT outperforms models specialized for pathology question answering, achieving state-of-the-art performance on four out of five benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
