面向病理图像解读的语言空间空间消息传递
Spatial Message Passing in Language Space for Pathology Image Interpretation
浏览论文内容
中文总结 AI 辅助
提出SLMP框架，在语言空间对病理图像执行空间推理，无需微调MLLM，提升肿瘤描述准确率，缩小通用与病理专用MLLM的性能差距。
中文摘要 AI 辅助
多模态大语言模型（MLLM）可从组织学图像生成病理描述，但千兆像素级全切片图像（WSI）超出其视觉上下文限制。标准的分块处理方案虽使WSI可处理，但破坏了定义肿瘤-基质界面及形态的组织邻域。我们提出空间语言消息传递（SLMP）框架，该框架完全在语言空间执行空间推理，具有天生的人类可读性。SLMP将WSI区域表示为空间文本图：分块作为节点，初始化为MLLM生成的描述，边编码空间邻接关系。对于每个分块，大语言模型（LLM）通过共享聚合策略整合相邻分块的语言消息来细化其描述，该策略在分块网格上表现为作用于文本而非学习到的嵌入的自适应局部核。此策略是可检查的提示，可通过文本梯度从模型观察到的组织表型中优化，无需微调MLLM权重即可实现从局部细胞背景到更广泛组织形态的自动语义优化。在代表性HER2和CAMELYON16区域上，SLMP在通用及病理专用主干模型的设置中均提升了分块级肿瘤描述准确率，提升幅度为3.3至19.6个百分点。随机邻域消融实验证实，这些提升源于空间上下文而非额外文本本身，且对优化后策略的检查揭示了可解释的、组织特异性决策规则。此外，无需更新或微调主干MLLM，SLMP即可显著提升通用MLLM性能并缩小其与病理专用模型的差距，为将空间推理纳入基于MLLM的病理分析提供了透明且灵活的机制。
英文摘要
Multimodal Large Language Models (MLLMs) can generate pathological descriptions from histological images, but gigapixel Whole Slide Images (WSIs) exceed their visual context limits. The standard tiling workaround makes WSIs tractable yet severs the tissue neighborhoods that define tumor-stroma interfaces and morphology. We introduce Spatial Language Message Passing (SLMP), a framework that performs spatial reasoning entirely in language space, human-readable by construction. SLMP represents a WSI region as a spatial text graph: tiles are nodes initialized with MLLM descriptions, and edges encode spatial adjacency. For each tile, an LLM refines its description by integrating language messages from adjacent tiles under a shared aggregation policy that, on the tile grid, acts as an adaptive local kernel operating on text rather than learned embeddings. This policy is an inspectable prompt that can be refined from model-observed tissue phenotypes via textual gradients, enabling automatic semantic optimization from local cellular context to broader tissue morphology without fine-tuning MLLM weights. On representative HER2 and CAMELYON16 regions, SLMP improves tile-level tumor description accuracy in settings spanning general-purpose and pathology-specialized backbones, with gains of +3.3 to +19.6 percentage points. Random-neighbor ablations confirm that these gains stem from spatial context rather than additional text alone, and inspecting the optimized policies reveals interpretable, tissue-specific decision rules. Besides, without any weight updates or fine-tuning the backbone MLLM, SLMP substantially improves general-purpose MLLMs and narrows its gap to pathology-specialized counterparts, offering a transparent and flexible mechanism for incorporating spatial reasoning into MLLM-based pathology analysis.
发表机构
- National Taiwan University（台湾大学）
- University of Oxford（牛津大学）
- ZYTCA Limited（ZYTCA有限公司）
机构由 AI 辅助整理，请以论文原文为准。
