LoC-Path：学习压缩用于病理多模态大语言模型
LoC-Path: Learning to Compress for Pathology Multimodal Large Language Models
AI总结：
LoC-Path通过压缩滑片图像特征，降低多模态模型的训练和推理成本，使在有限资源下实现高效病理大语言模型成为可能。
摘要：
整张滑片图像（WSI）大语言模型（MLLMs）难以构建和部署，因为吉像素滑片会诱导出数千个视觉标记，而只有少量区域具有诊断相关性。现有滑片级病理MLLMs通常结合了重载滑片级编码器和长视觉前缀，使得在有限计算资源下端到端的滑片级开发和部署成本高昂。我们重新审视这一领域，发现WSI瓷砖特征在全局和局部尺度上都高度冗余，而任务相关证据稀少且依赖查询。因此，我们引入LoC-Path，一种资源高效的滑片级MLLM，它在融合前进行压缩。LoC-Path使用稀疏标记合并（STM）和MAE预训练的重排器，将昂贵的滑片级编码替换为紧凑的潜在接口，然后使用标记重要性评分器（TIS）选择最相关的潜在特征，并使用交叉注意力路由适配器（CARA）将它们融合到少数LLM解码器层中。这种设计通过避免昂贵的滑片级编码和长视觉前缀，降低了多模态微调成本和推理时的延迟/内存消耗。大量实验表明，LoC-Path在与先前滑片级MLLMs竞争的同时，使在有限计算资源下端到端的开发和部署更加实用。
英文摘要：
Whole Slide Image (WSI) MLLMs are difficult to build and deploy because gigapixel slides induce thousands of visual tokens, while only a small fraction of regions is diagnostically relevant. Existing slide-level pathology MLLMs typically combine heavy slide-level encoders with long visual prefixes, making end-to-end slide-level development and deployment expensive under limited computational resources. We revisit this regime and show that WSI tile features are highly redundant at both global and local scales, while task-relevant evidence is sparse and query-dependent. We therefore introduce LoC-Path, a resource-efficient slide-level MLLM that compresses before fusion. LoC-Path uses a Sparse Token Merger (STM) and an MAE-pretrained resampler to replace expensive slide-level encoding with a compact latent interface, then uses a Token Importance Scorer (TIS) to select the most relevant latents and a Cross-Attention Routing Adapter (CARA) to fuse them into a few LLM decoder layers. This design lowers both multimodal tuning cost and inference-time latency/memory by avoiding heavy slide-level encoding and long visual prefixes. Extensive experiments show that LoC-Path remains competitive with prior slide-level MLLMs while making end-to-end development and deployment more practical under limited computational resources.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
