病理学组合检索的组织病理学多模态嵌入
Histopathology Multi-modal Embedding for Pathology Composed Retrieval
AI总结：
提出HOMIE框架，将生成式多模态大语言模型适配为病理检索专家，解决组合查询中的架构、任务和领域不匹配问题，在病理组合检索基准上显著优于现有方法。
摘要：
为了克服预测AI的黑箱特性和生成模型的幻觉风险，基于检索的模型为病理临床工作流提供了一种可解释、基于证据的范式。然而，现实世界的临床查询本质上是交织的（例如，病理图像和文本）。当前的双编码器存在\textbf{架构不匹配}问题，缺乏融合此类组合查询的机制。为此，我们正式定义了病理组合检索（PCR）任务。虽然多模态大语言模型（MLLM）具有深度融合能力，但直接应用它们会暴露\textbf{任务不匹配}和\textbf{领域不匹配}问题。为了解决这些挑战，我们提出了HOMIE，一个模型无关的适配框架，可将任何生成式MLLM转化为专门的病理检索专家。在我们新引入的PCR基准上评估，轻量级的2B参数HOMIE变体大幅优于现有范式，在组合检索上显著超过专门的7B病理MLLM和双编码器，同时在传统简单检索上保持强劲性能。项目页面见此https URL。
英文摘要：
To overcome the black-box nature of predictive AI and the hallucination risks of generative models, retrieval-based models offer an interpretable, evidence-based paradigm for pathology clinical workflow. However, real-world clinical queries are inherently interleaved (e.g., pathology images and text). Current dual-encoders suffer from an \textbf{Architectural Mismatch}, lacking the mechanism to fuse such composed queries. To address this, we formalize the task of Pathology Composed Retrieval (PCR). While Multimodal Large Language Models (MLLMs) offer deep-fusion capabilities, directly applying them exposes a \textbf{Task Mismatch} and a \textbf{Domain Mismatch}. To resolve these challenges, we propose HOMIE, a model-agnostic adaptation framework that transforms any generative MLLM into a specialized pathology retrieval expert. Evaluated on our newly introduced PCR Benchmark, a lightweight 2B-parameter HOMIE variant substantially outperforms existing paradigms, surpassing specialized 7B pathology MLLMs and dual-encoders by large margins on composed retrieval, while maintaining strong performance on traditional simple retrieval. The project page is available at https://qfchou.github.io/HOMIE_page/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
