MedP-CLIP：具有区域感知提示集成的医学CLIP
MedP-CLIP: Medical CLIP with Region-Aware Prompt Integration
AI总结：
提出MedP-CLIP，一种区域感知医学视觉语言模型，通过特征级区域提示集成机制，支持多种提示形式，在保持全局上下文的同时聚焦局部区域，在零样本识别、交互式分割等任务中显著优于基线方法。
摘要：
对比语言-图像预训练（CLIP）通过大规模文本-图像对齐在全图理解和零样本迁移方面表现出色。然而，医学图像分析的核心往往在于对特定解剖结构或病变区域的细粒度理解。因此，精确理解医学专业人员或感知模型提供的感兴趣区域（RoI）信息变得至关重要。为满足这一需求，我们提出MedP-CLIP，一种区域感知医学视觉语言模型（VLM）。MedP-CLIP创新性地整合了医学先验知识，并设计了特征级区域提示集成机制，使其能够灵活响应各种提示形式（如点、边界框、掩码），同时在关注局部区域时保持全局上下文感知。我们在精心构建的大规模数据集（包含超过640万张医学图像和9730万个区域级标注）上预训练该模型，使其具备跨疾病、跨模态的细粒度空间语义理解能力。实验表明，MedP-CLIP在多种医学任务中显著优于基线方法，包括零样本识别、交互式分割以及赋能多模态大语言模型。该模型为医学AI提供了一个可扩展、即插即用的视觉骨干，结合了整体图像理解与精确区域分析。
英文摘要：
Contrastive Language-Image Pre-training (CLIP) has demonstrated outstanding performance in global image understanding and zero-shot transfer through large-scale text-image alignment. However, the core of medical image analysis often lies in the fine-grained understanding of specific anatomical structures or lesion regions. Therefore, precisely comprehending region-of-interest (RoI) information provided by medical professionals or perception models becomes crucial. To address this need, we propose MedP-CLIP, a region-aware medical vision-language model (VLM). MedP-CLIP innovatively integrates medical prior knowledge and designs a feature-level region prompt integration mechanism, enabling it to flexibly respond to various prompt forms (e.g., points, bounding boxes, masks) while maintaining global contextual awareness when focusing on local regions. We pre-train the model on a meticulously constructed large-scale dataset (containing over 6.4 million medical images and 97.3 million region-level annotations), equipping it with cross-disease and cross-modality fine-grained spatial semantic understanding capabilities. Experiments demonstrate that MedP-CLIP significantly outperforms baseline methods in various medical tasks, including zero-shot recognition, interactive segmentation, and empowering multimodal large language models. This model provides a scalable, plug-and-play visual backbone for medical AI, combining holistic image understanding with precise regional analysis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
