MedMO：为医学图像构建和理解多模态大语言模型
MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images
AI总结：
MedMO是一种基于通用MLLM架构构建的医学多模态基础模型，通过多阶段训练提升跨模态和任务的性能，超越现有开源基线，在医学图像识别和报告生成中取得显著提升。
摘要：
多模态大语言模型已取得快速进展，但其在医学领域的应用受到领域覆盖有限、模态对齐不完美和基础推理不足的限制。我们介绍了MedMO，一种基于通用MLLM架构构建的医学多模态基础模型，仅在大规模领域专用数据上进行训练。MedMO采用多阶段训练方案，包括跨模态预训练以对齐异构视觉编码器与医学语言骨干，指令微调包含跨captioning、VQA、报告生成、检索和边界框疾病定位的多任务监督，以及强化学习与可验证奖励相结合，通过事实性检查和框级GIoU信号来提高在挑战性临床场景中的空间定位和逐步推理能力。在跨模态和任务上，MedMO超越了强大的开源医学基线。MedMO-8B-Next在VQA基准测试中取得一致提升，平均比Fleming-VL-8B提升6.6%，包括在MMMU-Med上提升6.0%、在PMC-VQA上提升9.8%、在MedXpertQA上提升21.3%。在文本基于的问答中，它比Fleming-VL-8B提升14.4%，这由在MMLU-Med上的8.4%提升和在MedQA上的30.1%提升推动。在医学报告生成方面，它在MIMIC-CXR上提升6.7%。MedMO-8B-Next还展示了强大的定位性能，达到56.1 IoU在Bacteria上，比Fleming-VL-8B高出47.8 IoU。在较小的规模下，MedMO-4B-Next仍然具有竞争力，并在VQA、QA和报告生成方面超越Fleming-VL-8B。覆盖放射学、眼科和病理学显微镜的评估进一步确认了广泛的跨模态泛化能力。该项目可在https://genmilab.github.io/MedMO-Page上获得。
英文摘要：
Multimodal large language models have advanced rapidly, but their adoption in medicine is constrained by limited domain coverage, imperfect modality alignment, and insufficient grounded reasoning. We introduce MedMO, a medical multimodal foundation model built on a general MLLM architecture and trained exclusively on large-scale domain-specific data. MedMO uses a multi-stage training recipe that includes cross-modal pretraining to align heterogeneous visual encoders with a medical language backbone, instruction tuning with multi-task supervision spanning captioning, VQA, report generation, retrieval, and bounding-box disease localization, and reinforcement learning with verifiable rewards that combine factuality checks with a box-level GIoU signal to improve spatial grounding and step-by-step reasoning in challenging clinical settings. Across modalities and tasks, MedMO surpasses strong open-source medical baselines. MedMO-8B-Next achieves consistent gains on VQA benchmarks, improving by 6.6% on average over Fleming-VL-8B, including gains of 6.0% on MMMU-Med, 9.8% on PMC-VQA, and 21.3% on MedXpertQA. On text-based QA, it improves by 14.4% over Fleming-VL-8B, driven by gains of 8.4% on MMLU-Med and 30.1% on MedQA. For medical report generation, it improves by 6.7% on MIMIC-CXR. MedMO-8B-Next also demonstrates strong grounding performance, reaching 56.1 IoU on Bacteria, which is a 47.8 IoU gain over Fleming-VL-8B. At smaller scale, MedMO-4B-Next remains competitive and exceeds Fleming-VL-8B across VQA, QA, and report generation. Evaluations spanning radiology, ophthalmology, and pathology microscopy further confirm broad cross-modality generalization. Project is available at https://genmilab.github.io/MedMO-Page
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
