MoMA：用于组织病理学图像分析的结合基于多头注意力的知识蒸馏的动量对比学习
MoMA: Momentum Contrastive Learning with Multi-head Attention-based Knowledge Distillation for Histopathology Image Analysis
AI总结：
针对计算病理学特定任务优质数据匮乏的问题，提出MoMA方法，通过结合多头注意力的动量对比学习实现无源数据的师生知识蒸馏，在多场景下性能优于相关方法，还为不同任务的学习策略提供指导。
摘要：
毫无疑问，先进的人工智能模型与高质量数据是开发计算病理学工具取得成功的关键。尽管病理学数据的总体规模持续增长，但由于患者数据存在隐私与伦理问题等多重原因，针对特定任务时优质数据匮乏是普遍问题。本研究提出利用知识蒸馏（即借助现有模型学习新的目标模型）来攻克计算病理学领域的此类难题。具体而言，我们采用师生框架，在无法直接访问源数据的情况下，从预训练的教师模型中学习目标模型，并通过结合多头注意力机制的动量对比学习来蒸馏相关知识，该机制可生成一致且具备上下文感知能力的特征表示。这使得目标模型既能吸收教师模型的有效表征，又能无缝适配目标数据的独特特性。我们在多种场景下对所提方法进行了严格评估，这些场景中教师模型分别在与目标模型相同、相关及无关的分类任务上完成训练。实验结果表明，我们的方法在向不同领域和任务迁移知识时具备准确性与鲁棒性，性能优于其他相关方法。此外，研究结果还为计算病理学中不同任务类型与场景下的学习策略提供了指导。代码可在以下链接获取：https://github.com/trinhvg/MoMA。
英文摘要：
There is no doubt that advanced artificial intelligence models and high quality data are the keys to success in developing computational pathology tools. Although the overall volume of pathology data keeps increasing, a lack of quality data is a common issue when it comes to a specific task due to several reasons including privacy and ethical issues with patient data. In this work, we propose to exploit knowledge distillation, i.e., utilize the existing model to learn a new, target model, to overcome such issues in computational pathology. Specifically, we employ a student-teacher framework to learn a target model from a pre-trained, teacher model without direct access to source data and distill relevant knowledge via momentum contrastive learning with multi-head attention mechanism, which provides consistent and context-aware feature representations. This enables the target model to assimilate informative representations of the teacher model while seamlessly adapting to the unique nuances of the target data. The proposed method is rigorously evaluated across different scenarios where the teacher model was trained on the same, relevant, and irrelevant classification tasks with the target model. Experimental results demonstrate the accuracy and robustness of our approach in transferring knowledge to different domains and tasks, outperforming other related methods. Moreover, the results provide a guideline on the learning strategy for different types of tasks and scenarios in computational pathology. Code is available at: \url{https://github.com/trinhvg/MoMA}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
