发表机构
Global College, Shanghai Jiao Tong University; Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences; Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学全球学院; 中国科学院深圳先进技术研究院; 上海交通大学医学院附属瑞金医院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对放射科报告生成任务中多粒度医学知识提取不足的问题，提出EMGKT方法，结合FGKD训练任务与疾病诊断专家分类器混合模型，在公开数据集上验证了其有效性与可迁移性。
AI 中文摘要
放射科报告生成可从X光图像自动生成临床描述，从而显著提高放射科医生的效率。该任务颇具挑战性，因为它需要医学知识来准确识别疾病并以专业方式进行描述。然而，现有方法往往忽视了在描述关键区域时增强医学知识的重要性，而这种能力要求模型有效提取和聚合多粒度知识。因此，我们在此提出一种新颖且紧凑的高效多粒度知识迁移（Efficient Multi-Granularity Knowledge Transfer，EMGKT）方法来解决上述问题。首先，我们使用医学视觉-语言模型编码全局知识嵌入，该模型提供上下文医学知识。此外，我们设计了一种新颖的细粒度知识蒸馏（Fine-Grained Knowledge Distillation，FGKD）训练任务，用于高效提取细粒度知识。具体而言，FGKD训练任务包含教师嵌入和学生嵌入：教师嵌入通过额外先验进行编码；学生嵌入则通过知识蒸馏从教师嵌入中学习。在推理阶段，学生嵌入用于增强细粒度知识，而教师嵌入被舍弃，这使得计算成本可忽略且无需额外先验。最后，我们进一步开发了一种疾病诊断专家分类器的混合模型以增强知识提取，这些分类器使用疾病嵌入进行初始化，并被建模为不同专家以应对各种粒度特征。值得注意的是，EMGKT可高效应用于大多数现有方法。我们在两个广泛使用的公开数据集和多个基准上进行了大量实验，证明了EMGKT的有效性和可迁移性。
英文摘要
Radiology report generation can automatically generate clinical descriptions from X-ray images, thereby significantly improving the efficiency of radiologists. This task is challenging because it requires medical knowledge to accurately identify diseases and describe them in a professional manner. However, existing methods often overlook the importance of enhancing medical knowledge in describing pivotal areas, a capability that requires models to effectively extract and aggregate knowledge at multiple levels of granularity. Accordingly, we herein propose a novel and compact Efficient Multi-Granularity Knowledge Transfer (\textbf{EMGKT}) method to address the above issues. First, we encode global knowledge embeddings using a medical vision-language model, which provides contextual medical knowledge. Moreover, we devise a novel Fine-Grained Knowledge Distillation (FGKD) training task which efficiently extract fine-grained knowledge. Specifically, the FGKD training task contains teacher embeddings and student embeddings. Teacher embeddings are encoded using extra priors; while student embeddings are learned from the teacher embeddings through knowledge distillation. During inference, the student embeddings are used to enhance fine-grained knowledge while the teacher embeddings are discarded, resulting in negligible computational costs and no need for extra priors. Finally, we further develop a mixture of disease diagnosis expert classifiers to enhance knowledge extraction. The classifiers are initialized using disease embeddings and are modeled as different experts to address various granularity features. Notably, \textbf{EMGKT} can be efficiently applied to most existing methods. Extensive experiments are conducted on two widely-used public datasets and various baselines, which demonstrates the effectiveness and transferability of \textbf{EMGKT}.
