2023全球人工智能技术创新大赛赛道一冠军方案
First Place Solution of 2023 Global Artificial Intelligence Technology Innovation Competition Track 1
AI总结:
本文提出2023全球人工智能技术创新大赛医学影像诊断报告生成赛道冠军方案,基于CPT-BASE结合跨度掩码预训练、迭代检索增强和噪声感知相似度分桶提示,最终在A榜和B榜均取得第一。
摘要:
本文介绍了我们在全球人工智能技术创新大赛赛道一:医学影像诊断报告生成中的冠军解决方案。我们选择CPT-BASE作为文本生成任务的基础模型。在预训练阶段,我们删除了CPT-BASE的掩码语言建模任务,改为重构词表,采用跨度掩码策略并逐步增加掩码比例来执行去噪自编码器预训练任务。在微调阶段,我们设计了迭代检索增强和噪声感知相似度分桶提示策略。检索增强构建了一个小型知识库,丰富了模型的输入信息,而相似度分桶进一步感知小型知识库中的噪声信息,引导模型基于相似度提示生成更高质量的诊断报告。令人惊喜的是,我们的单模型在A榜上取得了2.321的分数,多模型融合在A榜和B榜上的分数分别为2.362和2.320,最终在排行榜上获得第一名。
英文摘要:
In this paper, we present our champion solution to the Global Artificial Intelligence Technology Innovation Competition Track 1: Medical Imaging Diagnosis Report Generation. We select CPT-BASE as our base model for the text generation task. During the pre-training stage, we delete the mask language modeling task of CPT-BASE and instead reconstruct the vocabulary, adopting a span mask strategy and gradually increasing the number of masking ratios to perform the denoising auto-encoder pre-training task. In the fine-tuning stage, we design iterative retrieval augmentation and noise-aware similarity bucket prompt strategies. The retrieval augmentation constructs a mini-knowledge base, enriching the input information of the model, while the similarity bucket further perceives the noise information within the mini-knowledge base, guiding the model to generate higher-quality diagnostic reports based on the similarity prompts. Surprisingly, our single model has achieved a score of 2.321 on leaderboard A, and the multiple model fusion scores are 2.362 and 2.320 on the A and B leaderboards respectively, securing first place in the rankings.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。