arXiv 2411.165232025-10-06cs.CVcs.CL

LaB-RAG：标签增强的检索增强生成用于放射学报告生成

LaB-RAG: Label Boosted Retrieval Augmented Generation for Radiology Report Generation

Department of Computer Science（计算机科学系）；Center for Translational Data Science（转化数据科学中心）；Medical Scientist Training Program（医学科学家培训计划）；Section of Biomedical Data Science, Department of Medicine（医学部生物医学数据科学部门）

Steven Song, Anirudh Subramanyam, Irene Madejski, Robert L. Grossman

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出LaB-RAG，利用分类标签增强检索增强生成，结合小模型与预训练LLM生成放射学报告，在MIMIC-CXR和CheXpert Plus上优于其他检索方法，并与微调方法兼容。

摘要：

在当前图像字幕生成的范式中，深度学习模型被训练为从潜在特征的图像嵌入中生成文本。我们质疑需要微调大型定制模型以提高模型生成准确性的假设。在此，我们提出标签增强的检索增强生成（LaB-RAG），一种基于小模型的图像字幕生成方法，该方法利用分类标签形式的图像描述符来增强使用预训练大语言模型（LLM）的标准检索增强生成（RAG）。我们在MIMIC-CXR和CheXpert Plus数据集上研究了我们的方法在放射学报告生成（RRG）中的应用。我们认为，简单的分类模型结合零样本嵌入可以有效地将X射线转换为放射学特定标签的文本空间。结合标准RAG，我们表明这些派生的文本标签可以与通用领域的LLM一起用于生成放射学报告。我们从未针对该任务训练我们的生成语言模型或图像嵌入模型，也从未直接向LLM“展示”X射线，但我们证明，与其他基于检索的RRG方法相比，LaB-RAG在自然语言和放射学语言指标上取得了更好的结果，同时与其他微调的视觉-语言RRG模型相比也达到了有竞争力的结果。我们进一步进行了广泛的消融实验，以更好地理解LaB-RAG的组成部分。我们的结果表明，与微调方法具有更广泛的兼容性和协同作用，可进一步增强RRG性能。

英文摘要：

In the current paradigm of image captioning, deep learning models are trained to generate text from image embeddings of latent features. We challenge the assumption that fine-tuning of large, bespoke models is required to improve model generation accuracy. Here we propose Label Boosted Retrieval Augmented Generation (LaB-RAG), a small-model-based approach to image captioning that leverages image descriptors in the form of categorical labels to boost standard retrieval augmented generation (RAG) with pretrained large language models (LLMs). We study our method in the context of radiology report generation (RRG) over MIMIC-CXR and CheXpert Plus. We argue that simple classification models combined with zero-shot embeddings can effectively transform X-rays into text-space as radiology-specific labels. In combination with standard RAG, we show that these derived text labels can be used with general-domain LLMs to generate radiology reports. Without ever training our generative language model or image embedding models specifically for the task, and without ever directly "showing" the LLM an X-ray, we demonstrate that LaB-RAG achieves better results across natural language and radiology language metrics compared with other retrieval-based RRG methods, while attaining competitive results compared to other fine-tuned vision-language RRG models. We further conduct extensive ablation experiments to better understand the components of LaB-RAG. Our results suggest broader compatibility and synergy with fine-tuned methods to further enhance RRG performance.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。