评估合成临床笔记的策略以用于医疗多模态AI
Evaluating Strategies for Synthesizing Clinical Notes for Medical Multimodal AI
AI总结：
本文研究了合成临床笔记的生成策略，通过优化提示设计和医学元数据，提升医疗多模态AI在分类和跨模态检索任务中的性能。
摘要：
多模态（MM）学习正在成为生物医学人工智能（AI）应用中具有前景的范式，整合互补的模态，突出患者健康的不同方面。大规模异质生物医学MM数据的缺乏限制了医疗AI应用中稳健模型的发展。在皮肤病领域，例如，皮肤病变数据集通常只包含与少量描述病情的元数据链接的图像，从而限制了MM数据整合在可靠和可推广的预测中的好处。最近在大语言模型（LLMs）中的进展使得能够合成图像发现的文本描述，可能允许图像和文本表示的结合。然而，LLMs并非专门训练用于医疗领域，其简单引入引发了在临床相关背景下产生幻觉的风险。本工作调查了生成合成文本临床笔记的策略，包括提示设计和医学元数据的包含，并评估其对MM架构的影响，以提高分类和跨模态检索任务的性能。在多个异质皮肤病数据集上的实验表明，合成临床笔记不仅提高了分类性能，特别是在领域转移的情况下，还解锁了跨模态检索能力，这是一个在训练过程中未显式优化的下游任务。
英文摘要：
Multimodal (MM) learning is emerging as a promising paradigm in biomedical artificial intelligence (AI) applications, integrating complementary modality, which highlight different aspects of patient health. The scarcity of large heterogeneous biomedical MM data has restrained the development of robust models for medical AI applications. In the dermatology domain, for instance, skin lesion datasets typically include only images linked to minimal metadata describing the condition, thereby limiting the benefits of MM data integration for reliable and generalizable predictions. Recent advances in Large Language Models (LLMs) enable the synthesis of textual description of image findings, potentially allowing the combination of image and text representations. However, LLMs are not specifically trained for use in the medical domain, and their naive inclusion has raised concerns about the risk of hallucinations in clinically relevant contexts. This work investigates strategies for generating synthetic textual clinical notes, in terms of prompt design and medical metadata inclusion, and evaluates their impact on MM architectures toward enhancing performance in classification and cross-modal retrieval tasks. Experiments across several heterogeneous dermatology datasets demonstrate that synthetic clinical notes not only enhance classification performance, particularly under domain shift, but also unlock cross-modal retrieval capabilities, a downstream task that is not explicitly optimized during training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
