SentZero：一种用于多任务零样本胸部X光分析增强的句子中心视觉-语言预训练

中文摘要 AI 辅助

基于配对的胸部X光（CXR）图像和放射学报告的视觉-语言（VL）预训练在医学图像理解方面显示出巨大潜力。然而，现有方法往往依赖于特定任务的微调，因为放射学报告冗长、临床密集，且难以与简单的零样本提示对齐。最近的句子级方法通过使用大型语言模型（LLMs）提取的临床短语部分解决了这一局限性，但它们在很大程度上忽视了放射学话语的内在特征。特别是，有限的正样本对多样性限制了进一步的提升，而临床上等价的句子经常在患者之间重复出现，在对比学习中造成假阴性。为了解决这些问题，我们提出了SentZero，一种用于零样本、多任务CXR分析的增强句子中心VL预训练框架。SentZero引入了基于LLM的抽象级句子结构和映射，以扩展正样本对的多样性，并增加了一个额外的损失项来缓解假阴性。我们进一步引入了句子条件残差调制视觉嵌入，使视觉特征能够适应每个输入句子的语义特征。在多个下游任务和数据集上，SentZero提高了零样本泛化能力，并优于先前的多任务零样本方法。

英文摘要

Vision-language (VL) pretraining using paired chest X-ray (CXR) images and radiology reports has shown strong potential for medical image understanding. However, existing methods often remain dependent on task-specific finetuning because radiology reports are lengthy, clinically dense, and difficult to align with simple zero-shot prompts. Recent sentence-level approaches partially address this limitation using clinical phrases extracted by large language models (LLMs), but they largely overlook the intrinsic characteristics of radiology discourse. In particular, limited positive-pair diversity constrains further gains, while clinically equivalent sentences frequently recur across patients, creating false negatives in contrastive learning. To address these issues, we propose SentZero, an enhanced sentence-centric VL pretraining framework for zero-shot, multi-task CXR analysis. SentZero introduces LLM-based abstract-level sentence structuring and mapping to expand positive-pair diversity, together with an additional loss term to mitigate false negatives. We further introduce sentence-conditioned residual modulation of visual embeddings, enabling visual features to adapt to the semantic characteristics of each input sentence. Across diverse downstream tasks and datasets, SentZero improves zero-shot generalization and outperforms prior multi-task zero-shot methods.

发表机构

• Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））

机构由 AI 辅助整理，请以论文原文为准。