AutoRad-Lung：一种放射组学引导提示的自回归视觉语言模型用于肺结节恶性预测
AutoRad-Lung: A Radiomic-Guided Prompting Autoregressive Vision-Language Model for Lung Nodule Malignancy Prediction
AI总结：
针对CLIP-Lung模型依赖主观注释、推理时无法用文本及忽略先验知识等局限，提出AutoRad-Lung，结合自回归预训练VLM与放射组学提示，捕捉像素级差异并动态优化提示，提升肺结节恶性预测性能。
摘要：
肺癌仍然是全球癌症相关死亡的主要原因之一。早期诊断的一个关键挑战是区分具有相似视觉特征和接近注释分数的不确定病例。在临床实践中，放射科医生依赖从计算机断层扫描（CT）图像中提取的定量、手工制作的放射组学特征，而近期研究主要集中于深度学习解决方案。最近，视觉语言模型（VLM），特别是基于对比语言-图像预训练（CLIP）的模型，因其将文本知识整合到肺癌诊断中的能力而受到关注。尽管CLIP-Lung模型已显示出有前景的结果，我们识别出以下潜在局限性：（a）依赖放射科医生注释的属性，这些属性本质上主观且易出错，（b）仅在训练期间使用文本信息，限制了推理时的直接适用性，以及（c）基于卷积的视觉编码器具有随机初始化的权重，忽略了先验知识。为解决这些局限性，我们引入了AutoRad-Lung，它将一个自回归预训练的VLM与从手工制作的放射组学生成的提示相结合。AutoRad-Lung使用大规模自回归图像模型（AIMv2）的视觉编码器，该模型通过多模态自回归目标进行预训练。鉴于肺肿瘤通常较小、形状不规则且与健康组织视觉相似，AutoRad-Lung通过捕捉像素级差异，相对于基于CLIP的对应模型提供了显著优势。此外，我们引入了条件上下文优化，根据输入放射组学动态生成上下文特定的提示，改善跨模态对齐。
英文摘要：
Lung cancer remains one of the leading causes of cancer-related mortality worldwide. A crucial challenge for early diagnosis is differentiating uncertain cases with similar visual characteristics and closely annotation scores. In clinical practice, radiologists rely on quantitative, hand-crafted Radiomic features extracted from Computed Tomography (CT) images, while recent research has primarily focused on deep learning solutions. More recently, Vision-Language Models (VLMs), particularly Contrastive Language-Image Pre-Training (CLIP)-based models, have gained attention for their ability to integrate textual knowledge into lung cancer diagnosis. While CLIP-Lung models have shown promising results, we identified the following potential limitations: (a) dependence on radiologists' annotated attributes, which are inherently subjective and error-prone, (b) use of textual information only during training, limiting direct applicability at inference, and (c) Convolutional-based vision encoder with randomly initialized weights, which disregards prior knowledge. To address these limitations, we introduce AutoRad-Lung, which couples an autoregressively pre-trained VLM, with prompts generated from hand-crafted Radiomics. AutoRad-Lung uses the vision encoder of the Large-Scale Autoregressive Image Model (AIMv2), pre-trained using a multi-modal autoregressive objective. Given that lung tumors are typically small, irregularly shaped, and visually similar to healthy tissue, AutoRad-Lung offers significant advantages over its CLIP-based counterparts by capturing pixel-level differences. Additionally, we introduce conditional context optimization, which dynamically generates context-specific prompts based on input Radiomics, improving cross-modal alignment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
