重新审视用于3D CT报告生成的LLM适应：缩放与诊断先验研究
Revisiting LLM Adaptation for 3D CT Report Generation: A Study of Scaling and Diagnostic Priors
AI总结：
提出RAD3D-Prefix轻量级诊断先验框架，通过冻结大语言模型并融合多标签分类逻辑，在少量可训练参数下实现3D CT报告生成，优于全微调基线并展现强泛化性。
摘要：
多模态学习的最新进展，包括大型语言模型（LLM）和视觉-语言模型（VLM），已展现出对自然图像的强大适应性。然而，将其扩展到医学领域，特别是体积（3D）图像，由于高计算复杂度、体积依赖性和视觉特征与临床术语之间的语义差距而具有挑战性。在有限的医学数据上对LLM进行朴素微调常常导致过拟合和临床幻觉，其中语言流畅性优先于临床事实性。在本研究中，我们研究了用于体积CT报告生成的参数高效适应策略，并引入了RAD3D-Prefix，一种轻量级的诊断先验条件框架，最大限度地减少了对大量参数训练的需求。该模块将图像嵌入与多标签诊断分类逻辑相结合，保留了关键的临床细节，同时弥合了语义差距。通过保持LLM冻结，我们的方法需要最少的可训练参数，并减轻了在小规模、特定领域数据集上过拟合的风险。通过对从96.1M到1.6B参数的LLM进行系统研究，我们发现微调对较小的LLM最有益，而冻结较大的（约1B+）LLM并仅训练轻量级投影层在性能、泛化性和计算效率之间提供了优越的权衡。在多个自动指标和一项临床读者研究中，RAD3D-Prefix优于可比较的参数高效基线，并在使用比全微调替代方案少得多的可训练参数的情况下，展现出强大的域外泛化能力。
英文摘要：
Recent advances in multimodal learning, including large language models (LLMs) and vision-language models (VLMs), have demonstrated strong adaptability to natural images. However, extending their use to the medical domain, particularly for volumetric (3D) images, is challenging due to high computational complexity, volumetric dependencies and the semantic gap between visual features and clinical terminology. Naively fine-tuning LLMs on limited medical data often leads to overfitting and clinical hallucination, where linguistic fluency is prioritized over clinical factuality. In this study, we investigate parameter-efficient adaptation strategies for volumetric CT report generation and introduce RAD3D-Prefix, a lightweight diagnostic-prior conditioning framework that minimizes the need for extensive parameter training. This module integrates image embeddings with multi-label diagnostic classification logits, preserving critical clinical details while bridging the semantic gap. By keeping the LLM frozen, our method requires minimal trainable parameters and mitigates the risk of overfitting on small, domain-specific datasets. Through a systematic study spanning LLMs from 96.1M to 1.6B parameters, we find that fine-tuning is most beneficial for smaller LLMs, whereas freezing larger (~1B+ LLMs and training only lightweight projection layers provides a superior trade-off between performance, generalization, and computational efficiency. Across multiple automatic metrics and a clinical reader study, RAD3D-Prefix outperforms comparable parameter-efficient baselines and demonstrates strong out-of-domain generalization while using substantially fewer trainable parameters than fully fine-tuned alternatives.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
