GLINT：面向细粒度放射学表征的稀疏门控视觉-语言对齐
GLINT: Sparsely Gated Vision-Language Alignment for Fine-Grained Radiology Representations
AI总结：
针对放射学图像-报告全局对齐与局部病灶尺度不匹配的问题，提出GLINT框架，通过稀疏门控对齐和密集特征正则化实现零样本分类、定位和分割。
摘要：
放射学中的视觉-语言模型（VLM）通过利用临床工作流程中自然产生的图像-报告对，已成为一种可扩展的范式。然而，这种配对揭示了尺度上的不匹配：每个病灶仅占据图像的一小部分区域，但监督仅在全局图像-报告级别提供。这带来了一个核心挑战：先前的方法将权重密集地分布到所有补丁上，而不是集中在与给定查询相关的稀疏子集上。为了解决这个问题，我们提出了GLINT（门控语言-图像对齐）框架，该框架显式建模这种稀疏对应关系。在对齐方面，我们引入了稀疏门控对齐，这是一种新颖的架构，其中在单独的门控嵌入空间上的sigmoid门仅激活与每个文本查询相关的补丁，强制执行显式稀疏性。在表征方面，我们添加了密集特征正则化，将可训练编码器的中间特征锚定到冻结的自监督学习（SSL）教师模型上，从而保留门控所依赖的细粒度补丁特征。相同的方案适用于2D胸部X光片（CXR）和3D胸部计算机断层扫描（CT），分别基于DINOv3和V-JEPA 2.1构建。GLINT支持从自由文本查询进行零样本分类、定位和分割，据我们所知，这是首次在没有掩码监督的情况下在3D CT体积上展示零样本分割。值得注意的是，最显著的增益出现在零样本定位和分割上，这些任务需要稀疏的、特定于查询的定位，这与我们的设计意图一致。在下游评估中，GLINT在分类、报告生成和分割方面均优于SSL编码器和医学VLM。
英文摘要：
Vision-language models (VLMs) for radiology have emerged as a scalable paradigm by leveraging image-report pairs naturally produced in clinical workflows. However, this pairing reveals a mismatch in scale: each finding occupies only a small region of the image, yet supervision is provided only at the global image-report level. This poses a central challenge: prior approaches spread weight densely across all patches rather than concentrating on the sparse subset relevant to a given query. To address this, we present GLINT (Gated Language-Image alignmeNT), a framework that explicitly models this sparse correspondence. On the alignment side, we introduce Sparsely Gated Alignment, a novel architecture in which a sigmoid gate over a separate gate embedding space activates only the patches relevant to each textual query, enforcing explicit sparsity. On the representation side, we add Dense Feature Regularization, which anchors the trainable encoder's intermediate features to a frozen self-supervised learning (SSL) teacher, preserving the fine-grained patch features that the gate relies on. The same recipe applies to both 2D chest X-ray (CXR) and 3D chest computed tomography (CT), built with DINOv3 and V-JEPA 2.1, respectively. GLINT enables zero-shot classification, grounding, and segmentation from free-text queries, and to our knowledge is the first to demonstrate zero-shot segmentation on 3D CT volumes without mask supervision. Notably, the most pronounced gains arise on zero-shot grounding and segmentation, where sparse, query-specific localization is required, consistent with our design intent. In downstream evaluation, GLINT outperforms both SSL encoders and medical VLMs on classification, report generation, and segmentation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
