知识到视觉：通过知识分解进行视觉属性推理以实现异常定位
Knowledge to Sight: Reasoning over Visual Attributes via Knowledge Decomposition for Abnormality Grounding
AI总结：
针对医学图像异常定位中通用VLM的不足，提出K2Sight框架，通过将临床概念分解为视觉属性并编码为指令提示，实现数据高效训练，仅用1.5%数据训练的紧凑模型性能媲美7B+模型。
摘要：
在这项工作中，我们解决了医学图像中异常定位的问题，其目标是根据文本描述定位临床发现。虽然通用视觉-语言模型（VLMs）在自然定位任务中表现出色，但由于罕见、组合性和领域特定的术语与视觉模式对齐不良，它们常常在医学领域遇到困难。专门的医学VLM通过大规模领域预训练来解决这一挑战，但代价是大量的标注和计算资源。为了克服这些限制，我们提出了知识到视觉（K2Sight）框架，该框架通过将临床概念分解为可解释的视觉属性（如形状、密度和解剖位置）来引入结构化语义监督。这些属性从领域本体中提取，并编码为简洁的指令式提示，在训练期间指导区域-文本对齐。与传统的报告级监督不同，我们的方法明确地桥接了领域知识和空间结构，使得紧凑模型的数据高效训练成为可能。我们仅使用最先进的医学VLM所需数据的1.5%，训练了具有0.23B和2B参数的紧凑模型。尽管模型规模小且训练数据有限，这些模型的性能与7B+医学VLM相当或更好，在$mAP_{50}$上最高提升了9.82%。代码和模型：https://lijunrio.github.io/K2Sight/。
英文摘要：
In this work, we address the problem of grounding abnormalities in medical images, where the goal is to localize clinical findings based on textual descriptions. While generalist Vision-Language Models (VLMs) excel in natural grounding tasks, they often struggle in the medical domain due to rare, compositional, and domain-specific terms that are poorly aligned with visual patterns. Specialized medical VLMs address this challenge via large-scale domain pretraining, but at the cost of substantial annotation and computational resources. To overcome these limitations, we propose \textbf{Knowledge to Sight (K2Sight)}, a framework that introduces structured semantic supervision by decomposing clinical concepts into interpretable visual attributes, such as shape, density, and anatomical location. These attributes are distilled from domain ontologies and encoded into concise instruction-style prompts, which guide region-text alignment during training. Unlike conventional report-level supervision, our approach explicitly bridges domain knowledge and spatial structure, enabling data-efficient training of compact models. We train compact models with 0.23B and 2B parameters using only 1.5\% of the data required by state-of-the-art medical VLMs. Despite their small size and limited training data, these models achieve performance on par with or better than 7B+ medical VLMs, with up to 9.82\% improvement in $mAP_{50}$. Code and models: \href{https://lijunrio.github.io/K2Sight/}{\textcolor{SOTAPink}{https://lijunrio.github.io/K2Sight/}}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
