利用医学分类法测量和对齐视觉-语言模型中的抽象能力
Measuring and Aligning Abstraction in Vision-Language Models with Medical Taxonomies
AI总结：
本文提出通过医学分类法量化和缓解视觉-语言模型中的抽象错误，引入灾难性抽象错误概念，并通过风险约束阈值和分类法感知微调减少严重错误至2%以下。
摘要：
视觉-语言模型在胸片分类任务中表现出强大的零样本性能，但标准的扁平指标无法区分临床轻微错误和严重错误。本文研究如何通过利用医学分类法来量化和缓解抽象错误。我们通过层次化指标对几种最先进的VLMs进行了基准测试，并引入了灾难性抽象错误来捕捉跨分支的错误。我们的结果揭示了尽管具有高扁平性能，VLMs与临床分类法之间仍存在显著的不一致。为了解决这一问题，我们提出了风险约束阈值和基于径向嵌入的分类法感知微调，这将严重抽象错误减少到低于2%，同时保持具有竞争力的性能。这些发现突显了层次化评估和表示层面对齐对于更安全、更具有临床意义的VLMs部署的重要性。
英文摘要：
Vision-Language Models show strong zero-shot performance for chest X-ray classification, but standard flat metrics fail to distinguish between clinically minor and severe errors. This work investigates how to quantify and mitigate abstraction errors by leveraging medical taxonomies. We benchmark several state-of-the-art VLMs using hierarchical metrics and introduce Catastrophic Abstraction Errors to capture cross-branch mistakes. Our results reveal substantial misalignment of VLMs with clinical taxonomies despite high flat performance. To address this, we propose risk-constrained thresholding and taxonomy-aware fine-tuning with radial embeddings, which reduce severe abstraction errors to below 2 per cent while maintaining competitive performance. These findings highlight the importance of hierarchical evaluation and representation-level alignment for safer and more clinically meaningful deployment of VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
