arXiv 2605.249772026-05-24cs.CVcs.CL

通用增强，特定抑制：基于稀疏自编码器引导的医学视觉语言模型

Universal Boosts, Specific Suppressors: Sparse Autoencoder Steering of Medical Vision-Language Models

University of Zurich and University Hospital of Zurich（苏黎世大学及苏黎世大学医院）；Kobe University（Kobe大学）；ETH AI Center（苏黎世联邦理工学院人工智能中心）；ETH Zurich（苏黎世联邦理工学院）；Stanford University（斯坦福大学）；Zurich University of Applied Sciences（苏黎世应用科学大学）

Farhad Nooralahzadeh, Benjamin Gundersen, Nicolas Deperrois, Hidetoshi Matsuom, Mizuho Nishio, Thomas Frauenfelder, Ahmed Allam, Christian Blüthgen, Michael Moo… 展开作者

Farhad Nooralahzadeh, Benjamin Gundersen, Nicolas Deperrois, Hidetoshi Matsuom, Mizuho Nishio, Thomas Frauenfelder, Ahmed Allam, Christian Blüthgen, Michael Moor, Michael Krauthammer

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出一种无需权重更新的解码时残差引导方法，通过每token稀疏自编码器（SAE）对医学视觉语言模型进行干预，抑制幻觉并提升报告质量，在多个模型上取得显著改进。

摘要：

医学视觉语言模型（VLM）在生成胸部X光报告时经常出现幻觉：它们编造图像中不存在的发现，遗漏重要发现，或定位错误。我们通过解码时残差引导，基于每token稀疏自编码器（SAE）来缓解这一问题，无需权重更新：在后期层使用Top-$K$ SAE，针对临床错误进行因果引导，然后在推理时结合抑制/增强干预。在MIMIC-CXR测试集上，我们的纯推理方法提高了三个放射学VLM（RadVLM、LLaVA-Rad和CheXOne）生成报告的质量，临床复合指标的相对改进分别为+5.4%、+7.2%和+17.0%，并且所有骨干网络的GREEN得分均具有统计显著性。跨模型特征对齐表明，质量促进（增强）方向在不同架构间高度重叠，而与幻觉相关的（抑制）方向则是模型特定的。因此，可迁移的引导必须针对每个骨干网络进行抑制处理，而不是共享一个通用的抑制列表。相同的配方无需重新训练即可零样本迁移到IU-Xray（GREEN相对提升+7.7%），确认了所识别的特征是模型属性，而非训练语料库的属性。我们发布了因果特征集和一个交互式特征仪表板：https://cxr-sparse-feature-dashboard.netlify.app/。

英文摘要：

Medical vision-language models (VLMs) often hallucinate findings when generating chest X-ray reports: they fabricate findings that are not present in the image, miss important ones, or locate them incorrectly. We mitigate this without weight updates by decoding-time residual steering on a per-token sparse autoencoder (SAE) basis: Top-$K$ SAEs on late layers, causal steering against clinical errors, then combined suppress/boost intervention at inference time. On the MIMIC-CXR test split, our inference-only method improves the quality of generated reports for three radiology VLMs (RadVLM, LLaVA-Rad, and CheXOne), with relative improvements of +5.4%, +7.2%, and +17.0% in the clinical composite metric, and statistically significant GREEN gains on all backbones. A cross-model feature alignment shows that the quality-promoting (boost) directions overlap strongly across architectures, whereas hallucination-linked (suppress) directions are model-specific. Therefore, transferable steering must treat suppression per-backbone, rather than sharing a universal suppress list. The same recipe transfers zero-shot to IU-Xray (Green $+7.7\%$ rel.) without retraining, confirming that the identified features are properties of the model, not of the training corpus. We release causal feature sets and an interactive feature dashboard: https://cxr-sparse-feature-dashboard.netlify.app/.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。