发表机构
ETH Zurich; Agency for Science, Technology and Research (A*STAR)（苏黎世联邦理工学院; 新加坡科技研究局）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出AuthEval框架，通过记录和评分提议与所选行动来评估医学视觉语言助手的权威差距，揭示评估单位对结论的影响。
AI 中文摘要
医学视觉语言模型可以提出皮肤病变应被多紧急地审查，但当地服务机构保留根据转诊政策、容量和当地持有的患者背景接受或替换该提议的权威。因此，提议质量和所选行动质量是不同的评估目标，只有当当地审查保留了预期的行动分数时，基准证据才能在它们之间转移。我们引入了AuthEval，一个日志记录和评估框架，它记录两个行动，根据声明的当地标准对所选行动进行评分，并在可行的情况下，根据同一规则对拒绝的提议进行评分。它仅在记录支持的情况下报告由此产生的权威差距。因为差距是提议变更率和变更案例上的平均分数变化的乘积，仅该比率本身既不能决定其大小也不能决定其符号。在ISIC 2019上，使用MedGemma和模拟的当地审查，两个具有相似变更率的约束制度在容量下产生了乐观的图像相等差距（+0.744模拟器单位），但在安全下没有检测到差距。声明的评估单位也很重要：在混合约束下，当权重从图像转移到病变感知聚类时，差距从+0.374反转为-0.206。因此，AuthEval澄清了一项研究的记录是否支持关于模型、工作流程或两者的主张。
英文摘要
Medical vision-language models can propose how urgently a skin lesion should be reviewed, but the local service retains authority to accept or replace that proposal under referral policy, capacity, and locally held patient context. Proposal quality and selected-action quality are therefore distinct evaluation targets, and benchmark evidence transfers between them only when local review preserves the expected action score. We introduce AuthEval, a logging and evaluation framework that records both actions, scores the selected action under declared local criteria, and, where feasible, scores the declined proposal under the same rule. It reports the resulting authority gap only when the record supports it. Because the gap is the product of the proposal-change rate and the mean score change on changed cases, that rate alone determines neither its magnitude nor its sign. On ISIC 2019, with MedGemma and simulated local review, two constraint regimes with similar change rates produced an optimistic image-equal gap under capacity ($+0.744$ simulator units) but no detectable gap under safety. The declared evaluation unit also mattered: under mixed constraints the gap reversed from $+0.374$ to $-0.206$ when weighting shifted from image to lesion-aware cluster. AuthEval thus clarifies whether a study's records support claims about the model, the workflow, or both.
CommentsAccepted at 2nd Emerging LLM/LMM Applications in Medical Imaging (ELAMI) 2026, held in conjunction with MICCAI 2026. To appear in the Springer proceedings
