GAZE：基于视图级工具和文献检索的 grounded 零样本评估
GAZE: Grounded Agentic Zero-shot Evaluation with Viewer-Level Tools and Literature Retrieval on Rare Brain MRI
AI总结：
GAZE框架通过调用视图级工具和文献检索工具，使医学VLM在迭代方式下工作，实现对罕见脑MRI的零样本评估，提升病变定位和诊断准确率。
摘要：
视觉-语言模型（VLMs）通过单次前向传递读取图像并生成文本，而放射科医生通常多次检查图像并查阅文献后再撰写报告。我们介绍了GAZE（Grounded Agentic Zero-shot Evaluation），一个框架，使医学VLM能够以这种迭代方式工作，通过调用视图级工具（缩放、窗函数、对比度、边缘检测）和两个由美国国家医学图书馆（PubMed用于医学文献，Open-i用于放射图像）支持的检索工具，生成结构化输出并验证于模式，并记录完整的工具调用轨迹以确保可审计性。在NOVA基准测试中，包含906个脑MRI案例的906个案例涵盖281种罕见神经系统疾病，GAZE在交并比（IoU）0.3时达到58.2平均平均精度（mAP）的病变定位和34.9%的联合协议下的Top-1诊断准确率，无需任务特定的微调。在使用任何工具之前，结构化提示和模式验证的输出已经优于已发表的Gemini 2.0 Flash基线（20.2到29.4 mAP@0.3），因此框架设计本身是实验变量。工具使用对罕见病理学有不成比例的帮助：在IoU>0.3的病例比例从17%增加到58%（诊断有三个或更少例子）与25%到68%（常见条件（≥10个病例）），增益与参与度（Gemini 3 Flash：Cohen's d = 0.79，11.8次工具调用每案例；Gemini 2.0 Flash：工具在8.2%的案例中使用，无显著益处）相关。检索消融还揭示了模型依赖的权衡，其中诊断的增益可能伴随定位的损失，强化了在医学VLM中联合评估诊断、定位和描述的必要性。
英文摘要：
Vision-language models (VLMs) read an image and produce text in a single forward pass, whereas radiologists typically inspect an image several times and consult the literature before writing a report. We introduce GAZE (Grounded Agentic Zero-shot Evaluation), a framework that lets a medical VLM work in this iterative way by calling viewer-level tools (zoom, windowing, contrast, edge detection) and two retrieval tools backed by the U.S. National Library of Medicine (PubMed for medical literature, Open-i for radiological images), with structured outputs validated against a schema and full tool-call traces recorded for auditability. On NOVA, a benchmark of 906 brain MRI cases covering 281 rare neurological conditions, GAZE reaches 58.2 mean average precision (mAP) at intersection-over-union (IoU) 0.3 for lesion localisation and 34.9% Top-1 diagnostic accuracy under a joint protocol that scores captioning, diagnosis, and localisation from the image alone, without task-specific fine-tuning. Before any tool is used, structured prompting and schema-validated outputs already improve over the published Gemini 2.0 Flash baseline (20.2 to 29.4 mAP@0.3), so framework design is itself an experimental variable. Tool use helps rare pathologies disproportionately: the fraction of cases with IoU > 0.3 rises from 17% to 58% for diagnoses with three or fewer examples versus 25% to 68% for common conditions ($\geq$10 cases), with gains tracking engagement (Gemini 3 Flash: Cohen's d = 0.79, 11.8 tool calls per case; Gemini 2.0 Flash: tools used in 8.2% of cases, no significant benefit). Retrieval ablations additionally reveal a model-dependent trade-off in which gains in diagnosis can coincide with losses in localisation, reinforcing the case for joint evaluation of diagnosis, localisation, and captioning in medical VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
