MedGEN-Bench：面向开放式多模态医学生成的上下文纠缠基准
MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation
AI总结：
MedGEN-Bench提出一个上下文纠缠的开放式多模态医学生成基准，包含6,422对图像-文本，覆盖6种模态和15项临床任务，通过图像锚定指令评估图像生成与编辑，实验显示上下文增强提升相似度，且任务尚未饱和。
摘要：
医学视觉语言模型（VLM）日益被期望通过诊断文本和相关医学图像来支持临床工作流程。然而，当前的医学视觉基准存在三个反复出现的局限性：查询与图像错位（查询与特定图像实例的关联性弱）、封闭式格式（缩小了答案空间并鼓励基于捷径的预测），以及以文本为中心的输出范式（限制了对图像生成和图像编辑能力的评估）。我们引入了MedGEN-Bench，一个用于开放式多模态医学生成的基准。本手稿中报告的评估快照包含6,422对由临床专家和模型审查的图像-文本对，涵盖6种标准成像模态、15项临床任务和27个命名子任务。其中包括1,100对视觉问答（VQA）、3,872对图像编辑和1,450对上下文多模态生成。MedGEN-Bench的核心是上下文纠缠：指令的预期输出依赖于特定的图像实例，而不仅仅是任务措辞。该基准通过图像锚定指令将这一概念具体化，并将评估扩展到开放式多模态输出。其分层评估协议结合了可复现的基于参考的保真度和相似性度量，以及由医学VLM评判者进行的结构化、清单引导评估。我们评估了10个组合框架、2个专用图像编辑模型、3个统一模型和5个VLM。结果表明，图像输出任务尚未饱和。上下文增强将平均图像-指令相似度从0.273提高到0.372，而一项1,000例的医学专家审计显示，评判者评分与临床医生评级之间具有中等一致性。源代码和数据集可在以下网址获取：此https URL。
英文摘要：
Medical vision-language models (VLMs) are increasingly expected to support clinical workflows through diagnostic text and relevant medical images. However, current medical visual benchmarks have three recurring limitations: query-image misalignment from queries weakly grounded in specific image instances, closed-ended formats that narrow answer space and encourage shortcut-based prediction, and text-centric output paradigms that limit evaluation of image-generation and image-editing capabilities. We introduce MedGEN-Bench, a benchmark for open-ended multimodal medical generation. The evaluation snapshot reported in this manuscript comprises 6,422 image-text pairs reviewed by clinical experts and models, spanning 6 canonical imaging modalities, 15 clinical tasks, and 27 named subtasks. It includes 1,100 Visual Question Answering (VQA) pairs, 3,872 Image Editing pairs, and 1,450 Contextual Multimodal Generation pairs. MedGEN-Bench centers on contextual entanglement: dependence of an instruction's intended output on the particular image instance rather than on task wording alone. The benchmark operationalizes this concept through image-grounded instructions and extends evaluation to open-ended multimodal outputs. Its tiered evaluation protocol combines reproducible reference-based fidelity and similarity measures with a structured, checklist-guided assessment by a medical VLM judge. We evaluate 10 compositional frameworks, 2 dedicated image-editing models, 3 unified models, and 5 VLMs. The results show image-output tasks remain unsaturated. Contextual augmentation increases mean image-instruction similarity from 0.273 to 0.372, while a 1,000-case medical-expert audit shows moderate agreement between judge scores and clinician ratings. Source code and dataset are available at https://yangjj007.github.io/medgen.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
