KorMedMCQA-V: 一种用于评估视觉语言模型在韩国医学资格考试中多模态多项选择问答能力的基准测试
KorMedMCQA-V: A Multimodal Benchmark for Evaluating Vision-Language Models on the Korean Medical Licensing Examination
AI总结：
KorMedMCQA-V是一个用于评估视觉语言模型在韩国医学资格考试中多模态多项选择问答能力的基准测试，展示了不同模型在医疗领域中的表现差异。
摘要：
我们介绍了KorMedMCQA-V，一个具有韩国医学资格考试风格的多模态多项选择问答基准测试，用于评估视觉语言模型（VLMs）。该数据集包含1,534个问题，其中2,043张相关图像来自韩国医学资格考试（2012-2023年），其中大约30%的问题包含多个图像，需要跨图像证据整合。图像涵盖临床模态，包括X光、计算机断层扫描（CT）、心电图（ECG）、超声波、内窥镜和其他医学图像。我们对50多个VLMs进行了评估，涵盖专有和开源类别，横跨通用型、医学专用型和韩国专用型家族，在统一的零样本评估协议下进行评估。最好的专有模型（Gemini-3.0-Pro）达到96.9%的准确率，最好的开源模型（Qwen3-VL-32B-Thinking）达到83.7%，而最好的韩国专用模型（VARCO-VISION-2.0-14B）仅为43.2%。我们进一步发现，以推理为导向的模型变体在指令训练的模型上可提升高达20个百分点，医学领域专业化在强通用基础模型上产生的增益不一致，所有模型在多图像问题上表现下降，不同成像模态的性能差异显著。通过补充纯文本的KorMedMCQA基准测试，KorMedMCQA-V形成了一套统一的评估套件，用于评估韩国医学推理在纯文本和多模态条件下的表现。该数据集可通过Hugging Face Datasets获取：https://huggingface.co/datasets/seongsubae/KorMedMCQA-V。
英文摘要：
We introduce KorMedMCQA-V, a Korean medical licensing-exam-style multimodal multiple-choice question answering benchmark for evaluating vision-language models (VLMs). The dataset consists of 1,534 questions with 2,043 associated images from Korean Medical Licensing Examinations (2012-2023), with about 30% containing multiple images requiring cross-image evidence integration. Images cover clinical modalities including X-ray, computed tomography (CT), electrocardiography (ECG), ultrasound, endoscopy, and other medical visuals. We benchmark over 50 VLMs across proprietary and open-source categories-spanning general-purpose, medical-specialized, and Korean-specialized families-under a unified zero-shot evaluation protocol. The best proprietary model (Gemini-3.0-Pro) achieves 96.9% accuracy, the best open-source model (Qwen3-VL-32B-Thinking) 83.7%, and the best Korean-specialized model (VARCO-VISION-2.0-14B) only 43.2%. We further find that reasoning-oriented model variants gain up to +20 percentage points over instruction-tuned counterparts, medical domain specialization yields inconsistent gains over strong general-purpose baselines, all models degrade on multi-image questions, and performance varies notably across imaging modalities. By complementing the text-only KorMedMCQA benchmark, KorMedMCQA-V forms a unified evaluation suite for Korean medical reasoning across text-only and multimodal conditions. The dataset is available via Hugging Face Datasets: https://huggingface.co/datasets/seongsubae/KorMedMCQA-V.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
