RxScribe Bench：用于评估视觉语言模型在印度门诊处方上的多轴基准
RxScribe Bench: A Multi-Axis Benchmark for Evaluating Vision-Language Models on Indian Outpatient Prescriptions
- Zstate AI
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对处方转录错误不可互换的问题，提出多轴基准RxScribe Bench，按临床严重性分解性能，评估发现无单一视觉语言模型在所有轴上占优。
AI中文摘要：
处方转录错误是不可互换的。一个编造药物的模型与一个误读清晰剂量的模型会带来截然不同的临床风险，然而处方转录准确性通常以单一的混合数字报告，将两者视为等同。我们引入了RxScribe Bench，一个用于评估视觉语言模型在手写处方数字化方面的基准，该基准将性能分解为与临床严重程度相关的四个轴，而不是将所有内容折叠成一个单一的汇总分数。仅给定处方图像和输出模式，模型生成结构化记录，然后逐字段与人工编写的相同形状的真实值进行比较，每个字段还标注了可见性和清晰度。这四个轴隔离了不同的失败模式，即正确性、幻觉、参与度和鲁棒性。鲁棒性轴在支持样本低于最低证据阈值时，会保留其困难字段的结果，而不是报告一个结果。我们在每张图像的独立冷运行中，对前沿视觉语言模型在真实处方上进行了评估，发现没有单一模型在所有四个轴上均获胜。
英文摘要：
Prescription transcription errors are not interchangeable. A model that fabricates a drug and a model that misreads a legible dose pose very different clinical risks, yet prescription-transcription accuracy is typically reported as a single blended figure that treats the two as equivalent. We introduce RxScribe Bench, a benchmark for evaluating vision-language models on handwritten prescription digitization that decomposes performance into four axes tied to clinical severity, rather than folding everything into a single aggregated score. Given only a prescription image and an output schema, a model produces a structured record, which is then compared field by field against a human-authored ground truth of identical shape, with each field also labeled for visibility and legibility. The four axes isolate distinct failure modes, namely Correctness, Hallucination, Engagement, and Robustness. The Robustness axis withholds its hard-field results rather than reporting one when the supporting sample falls below a minimum-evidence threshold. We evaluate frontier vision-language models on real prescriptions across independent cold runs per image, and find that no single model wins across all four axes.
