哪些医学问题需要推理依据？基于扰动敏感性的选择用于稳健问答
Which Medical Questions Deserve Rationales? Perturbation-Sensitive Selection for Robust QA
查看机构详情
- University of Memphis（孟菲斯大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
针对医学问答中推理依据稀缺且成本高的问题，提出RMS-RSP方法，通过扰动推理依据令牌的隐藏状态选择关键样本，在固定预算下提升模型对格式变化的稳健性，而非普遍准确率。
中文摘要 AI 辅助
医学问答数据集通常包含答案标签，而高质量的推理依据仍然稀缺、嘈杂或验证成本高昂。这改变了获取问题的性质：我们不再询问哪些问题应该被标注，而是询问在固定的令牌预算下，哪些已标注的问题应该接受推理依据监督。我们研究了该问题的离线版本，其中候选推理依据对选择器可见，但除非被选中，否则不会用于下游训练。我们提出了基于均方根稳健性的样本优先级排序（RMS-RSP），该方法仅扰动推理依据令牌处的隐藏状态，并测量由此引起的黄金答案与最佳干扰项之间边际的偏移。在五个医学问答数据集、MedGemma-4B-IT、三个训练种子、十个有预算的非RSP选择器以及一个无预算的全监督参考上，RMS-RSP提供了一个经过深思熟虑的限定结果。其锁定预算的准确率平均为60.61%，而随机选择为60.08%，仅在AfriMed-QA上具有统计学上显著的提升（+1.44个百分点）。其全预算准确率区域并不优于随机选择。然而，在三次答案选项重排后，RMS-RSP平均将稳健准确率和语义一致性分别提高了1.91和2.85个百分点，且在全部五个数据集上方向一致。使用每个池中的推理依据进行训练可将宏观准确率提升至63.74%，但消耗的推理依据令牌数量是原来的29至254倍，且并未一致地提高稳健性。这些发现并未确立普遍的准确率提升；相反，它们表明推理依据局部的边界敏感性可以识别出能改善对语义等价格式变化不变性的监督。
英文摘要
Medical question-answering datasets often contain answer labels, whereas high-quality rationales remain scarce, noisy, or costly to validate. This changes the acquisition question: rather than asking which questions should be labeled, we ask which already-labeled questions should receive rationale supervision under a fixed token budget. We study an offline version of this problem in which candidate rationales are visible to the selector but withheld from downstream training unless selected. We propose root-mean-square Robustness-based Sample Prioritization (RMS-RSP), which perturbs hidden states only at rationale tokens and measures the resulting shift in the gold-versus-best-distractor margin. Across five medical QA datasets, MedGemma-4B-IT, three training seeds, ten budgeted non-RSP selectors, and an unbudgeted full-supervision reference, RMS-RSP provides a deliberately qualified result. Its locked-budget accuracy is 60.61% on average versus 60.08% for Random, with a statistically resolved gain only on AfriMed-QA (+1.44 points). Its full-budget accuracy area is not better than Random. However, after three answer-option reorderings, RMS-RSP improves robust accuracy and semantic consistency by 1.91 and 2.85 points on average, respectively, with the same direction on all five datasets. Training on every pool rationale raises macro accuracy to 63.74%, but consumes 29--254 times more rationale tokens and does not uniformly improve robustness. These findings do not establish universal accuracy gains; they instead suggest that rationale-local boundary sensitivity can identify supervision that improves invariance to semantically equivalent formatting changes.
