AI 中文总结
该研究针对临床多智能体系统,发现智能体间存在符合社会直觉的捷径级联博弈,仅独立于自我报告的裁判能捕捉该博弈,相关结果对临床决策支持系统的可靠性评估有重要意义。
AI 中文摘要
临床决策支持正朝着由语言模型智能体组成的、在共享工作空间中进行审议的委员会方向发展。我们探究此类委员会是否会被捷径——即基准奖励但临床医生会忽略的线索——所博弈。我们在涵盖文本(MedQA-USMLE、MedMCQA、MIMIC-CXR报告)、影像(NIH ChestX-ray14、MIMIC-CXR-JPG、CheXpert)和表格ICU记录(SUPPORT2)的6个公共数据集的7个队列上开展研究。Gemini委员会单独抵抗这些线索时,翻转率为5%-16%;但一种符合社会直觉的捷径会传播:当两个对等智能体断言相同错误答案时,被测试的保留智能体在38%的情况下会采纳该错误答案,虚假的“预筛选”系统标记也会产生同样结果,且在两种能力层级上均如此。在三个监督智能体中,网关无法区分采纳行为与诚实一致(假阳性率100%);仅读取文本记录的同谱系法官在文本任务中标记采纳行为(精度100%,召回率93%),但在影像任务中与网关表现一致;私下重新查询保留智能体的裁判在影像任务中表现提升(精度77%-88%,假阳性率13%-21%)。将线索的视觉显著性提升至三倍不会改变传染率,而增加一个对等智能体的声音会使传染率再提高一半。对隐藏规则的博弈几乎无声:仅1/10的文本任务和1/134的影像任务漂移智能体提及它们转向的规则。委员会博弈的关键在于社会直觉,且仅独立于自我报告的裁判能捕捉到这种博弈。代码:this https URL
英文摘要
Clinical decision support is moving toward committees of language-model agents deliberating on a shared workspace. We ask whether such committees can be gamed by shortcuts, cues a benchmark rewards but a clinician would ignore. Across seven cohorts on six public datasets spanning text (MedQA-USMLE, MedMCQA, MIMIC-CXR reports), imaging (NIH ChestX-ray14, MIMIC-CXR-JPG, CheXpert) and tabular ICU records (SUPPORT2), Gemini committees resist these cues in isolation (flip 5-16%), yet a socially plausible shortcut spreads: when two peers assert the same wrong answer, the holdout under test adopts it in 38% of cases, as does a false "pre-screen" system flag, on both capability tiers. Of three oversight agents, a gate cannot separate adoption from honest agreement (false-positive rate 100%); a same-lineage judge reading only the transcript flags adoption on text (precision 100%, recall 93%) but collapses onto the gate in imaging; a referee that privately re-queries the holdout transfers to imaging (77-88% precision, 13-21% false-positive rate). Tripling a cue's visual salience does not move contagion, whereas a second peer voice raises it by half again. Gaming a hidden rubric is near-silent: only 1/10 text and 1/134 imaging drifters name the rubric they moved toward. What games a committee is social plausibility, and only a referee independent of self-report catches it. Code: https://github.com/criticaldata/benchmaxxing