代理检索增强推理重塑放射学问答中的集体可靠性在模型变异性下
Agentic retrieval-augmented reasoning reshapes collective reliability under model variability in radiology question answering
AI总结：
代理检索增强推理在放射学问答中提升了集体可靠性，通过减少模型决策分散和增强跨模型正确性鲁棒性，但高一致性不保证正确性。
摘要：
代理检索增强推理管道越来越多地用于结构化地指导大型语言模型（LLMs）如何在临床决策支持中整合外部证据。这些系统通过迭代检索经过精心编纂的领域知识并将其合成结构化报告，然后进行答案选择。尽管这些管道可以提高性能，但它们在模型变异性下的影响仍不清楚。在真实世界部署中，异质模型可能会以无法通过准确性捕捉的方式对齐、分歧或同步错误。我们评估了34个LLMs在169个专家编纂的公开可用放射学问题上的表现，比较了零样本推理与一个放射学特定的多步代理检索条件，其中所有模型都接收了从编纂的放射学知识中衍生出的相同结构化证据报告。代理推理减少了模型间决策分散（中位熵0.48 vs. 0.13）并增加了模型间正确性的鲁棒性（均值0.74 vs. 0.81）。多数共识也增加了整体（P<0.001）。共识强度和鲁棒正确性在两种策略下仍保持相关（零样本rho=0.88；代理rho=0.87），尽管高一致性并不保证正确性。响应冗余度与正确性之间没有有意义的关联。在572个错误输出中，72%与中等或高临床评估严重程度相关，尽管评分者间一致性很低（kappa=0.02）。因此，代理检索与更集中的决策分布、更强的共识和更高的跨模型正确性鲁棒性相关。这些发现表明，仅通过准确性或一致性来评估代理系统可能并不总是足够，需要互补分析稳定性、跨模型鲁棒性和潜在的临床影响，以表征在模型变异性下的可靠性。
英文摘要：
Agentic retrieval-augmented reasoning pipelines are increasingly used to structure how large language models (LLMs) incorporate external evidence in clinical decision support. These systems iteratively retrieve curated domain knowledge and synthesize it into structured reports before answer selection. Although such pipelines can improve performance, their impact on reliability under model variability remains unclear. In real-world deployment, heterogeneous models may align, diverge, or synchronize errors in ways not captured by accuracy. We evaluated 34 LLMs on 169 expert-curated publicly available radiology questions, comparing zero-shot inference with a radiology-specific multi-step agentic retrieval condition in which all models received identical structured evidence reports derived from curated radiology knowledge. Agentic inference reduced inter-model decision dispersion (median entropy 0.48 vs. 0.13) and increased robustness of correctness across models (mean 0.74 vs. 0.81). Majority consensus also increased overall (P<0.001). Consensus strength and robust correctness remained correlated under both strategies (\r{ho}=0.88 for zero-shot; \r{ho}=0.87 for agentic), although high agreement did not guarantee correctness. Response verbosity showed no meaningful association with correctness. Among 572 incorrect outputs, 72% were associated with moderate or high clinically assessed severity, although inter-rater agreement was low (\k{appa}=0.02). Agentic retrieval therefore was associated with more concentrated decision distributions, stronger consensus, and higher cross-model robustness of correctness. These findings suggest that evaluating agentic systems through accuracy or agreement alone may not always be sufficient, and that complementary analyses of stability, cross-model robustness, and potential clinical impact are needed to characterize reliability under model variability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
