《治愈性对话：医生智能体问诊能力的综合评估》
The Dialogue That Heals: A Comprehensive Evaluation of Doctor Agents' Inquiry Capability
AI总结：
该研究提出了迄今规模最大的医疗多轮问诊评估基准MAQuE及多维度评估框架，实验发现当前大语言模型的医生智能体问诊能力仍有明显短板，且对患者行为差异敏感，存在性能与实用性的平衡难题。
摘要：
一名合格的医师在接诊患者时，应当兼具共情能力、专业知识、耐心与清晰的沟通技巧。近期的研究进展已成功赋予AI医生专业的诊断技能，尤其是通过问诊主动获取信息的能力，但优秀医生的其他核心素养仍被忽视。为填补这一空白，我们提出MAQuE（医疗智能体问诊评估）——迄今规模最大的医疗多轮问诊自动化综合评估基准，包含3000个模拟真实情况的患者智能体，这些智能体展现出多样的语言模式、认知局限、情绪反应以及被动告知信息的倾向。我们还提出了多维度评估框架，涵盖任务成功率、问诊专业度、对话能力、问诊效率与患者体验五个维度。针对不同大语言模型（LLM）的实验显示，模型在各评估维度上均存在明显短板，即便最先进的模型，其问诊能力仍有很大提升空间。这些模型对真实患者的行为差异高度敏感，而这会显著影响诊断准确率。此外，我们提出的细粒度指标揭示了不同评估维度间的权衡关系，凸显了在真实临床场景中平衡性能与实用性的挑战。
英文摘要：
An effective physician should possess a combination of empathy, expertise, patience, and clear communication when treating a patient. Recent advances have successfully endowed AI doctors with expert diagnostic skills, particularly the ability to actively seek information through inquiry. However, other essential qualities of a good doctor remain overlooked. To bridge this gap, we present MAQuE(Medical Agent Questioning Evaluation), the largest-ever benchmark for the automatic and comprehensive evaluation of medical multi-turn questioning. It features 3,000 realistically simulated patient agents that exhibit diverse linguistic patterns, cognitive limitations, emotional responses, and tendencies for passive disclosure. We also introduce a multi-faceted evaluation framework, covering task success, inquiry proficiency, dialogue competence, inquiry efficiency, and patient experience. Experiments on different LLMs reveal substantial challenges across the evaluation aspects. Even state-of-the-art models show significant room for improvement in their inquiry capabilities. These models are highly sensitive to variations in realistic patient behavior, which considerably impacts diagnostic accuracy. Furthermore, our fine-grained metrics expose trade-offs between different evaluation perspectives, highlighting the challenge of balancing performance and practicality in real-world clinical settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
