arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2501.17860cs.CLcs.AI

对话优于独白:通过战略对话指导医学LLM

Dialogue is Better Than Monologue: Instructing Medical LLMs via Strategical Conversations

UNC at Chapel Hill(北卡罗来纳大学教堂山分校);University of Science and Technology of China(中国科学技术大学);George Mason University(乔治·马歇尔大学);Yale University(耶鲁大学);City University of Hong Kong(香港城市大学);Rice University(得克萨斯大学奥斯汀分校)

Zijie Liu, Xinyu Zhao, Jie Peng, Zhuangdi Zhu, Qingyu Chen, Kaidi Xu, Xia Hu, Tianlong Chen

AI总结:

通过战略对话指导医学LLM,提升多轮推理和嘈杂环境下的准确率

摘要:

当前的医疗AI系统往往无法复制现实中的临床推理,因为它们主要是在静态文本和问答任务上进行训练和评估。这些调优方法和基准忽略了诸如基于证据的推理和处理干扰信息等关键方面。为弥补这一差距,我们引入了一个新的基准,模拟现实中的诊断场景,整合与USMLE标准对齐的噪声和难度水平。此外,我们探索了基于对话的微调,将静态数据集转换为对话格式,以更好地捕捉迭代推理过程。实验表明,对话微调的模型在多轮推理场景中比传统方法提高了9.64%,在嘈杂环境中准确率提高了6.18%。我们的发现强调对话微调作为推进临床对齐和稳健医疗AI系统的重要方法。

英文摘要:

Current medical AI systems often fail to replicate real-world clinical reasoning, as they are predominantly trained and evaluated on static text and question-answer tasks. These tuning methods and benchmarks overlook critical aspects like evidence-based reasoning and handling distracting information. To bridge this gap, we introduce a novel benchmark that simulates real-world diagnostic scenarios, integrating noise and difficulty levels aligned with USMLE standards. Moreover, we explore dialogue-based fine-tuning, which transforms static datasets into conversational formats to better capture iterative reasoning processes. Experiments show that dialogue-tuned models outperform traditional methods, with improvements of $9.64\%$ in multi-round reasoning scenarios and $6.18\%$ in accuracy in a noisy environment. Our findings highlight dialogue tuning as a promising approach for advancing clinically aligned and robust medical AI systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑