MAM：基于角色专业化协作的多模态医学诊断模块化多智能体框架
MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration
- University of Macau（澳门大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出多模态医学诊断模块化多智能体框架MAM，通过将诊断流程分解为全科医生、专家团队等五个基于LLM的角色化智能体，实现高效知识更新与协作，在多模态医疗数据集上性能显著超越基线模型18%至365%。
AI中文摘要：
医疗大语言模型（LLM）的最新进展展示了其强大的推理和诊断能力。尽管取得了成功，当前统一的多模态医疗LLM在知识更新成本、全面性和灵活性方面仍面临局限。为应对这些挑战，我们引入了多模态医学诊断模块化多智能体框架（MAM）。受我们关于角色分配和诊断辨别力在LLM中益处的实证发现启发，MAM将医疗诊断流程分解为专业化角色：全科医生、专家团队、放射科医生、医疗助理和主任，每个角色均由基于LLM的智能体承担。这种模块化且协作的框架支持高效的知识更新，并利用现有的医疗LLM和知识库。在涵盖文本、图像、音频和视频模态的广泛公开多模态医疗数据集上进行的广泛实验评估表明，MAM持续超越特定模态LLM的性能。值得注意的是，与基线模型相比，MAM实现了从18%到365%的显著性能提升。我们的代码已在 https://github.com/yczhou001/MAM 发布。
英文摘要：
Recent advancements in medical Large Language Models (LLMs) have showcased their powerful reasoning and diagnostic capabilities. Despite their success, current unified multimodal medical LLMs face limitations in knowledge update costs, comprehensiveness, and flexibility. To address these challenges, we introduce the Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis (MAM). Inspired by our empirical findings highlighting the benefits of role assignment and diagnostic discernment in LLMs, MAM decomposes the medical diagnostic process into specialized roles: a General Practitioner, Specialist Team, Radiologist, Medical Assistant, and Director, each embodied by an LLM-based agent. This modular and collaborative framework enables efficient knowledge updates and leverages existing medical LLMs and knowledge bases. Extensive experimental evaluations conducted on a wide range of publicly accessible multimodal medical datasets, incorporating text, image, audio, and video modalities, demonstrate that MAM consistently surpasses the performance of modality-specific LLMs. Notably, MAM achieves significant performance improvements ranging from 18% to 365% compared to baseline models. Our code is released at https://github.com/yczhou001/MAM.
