发表机构
University of Georgia; New Jersey Institute of Technology（佐治亚大学; 新泽西理工学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对现有医疗AI智能体无法内化临床经验的问题，提出MSM-Mem框架，整合多类型临床经验并逐步更新，经MoE-LLaVA评估可提升性能，助力医疗AI智能体随实践进化推理能力。
AI 中文摘要
临床决策本质上是经验驱动的：医生通过整合患者病史、多模态观察结果以及跨交互的先前诊断经验，逐步完善其推理过程。相比之下，当前基于多模态大语言模型（MLLM）的医疗AI智能体大多作为无状态推理系统运行，每次交互独立生成决策，无法保留或内化经验知识。这种差异限制了它们在使用过程中逐步提高推理可靠性，以及适应现实临床工作流程中纵向患者情境的能力。本研究提出医疗结构化多模态记忆（MSM-Mem），这是一种使医疗AI智能体能够通过积累临床经验不断进化的智能体记忆框架。它将异构临床经验组织为语义记忆、情景记忆和视觉记忆，并在推理过程中逐步更新，使智能体能够检索先前经验为当前推理提供信息，并随时间逐步完善决策。对MoE-LLaVA主干的评估显示，其性能持续提升，且随着持续使用还会获得进一步增益。总体而言，MSM-Mem为医疗AI智能体提供了一条可行路径，使其能够像临床医生随时间从实践中学习那样，不断进化自身的推理能力。
英文摘要
Clinical decision-making is inherently experience-driven: physicians progressively refine their reasoning by synthesizing patient history, multimodal observations, and prior diagnostic experiences across interactions. In contrast, current multimodal large language model (MLLM)-based medical AI agents largely operate as stateless inference systems, generating decisions independently for each interaction without retaining or internalizing experiential knowledge. This discrepancy limits their ability to progressively improve reasoning reliability through usage and adapt to longitudinal patient contexts in real-world clinical workflows. In this study, we propose Medical Structured Multimodal Memory (MSM-Mem), an agentic memory framework that enables medical AI agents to evolve through accumulated clinical experiences. MSM-Mem organizes heterogeneous clinical experiences into semantic, episodic, and visual memory and incrementally updates them during inference, allowing the agent to retrieve prior experiences to inform current reasoning and progressively refine decision-making over time. Evaluations on MoE-LLaVA backbones demonstrate consistent performance improve- ments with further gains observed through continued usage. In general, MSM-Mem offers a viable pathway toward medical AI agents capable of evolving their reasoning competence in a manner analogous to the way clinicians learn from practice over time.
Comments8 pages
Journal refGenAI4Health @ NeurIPS 2026
