Polaris：一种面向医疗保健的注重安全的 LLM 星座架构
Polaris: A Safety-focused LLM Constellation Architecture for Healthcare
AI总结：
开发了首个面向实时医疗对话的安全LLM星座系统Polaris，包含主智能体和专家支持智能体，经大规模临床医生评估表现与人类护士相当，且专家智能体显著优于GPT-4和LLaMA-2 70B。
摘要：
我们开发了 Polaris，这是首个专注于安全的 LLM 星座系统，用于实时的患者与 AI 医疗对话。与以往医疗保健领域专注于问答等任务的 LLM 研究不同，我们的工作特别关注多轮长语音对话。我们的一万亿参数星座系统由多个数十亿参数的 LLM 作为协作智能体组成：一个有状态的主智能体专注于推动引人入胜的对话，以及多个专注于护士执行的医疗保健任务的专家支持智能体，以提高安全性并减少幻觉。我们开发了一种复杂的训练协议，用于智能体的迭代协同训练，以优化多种目标。我们在专有数据、临床护理计划、医疗监管文件、医疗手册和其他医疗推理文档上训练模型。我们使用真实的医疗保健对话以及患者演员与经验丰富的护士之间的模拟对话，将模型对齐为像医疗专业人员那样说话。这使得我们的系统能够表现出独特的能力，如建立融洽关系、建立信任、同理心和临床态度。最后，我们提出了对医疗保健 LLM 系统的首次全面临床医生评估。我们招募了超过 1100 名美国注册护士和超过 130 名美国注册医生，通过扮演患者并在多个维度上对系统进行评分，对我们系统进行端到端的对话评估。我们证明 Polaris 在医疗安全、临床准备度、对话质量和临床态度等维度上的总体表现与人类护士相当。此外，我们对各个专家支持智能体进行了具有挑战性的基于任务的评估，其中我们证明我们的 LLM 智能体显著优于大得多的通用 LLM（GPT-4）以及其自身中等规模类别（LLaMA-2 70B）。
英文摘要：
We develop Polaris, the first safety-focused LLM constellation for real-time patient-AI healthcare conversations. Unlike prior LLM works in healthcare focusing on tasks like question answering, our work specifically focuses on long multi-turn voice conversations. Our one-trillion parameter constellation system is composed of several multibillion parameter LLMs as co-operative agents: a stateful primary agent that focuses on driving an engaging conversation and several specialist support agents focused on healthcare tasks performed by nurses to increase safety and reduce hallucinations. We develop a sophisticated training protocol for iterative co-training of the agents that optimize for diverse objectives. We train our models on proprietary data, clinical care plans, healthcare regulatory documents, medical manuals, and other medical reasoning documents. We align our models to speak like medical professionals, using organic healthcare conversations and simulated ones between patient actors and experienced nurses. This allows our system to express unique capabilities such as rapport building, trust building, empathy and bedside manner. Finally, we present the first comprehensive clinician evaluation of an LLM system for healthcare. We recruited over 1100 U.S. licensed nurses and over 130 U.S. licensed physicians to perform end-to-end conversational evaluations of our system by posing as patients and rating the system on several measures. We demonstrate Polaris performs on par with human nurses on aggregate across dimensions such as medical safety, clinical readiness, conversational quality, and bedside manner. Additionally, we conduct a challenging task-based evaluation of the individual specialist support agents, where we demonstrate our LLM agents significantly outperform a much larger general-purpose LLM (GPT-4) as well as from its own medium-size class (LLaMA-2 70B).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
