发表机构
Google Research; Google DeepMind（谷歌研究院; 谷歌DeepMind）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究开发了基于Gemini的多智能体系统AMIE（视频版），在随机OSCE研究中其临床问诊表现与初级保健医生相当或更优，为专家级实时视频问诊医疗AI的发展奠定了重要基础。
AI 中文摘要
视听互动是医患问诊的标准模式，可通过非语言线索实现自然沟通与疾病有效评估。尽管文本型AI已展现潜力，但它会丢失关键感知维度，且限制了无法用文字清晰表述症状的患者。早期将医疗AI扩展至视听互动的尝试已证明可行性，但未达到临床医生水平。本研究首次在实时临床视频问诊中展示了专家级AI，采用视频配置的AMIE（Articulate Medical Intelligence Explorer，可译为“精准医疗智能探索器”）。AMIE（视频版）是基于Gemini的多智能体系统，整合了低延迟对话、临床推理及实时视听感知。为指导开发，我们建立了远程医疗场景下临床视听线索的分类体系与自动评估方法。在一项随机客观结构化临床考试（OSCE）研究中，纳入30名初级保健医生（PCP）、15名患者演员及100种临床场景，我们对比了AMIE（视频版）、仅文本的AMIE（文本版）及PCP的视频问诊表现。临床评估者在病史采集、诊断、处理方案及体格观察与检查方面，对AMIE（视频版）的评分与PCP相当或更优。患者演员更偏好AMIE评估与解释病情的方式，而PCP在医患关系建立与伙伴关系构建上更受青睐。模态消融实验显示，患者演员在沟通有效性、便利性及被理解的感受上，更偏好AMIE（视频版）的界面而非文本聊天。其局限性在于精细解剖精度、细微情感差异及高频动作处理。尽管实际应用前仍需进一步研究，但这些结果标志着AI系统在应对临床实践中复杂感官维度辅助医疗方面的重要里程碑。
英文摘要
Audio-visual interaction is the standard for patient-physician consultations, enabling natural communication and effective assessment of illness through non-verbal cues. While text-based AI has shown promise, it discards essential perceptual dimensions and limits patients who cannot articulate symptoms in writing. Early efforts to extend medical AI to audio-visual interaction have demonstrated feasibility but not reached clinician-level performance. Here, we provide the first demonstration of expert-level AI in real-time clinical video consultations using AMIE (Articulate Medical Intelligence Explorer) in a video configuration. AMIE (Video) is a Gemini-based multi-agent system integrating low-latency dialogue, clinical reasoning, and real-time audio-visual perception. To guide development, we established a taxonomy and automated evaluations for clinical audio-visual cues in telehealth settings. In a randomized Objective Structured Clinical Examination (OSCE) study with 30 primary care physicians (PCPs), 15 patient actors and 100 clinical scenarios, we compared AMIE (Video), its text-only counterpart AMIE (Text), and PCPs consulting via video. Clinical evaluators rated AMIE (Video) on par or better than PCPs in history-taking, diagnosis, management, and physical observation and examination. Patient actors preferred AMIE's approach to assessing and explaining conditions, while PCPs were preferred for rapport and partnership building. In modality ablation, patient actors preferred AMIE (Video)'s interface over text chat for communicative effectiveness, convenience, and feeling understood. Limitations remain in fine anatomical precision, subtle affective nuances, and high-frequency movements. While further research is needed before real-world translation, these results mark an important milestone toward AI systems capable of augmenting care across the sensory complexity of clinical practice.
