用于多视图超声心动图解读的视频 CLIP 模型
Video CLIP Model for Multi-View Echocardiography Interpretation
AI总结：
本文提出一个基于五个标准视图完整视频序列训练的超声心动图视频-语言模型，利用多视图和视频输入提升诊断检索性能，并在 60,747 个视频-报告对上验证了其效果。
摘要：
超声心动图记录心脏的超声视频，使临床医生能够评估心脏功能。大规模视觉-语言模型（VLMs）的最新进展激发了人们对自动化超声心动图解读的兴趣。然而，大多数现有医学 VLMs 依赖单帧（图像）输入，这可能降低对仅能通过心脏运动识别的疾病的诊断准确性。此外，超声心动图视频从多个视图采集，每个视图在检测特定疾病方面的适用性各不相同。因此，利用多个视图可能提高诊断性能。我们开发了一个视频-语言模型，该模型处理来自五个标准视图的完整视频序列，并在 60,747 个超声心动图视频-报告对上进行了训练。我们评估了视频输入和多视图支持带来的检索性能提升，包括各种预训练模型的贡献。代码和模型权重可在 https://github.com/UTcardiology/video-echo-clip 获取。
英文摘要：
Echocardiography records ultrasound videos of the heart, enabling clinicians to assess cardiac function. Recent advances in large-scale vision-language models (VLMs) have spurred interest in automating echocardiographic interpretation. However, most existing medical VLMs rely on single-frame (image) inputs, which can reduce diagnostic accuracy for conditions identifiable only through cardiac motion. In addition, echocardiographic videos are captured from multiple views, each varying in suitability for detecting specific conditions. Leveraging multiple views may therefore improve diagnostic performance. We developed a video-language model that processes full video sequences from five standard views, trained on 60,747 echocardiographic video-report pairs. We evaluated the gains in retrieval performance from video input and multi-view support, including the contributions of various pretrained models. Code and model weights are available at https://github.com/UTcardiology/video-echo-clip
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
