教多模态大语言模型理解心电图图像
Teach Multimodal LLMs to Comprehend Electrocardiographic Images
AI总结：
针对ECG图像理解缺乏指令数据与评测基准的问题，本文构建百万级ECGInstruct和跨九数据集的ECGBench，并训练专用MLLM PULSE，使其准确率较通用模型提升15%至30%。
摘要：
心电图（ECG）是评估心脏疾病的重要非侵入性诊断工具。现有自动解读方法泛化能力有限，只关注范围狭窄的心脏疾病，并且通常依赖原始生理信号；在资源有限、仅能获取打印或数字ECG图像的环境中，这些信号可能并不容易获得。多模态大语言模型（MLLM）的最新进展为应对这些挑战提供了有前景的机会。然而，由于缺乏指令微调数据集以及用于定量评估的成熟ECG图像基准，将MLLM应用于ECG图像解读仍具挑战性。为应对这些挑战，我们提出ECGInstruct，这是一个包含超过100万个样本的综合性ECG图像指令微调数据集，覆盖来自多样化数据源的广泛ECG相关任务。利用ECGInstruct，我们开发了PULSE，这是一种专为ECG图像理解定制的MLLM。此外，我们构建了ECGBench，这是一个新的评估基准，涵盖跨9个不同数据集的四项关键ECG图像解读任务。实验表明，PULSE创下了新的最优水平，相比通用MLLM平均准确率提升15%至30%。本工作凸显了PULSE在临床实践中增强ECG解读的潜力。
英文摘要：
The electrocardiogram (ECG) is an essential non-invasive diagnostic tool for assessing cardiac conditions. Existing automatic interpretation methods suffer from limited generalizability, focusing on a narrow range of cardiac conditions, and typically depend on raw physiological signals, which may not be readily available in resource-limited settings where only printed or digital ECG images are accessible. Recent advancements in multimodal large language models (MLLMs) present promising opportunities for addressing these challenges. However, the application of MLLMs to ECG image interpretation remains challenging due to the lack of instruction tuning datasets and well-established ECG image benchmarks for quantitative evaluation. To address these challenges, we introduce ECGInstruct, a comprehensive ECG image instruction tuning dataset of over one million samples, covering a wide range of ECG-related tasks from diverse data sources. Using ECGInstruct, we develop PULSE, an MLLM tailored for ECG image comprehension. In addition, we curate ECGBench, a new evaluation benchmark covering four key ECG image interpretation tasks across nine different datasets. Our experiments show that PULSE sets a new state-of-the-art, outperforming general MLLMs with an average accuracy improvement of 15% to 30%. This work highlights the potential of PULSE to enhance ECG interpretation in clinical practice.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
