arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.11896cs.CV
收藏

数字FAST:一种基于AI的多模态框架,用于快速和早期中风筛查

Digital FAST: An AI-Driven Multimodal Framework for Rapid and Early Stroke Screening

Ngoc-Khai Hoang, Thi-Nhu-Mai Nguyen, Huy-Hieu Pham

AI总结:

本文提出一种快速非侵入式多模态深度学习框架,用于基于F.A.S.T.评估数据的自动二分类中风筛查,通过融合面部表情、语音信号和上半身运动信息提升诊断鲁棒性,实验表明模型在准确率和F1分数上均优于单模态基线。

摘要:

早期识别中风症状对于实现及时干预和改善患者预后至关重要,尤其是在院前设置中。本研究提出了一种快速、非侵入式的多模态深度学习框架,用于基于F.A.S.T.评估数据的自动二分类中风筛查。所提出的方法整合了面部表情、语音信号和上半身运动的互补信息以增强诊断鲁棒性。面部动态通过基于关键点的特征进行表示,并使用Transformer架构建模以捕捉时间依赖性。语音信号转换为梅尔频谱图并通过音频频谱Transformer进行处理,而上半身姿态序列则通过MLP-Mixer网络分析以建模时空运动模式。提取的模态特定表示通过基于注意力的融合机制结合,以有效学习跨模态交互。在自采集的包含222个视频的37名受试者数据集上进行的实验表明,所提出的多模态模型在准确率和F1分数上均优于单模态基线,达到95.83%的准确率和96.00%的F1分数。该模型在灵敏度和特异性之间取得了良好的平衡,并成功检测了测试集中的所有中风病例。这些结果突显了多模态学习和迁移学习在早期中风筛查中的潜力,同时强调了需要更大的、具有临床代表性的数据集以支持可靠的现实世界部署。

英文摘要:

Early identification of stroke symptoms is essential for enabling timely intervention and improving patient outcomes, particularly in prehospital settings. This study presents a fast, non-invasive multimodal deep learning framework for automatic binary stroke screening based on data collected during the F.A.S.T. assessment. The proposed approach integrates complementary information from facial expressions, speech signals, and upper-body movements to enhance diagnostic robustness. Facial dynamics are represented using landmark based features and modeled with a Transformer architecture to capture temporal dependencies. Speech signals are converted into mel spectrograms and processed using an Audio Spectrogram Transformer, while upper-body pose sequences are analyzed with an MLP-Mixer network to model spatiotemporal motion patterns. The extracted modality specific representations are combined through an attention-based fusion mechanism to effectively learn cross modal interactions. Experiments conducted on a self-collected dataset of 222 videos from 37 subjects demonstrate that the proposed multimodal model consistently outperforms unimodal baselines, achieving 95.83% accuracy and a 96.00% F1-score. The model attains a strong balance between sensitivity and specificity and successfully detects all stroke cases in the test set. These results highlight the potential of multimodal learning and transfer learning for early stroke screening, while emphasizing the need for larger, clinically representative datasets to support reliable real-world deployment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑