arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2308.16552cs.CV
收藏

提示增强的分层Transformer通过时间动作分割提升心肺复苏指导

Prompt-enhanced Hierarchical Transformer Elevating Cardiopulmonary Resuscitation Instruction via Temporal Action Segmentation

Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);Tsinghua-Berkeley Shenzhen Institute(清华-伯克利深圳学院);Miragestars Inc.(Mirage星公司);Shenzhen ZNV Technology Co., Ltd(深圳中兴力维技术有限公司);Shandong University(山东大学)

Yang Liu, Xiaoyun Zhong, Shiyao Zhai, Zhicheng Du, Zhenyuan Gao, Qiming Huang, Canyang Zhang, Bin Jiang, Vijay Kumar Pandey, Sanyang Han, Runming Wang, Yuxing Han, Peiwu Qin

AI总结:

提出PhiTrans模型,将CPR视频指导视为时间动作分割任务,通过提示增强的分层Transformer提升复苏操作合格率,实验指标超91%。

摘要:

绝大多数遭遇突发心脏骤停的人,是由旁观者试图通过心肺复苏(CPR)来挽救生命,但这些努力往往因操作不合格而徒劳无功。幸运的是,多项研究表明,规范训练有助于提高复苏成功率,这不断需要新技术无缝结合以取得进一步进展。为此,我们收集了一个定制的CPR视频数据集,其中受训者按照批准的指南独立在人体模型上进行复苏操作,从而设计了一个辅助工具箱,通过现代深度学习方法协助监督和纠正过程中的潜在问题。我们的研究经验性地将此问题视为计算机视觉中的时间动作分割(TAS)任务,旨在帧级别对未修剪视频进行分割。在此,我们提出了一种提示增强的分层Transformer(PhiTrans),它整合了三个不可或缺的模块,包括基于文本提示的视频特征提取器(VFE)、基于Transformer的动作分割执行器(ASE)和基于回归的预测细化校准器(PRC)。模型的骨干网络优先来源于三个已批准用于TAS任务的公共数据集(GTEA、50Salads和Breakfast)的应用,这有助于在CPR数据集上挖掘分割流程。总的来说,我们前所未有地探索了一条可行的流程,通过动作分割结合前沿深度学习技术,真正提升了CPR指导的合格性。相关实验支持我们的实现,多项指标超过91.0%。

英文摘要:

The vast majority of people who suffer unexpected cardiac arrest are performed cardiopulmonary resuscitation (CPR) by passersby in a desperate attempt to restore life, but endeavors turn out to be fruitless on account of disqualification. Fortunately, many pieces of research manifest that disciplined training will help to elevate the success rate of resuscitation, which constantly desires a seamless combination of novel techniques to yield further advancement. To this end, we collect a custom CPR video dataset in which trainees make efforts to behave resuscitation on mannequins independently in adherence to approved guidelines, thereby devising an auxiliary toolbox to assist supervision and rectification of intermediate potential issues via modern deep learning methodologies. Our research empirically views this problem as a temporal action segmentation (TAS) task in computer vision, which aims to segment an untrimmed video at a frame-wise level. Here, we propose a Prompt-enhanced hierarchical Transformer (PhiTrans) that integrates three indispensable modules, including a textual prompt-based Video Features Extractor (VFE), a transformer-based Action Segmentation Executor (ASE), and a regression-based Prediction Refinement Calibrator (PRC). The backbone of the model preferentially derives from applications in three approved public datasets (GTEA, 50Salads, and Breakfast) collected for TAS tasks, which accounts for the excavation of the segmentation pipeline on the CPR dataset. In general, we unprecedentedly probe into a feasible pipeline that genuinely elevates the CPR instruction qualification via action segmentation in conjunction with cutting-edge deep learning techniques. Associated experiments advocate our implementation with multiple metrics surpassing 91.0%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑