arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.00878cs.CV
收藏

MMTA: 多成员时序注意机制用于细粒度笔画康复评估

MMTA: Multi Membership Temporal Attention for Fine-Grained Stroke Rehabilitation Assessment

Department of Electrical and Computer Engineering, University of Kentucky, Lexington, KY, USA(电气与计算机工程系,肯塔基大学,路易斯维尔,KY,美国);College of Medicine, University of Kentucky, Lexington, KY, USA(医学学院,肯塔基大学,路易斯维尔,KY,美国);Department of Electrical and Computer Engineering, University of California, Davis, CA, USA(电气与计算机工程系,加州大学戴维斯分校,CA,美国)

Halil Ismail Helvaci, Justin Huber, Jihye Bae, Sen-ching Samson Cheung

AI总结:

MMTA通过多成员时序注意机制提升细粒度笔画康复评估的精度和鲁棒性,适用于视频和可穿戴设备输入。

摘要:

为了增强康复过程中迭代评估的能力,需要在日常活动中自动评估个人的能力,这要求在治疗视频中对细粒度动作进行时间精确的分割。现有的时序动作分割(TAS)模型在捕捉亚秒级微小动作的同时保留锻炼上下文方面存在困难,模糊了快速相变并限制了对运动恢复的可靠下游评估。我们引入了多成员时序注意机制(MMTA),一种高分辨率的时序Transformer用于细粒度康复评估。与标准时序注意机制不同,后者为每个帧分配每个层一个注意力上下文,而MMTA允许每个帧在同一个层内关注多个局部归一化的时序注意窗口。我们通过特征空间重叠解决这些并发时序视图,保留接近转换的竞争局部上下文,同时通过层间传播实现更长范围的推理。这在不增加深度或多阶段细化的情况下提高了边界敏感性。MMTA在统一的单阶段架构内支持视频和可穿戴IMU输入,使其适用于临床和家庭环境。MMTA在StrokeRehab上比全局注意Transformer提升Edit Score,视频上提升+1.3,IMU上提升+1.6,并进一步提升50Salads的+3.3。消融实验确认性能提升源于多成员时序视图而非架构复杂性,提供了一种适用于资源受限康复评估的实用解决方案。

英文摘要:

To empower the iterative assessments involved during a person's rehabilitation, automated assessment of a person's abilities during daily activities requires temporally precise segmentation of fine-grained actions in therapy videos. Existing temporal action segmentation (TAS) models struggle to capture sub-second micro-movements while retaining exercise context, blurring rapid phase transitions and limiting reliable downstream assessment of motor recovery. We introduce Multi-Membership Temporal Attention (MMTA), a high-resolution temporal transformer for fine-grained rehabilitation assessment. Unlike standard temporal attention, which assigns each frame a single attention context per layer, MMTA lets each frame attend to multiple locally normalized temporal attention windows within the same layer. We fuse these concurrent temporal views via feature-space overlap resolution, preserving competing local contexts near transitions while enabling longer-range reasoning through layer-wise propagation. This increases boundary sensitivity without additional depth or multi-stage refinement. MMTA supports both video and wearable IMU inputs within a unified single-stage architecture, making it applicable to both clinical and home settings. MMTA consistently improves over the Global Attention transformer, boosting Edit Score by +1.3 (Video) and +1.6 (IMU) on StrokeRehab while further improving 50Salads by +3.3. Ablations confirm that performance gains stem from multi-membership temporal views rather than architectural complexity, offering a practical solution for resource-constrained rehabilitation assessment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑