MMTA: 多成员时序注意机制用于细粒度笔画康复评估
MMTA: Multi Membership Temporal Attention for Fine-Grained Stroke Rehabilitation Assessment
AI总结:
MMTA通过多成员时序注意机制提升细粒度笔画康复评估的精度和鲁棒性,适用于视频和可穿戴设备输入。
摘要:
为了增强康复过程中迭代评估的能力,需要在日常活动中自动评估个人的能力,这要求在治疗视频中对细粒度动作进行时间精确的分割。现有的时序动作分割(TAS)模型在捕捉亚秒级微小动作的同时保留锻炼上下文方面存在困难,模糊了快速相变并限制了对运动恢复的可靠下游评估。我们引入了多成员时序注意机制(MMTA),一种高分辨率的时序Transformer用于细粒度康复评估。与标准时序注意机制不同,后者为每个帧分配每个层一个注意力上下文,而MMTA允许每个帧在同一个层内关注多个局部归一化的时序注意窗口。我们通过特征空间重叠解决这些并发时序视图,保留接近转换的竞争局部上下文,同时通过层间传播实现更长范围的推理。这在不增加深度或多阶段细化的情况下提高了边界敏感性。MMTA在统一的单阶段架构内支持视频和可穿戴IMU输入,使其适用于临床和家庭环境。MMTA在StrokeRehab上比全局注意Transformer提升Edit Score,视频上提升+1.3,IMU上提升+1.6,并进一步提升50Salads的+3.3。消融实验确认性能提升源于多成员时序视图而非架构复杂性,提供了一种适用于资源受限康复评估的实用解决方案。
英文摘要:
To empower the iterative assessments involved during a person's rehabilitation, automated assessment of a person's abilities during daily activities requires temporally precise segmentation of fine-grained actions in therapy videos. Existing temporal action segmentation (TAS) models struggle to capture sub-second micro-movements while retaining exercise context, blurring rapid phase transitions and limiting reliable downstream assessment of motor recovery. We introduce Multi-Membership Temporal Attention (MMTA), a high-resolution temporal transformer for fine-grained rehabilitation assessment. Unlike standard temporal attention, which assigns each frame a single attention context per layer, MMTA lets each frame attend to multiple locally normalized temporal attention windows within the same layer. We fuse these concurrent temporal views via feature-space overlap resolution, preserving competing local contexts near transitions while enabling longer-range reasoning through layer-wise propagation. This increases boundary sensitivity without additional depth or multi-stage refinement. MMTA supports both video and wearable IMU inputs within a unified single-stage architecture, making it applicable to both clinical and home settings. MMTA consistently improves over the Global Attention transformer, boosting Edit Score by +1.3 (Video) and +1.6 (IMU) on StrokeRehab while further improving 50Salads by +3.3. Ablations confirm that performance gains stem from multi-membership temporal views rather than architectural complexity, offering a practical solution for resource-constrained rehabilitation assessment.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。