OpenWatch: 一种用于智能手表手部动作识别的多模态基准
OpenWatch: A Multimodal Benchmark for Hand Gesture Recognition on Smartwatches
AI总结：
本文提出OpenWatch多模态基准，通过同步惯性与生理传感数据，评估手部动作识别方法，展示了MixToken和NormWear-Lora等新方法的优越性。
摘要：
尽管智能手表在全球普及，但基于手腕的手部动作识别开放基准仍有限。本文介绍了首个开源多模态基准OpenWatch，包含50名参与者超过10小时的IMU和PPG数据，以及59种标记的手部动作序列。我们提出了一个独立于主体的评估协议，包括传统和深度学习方法用于时间序列分类。此外，我们开发了两种新的手部动作识别方法：(i) MixToken，一种任务特定的混合专家模型，融合单通道IMU滤波器特征与跨通道统计令牌；(ii) NormWear-Lora，一种用于智能手表基础模型的低秩适应模块。基准测试结果表明，PPG信号对基础智能手表模型有显著的预测优势（+12.5% F1分数）。此外，我们显示任务特定架构（即MixToken）在准确率（F1分数=90% vs 66%）和内存效率（223k vs 136M参数）方面明显优于微调的智能手表基础模型。最后，我们还提供了关于在资源受限的可穿戴传感中专门架构设计、模态融合、数据增强和基础模型适应之间权衡的清晰实证指导。
英文摘要：
Despite widespread adoption of smartwatches worldwide, open-benchmarks for wrist-based gesture recognition remain surprisingly limited. In this work, we introduce the first open-access multi-modal benchmark, OpenWatch, for wrist-based gesture recognition using synchronized inertial and physiological sensing on a commercial smartwatch. It contains over 10 hours of Inertial Measurement Unit (IMU) and Photoplethysmography (PPG) data across 50 participants and a vocabulary of 59 labelled gesture sequences. Furthermore, we present a subject-independent evaluation protocol including traditional and deep learning methods for time-series classification. On top of this, we develop two novel methodologies for hand-gesture recognition: (i) MixToken, a task-specific mixture-of-experts that fuses per-channel IMU filterbank features with cross-channel statistical tokens through learned logit mixing, and (ii) NormWear-Lora, a low-rank adaptation module for smartwatch foundation models. Our benchmarking results reveal that PPG signals carries a substantial predictive benefit (+12.5% F1-score) for foundational smartwatch models. In addition, we show that task-specific architectures (i.e. MixToken) substantially outperforms finetuned smartwatch foundation models in terms of accuracy (F1-score=90% vs 66%) and memory efficiency (223k vs 136M parameters). Finally, we also provide clear empirical guidance on the trade-offs between specialized architecture design, modality fusion, data augmentations, and foundation-model adaptation for resource-constrained wearable sensing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
