学习技能属性以实现视频中可迁移的评估
Learning Skill-Attributes for Transferable Assessment in Video
AI总结：
针对视频技能评估模型泛化性差、专家监督成本高的问题，提出CrossTrainer方法，通过发现跨运动通用的技能属性并训练多模态语言模型生成反馈和熟练度评分，在跨运动和运动内设置中实现60%性能提升。
摘要：
视频技能评估需要对人的身体表现质量进行评分，并解释可改进之处。当前模型针对单个运动项目专门设计，且因专家级监督在长尾运动项目中成本高、数据稀缺而受限。为缩小这一差距，我们探索可迁移的视频表示用于技能评估。我们的CrossTrainer方法发现平衡、控制和手部姿势等技能属性——其含义超越特定运动项目的边界，然后训练多模态语言模型为新视频生成可操作反馈（如“多抬手以产生更多力量”）及其熟练程度（如初级专家）。我们在跨运动（迁移）和运动内（域内）设置的多个数据集上验证新模型，其相对最先进技术实现了高达60%的提升。通过提取人类技能共享行为，所提视频表示的泛化性显著优于现有技术，丰富了多模态大语言模型。
英文摘要：
Skill assessment from video entails rating the quality of a person's physical performance and explaining what could be done better. Today's models specialize for an individual sport, and suffer from the high cost and scarcity of expert-level supervision across the long tail of sports. Towards closing that gap, we explore transferable video representations for skill assessment. Our CrossTrainer approach discovers skill-attributes, such as balance, control, and hand positioning -- whose meaning transcends the boundaries of any given sport, then trains a multimodal language model to generate actionable feedback for a novel video, e.g., "lift hands more to generate more power" as well as its proficiency level, e.g., early expert. We validate the new model on multiple datasets for both cross-sport (transfer) and intra-sport (in-domain) settings, where it achieves gains up to 60% relative to the state of the art. By abstracting out the shared behaviors indicative of human skill, the proposed video representation generalizes substantially better than an array of existing techniques, enriching today's multimodal large language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
