ViSTAR: 基于增强现实的虚拟技能训练系统,结合3D虚拟角色和LLM教练代理
ViSTAR: Virtual Skill Training with Augmented Reality with 3D Avatars and LLM coaching agent
AI总结:
ViSTAR通过AR技术提供篮球技能训练,利用3D虚拟角色和LLM教练代理提供平衡、姿势和节奏反馈,提升自我观察和运动修正能力。
摘要:
我们提出了ViSTAR,一种基于增强现实的虚拟技能训练系统,支持自主篮球技能练习,并提供平衡、姿势和节奏的反馈。通过与篮球运动员和教练的形成性研究,系统解决了三个挑战:理解技能、识别错误和纠正错误。ViSTAR遵循行为技能训练(BST)框架——指导、建模、练习和反馈。它通过视觉叠加、节奏和节奏提示以及使用3D运动重建的AI教练代理提供反馈。我们通过分析时空关节数据并利用大型语言模型(LLM)将特征映射到自然语言指导提示来生成口头反馈。关键创新是这种反馈生成:运动特征转化为简洁的指导见解。在两项研究(N=16)中,参与者普遍更喜欢我们的AI生成反馈而不是教练反馈,并报告说ViSTAR帮助他们注意到姿势和平衡问题并改进了动作。
英文摘要:
We present ViSTAR, a Virtual Skill Training system in AR that supports self-guided basketball skill practice, with feedback on balance, posture, and timing. From a formative study with basketball players and coaches, the system addresses three challenges: understanding skills, identifying errors, and correcting mistakes. ViSTAR follows the Behavioral Skills Training (BST) framework-instruction, modeling, rehearsal, and feedback. It provides feedback through visual overlays, rhythm and timing cues, and an AI-powered coaching agent using 3D motion reconstruction. We generate verbal feedback by analyzing spatio-temporal joint data and mapping features to natural-language coaching cues via a Large Language Model (LLM). A key novelty is this feedback generation: motion features become concise coaching insights. In two studies (N=16), participants generally preferred our AI-generated feedback to coach feedback and reported that ViSTAR helped them notice posture and balance issues and refine movements beyond self-observation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。