arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.22077cs.HC
收藏

ViSTAR: 基于增强现实的虚拟技能训练系统,结合3D虚拟角色和LLM教练代理

ViSTAR: Virtual Skill Training with Augmented Reality with 3D Avatars and LLM coaching agent

Harvard University(哈佛大学);University of Tsukuba(筑波大学);Dolby Laboratories(杜比实验室);University of Minnesota-Twin Cities(明尼苏达大学双城分校)

Chunggi Lee, Hayato Saiki, Tica Lin, Eiji Ikeda, Kenji Suzuki, Chen Zhu-Tian, Hanspeter Pfister

AI总结:

ViSTAR通过AR技术提供篮球技能训练,利用3D虚拟角色和LLM教练代理提供平衡、姿势和节奏反馈,提升自我观察和运动修正能力。

摘要:

我们提出了ViSTAR,一种基于增强现实的虚拟技能训练系统,支持自主篮球技能练习,并提供平衡、姿势和节奏的反馈。通过与篮球运动员和教练的形成性研究,系统解决了三个挑战:理解技能、识别错误和纠正错误。ViSTAR遵循行为技能训练(BST)框架——指导、建模、练习和反馈。它通过视觉叠加、节奏和节奏提示以及使用3D运动重建的AI教练代理提供反馈。我们通过分析时空关节数据并利用大型语言模型(LLM)将特征映射到自然语言指导提示来生成口头反馈。关键创新是这种反馈生成:运动特征转化为简洁的指导见解。在两项研究(N=16)中,参与者普遍更喜欢我们的AI生成反馈而不是教练反馈,并报告说ViSTAR帮助他们注意到姿势和平衡问题并改进了动作。

英文摘要:

We present ViSTAR, a Virtual Skill Training system in AR that supports self-guided basketball skill practice, with feedback on balance, posture, and timing. From a formative study with basketball players and coaches, the system addresses three challenges: understanding skills, identifying errors, and correcting mistakes. ViSTAR follows the Behavioral Skills Training (BST) framework-instruction, modeling, rehearsal, and feedback. It provides feedback through visual overlays, rhythm and timing cues, and an AI-powered coaching agent using 3D motion reconstruction. We generate verbal feedback by analyzing spatio-temporal joint data and mapping features to natural-language coaching cues via a Large Language Model (LLM). A key novelty is this feedback generation: motion features become concise coaching insights. In two studies (N=16), participants generally preferred our AI-generated feedback to coach feedback and reported that ViSTAR helped them notice posture and balance issues and refine movements beyond self-observation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑