arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.16892cs.HC
收藏

用于基于人工智能和混合现实的物理任务指导的交互设计工具包

An Interaction Design Toolkit for Physical Task Guidance with Artificial Intelligence and Mixed Reality

Arthur Caetano, Alejandro Aponte, Misha Sra

AI总结:

本文针对人工智能与混合现实物理任务指导系统中的交互设计挑战,提出包含设计考量、设计模式和交互画布的交互设计工具包,为从业者和研究人员提供宝贵资源。

摘要:

从运动技巧到外科手术的物理技能习得,需要指导和反馈。在缺乏人类专家的情况下,物理任务指导(PTG)系统可以提供一种有前景的替代方案。这些系统集成了人工智能(AI)和混合现实(MR),在用户使用物理工具和物体练习并学习技能时提供实时反馈和指导。然而,设计PTG系统带来的挑战超出了工程复杂性。用户、AI、MR界面和物理环境之间复杂的相互作用产生了独特的交互设计障碍。为了应对这些挑战,我们提出了一个交互设计工具包,该工具包源于我们对八支学生团队在一次为期10周的研究生课程中开发的PTG原型所做的分析。该工具包包含设计考量、设计模式和交互画布。我们的评估表明,该工具包可以作为设计PTG系统的从业者以及开发人机AI交互设计新工具的研究人员的宝贵资源。

英文摘要:

Physical skill acquisition, from sports techniques to surgical procedures, requires instruction and feedback. In the absence of a human expert, Physical Task Guidance (PTG) systems can offer a promising alternative. These systems integrate Artificial Intelligence (AI) and Mixed Reality (MR) to provide realtime feedback and guidance as users practice and learn skills using physical tools and objects. However, designing PTG systems presents challenges beyond engineering complexities. The intricate interplay between users, AI, MR interfaces, and the physical environment creates unique interaction design hurdles. To address these challenges, we present an interaction design toolkit derived from our analysis of PTG prototypes developed by eight student teams during a 10-week-long graduate course. The toolkit comprises Design Considerations, Design Patterns, and an Interaction Canvas. Our evaluation suggests that the toolkit can serve as a valuable resource for practitioners designing PTG systems and researchers developing new tools for human-AI interaction design.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑