arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.19281cs.HCcs.ETcs.RO
收藏

面向反馈式混合现实与机器人建造的手势识别:以 UnLog Tower 为例

Gesture Recognition for Feedback Based Mixed Reality and Robotic Fabrication: A Case Study of the UnLog Tower

Cornell University(康奈尔大学)

Alexander Htet Kyaw, Lawson Spencer, Sasa Zivkovic, Leslie Lok

AI总结:

本研究以 UnLog Tower 为例,提出通过实时手势识别将用户与物理对象的交互作为输入,驱动 MR 环境中的数字信息修改与生成,实现物理与虚拟环境间的双向反馈,并应用于机器人建造、人工装配和质量控制。

摘要:

混合现实(Mixed Reality, MR)平台使用户能够在高度定制和参数化建筑构件的装配与建造过程中与三维全息指令进行交互,而无需二维图纸。以往的 MR 建造项目主要依赖数字菜单和自定义按钮作为用户与 MR 环境交互的界面。尽管这种方法被广泛采用,但它在允许人类直接与物理对象交互以在 MR 环境中修改建造指令方面存在局限。本研究通过实时手势识别将用户与物理对象的交互集成为输入,以修改、更新或生成新的数字信息,从而实现物理环境与虚拟环境之间的相互刺激。因此,数字环境能够根据用户提供的与物理对象的交互生成反馈,从而在建造过程中实现无缝反馈。本研究探讨了面向反馈式 MR 工作流的手势识别,用于 UnLog Tower 建造中的机器人建造、人工装配和质量控制。

英文摘要:

Mixed Reality (MR) platforms enable users to interact with three-dimensional holographic instructions during the assembly and fabrication of highly custom and parametric architectural constructions without the necessity of two-dimensional drawings. Previous MR fabrication projects have primarily relied on digital menus and custom buttons as the interface for user interaction with the MR environment. Despite this approach being widely adopted, it is limited in its ability to allow for direct human interaction with physical objects to modify fabrication instructions within the MR environment. This research integrates user interactions with physical objects through real-time gesture recognition as input to modify, update or generate new digital information enabling reciprocal stimuli between the physical and the virtual environment. Consequently, the digital environment is generative of the user's provided interaction with physical objects to allow seamless feedback in the fabrication process. This research investigates gesture recognition for feedback-based MR workflows for robotic fabrication, human assembly, and quality control in the construction of the UnLog Tower.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑