arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.11535cs.HCcs.AI
收藏

增强装配:面向增强现实自适应装配指令的对象识别与手部追踪

Augmented Assembly: Object Recognition and Hand Tracking for Adaptive Assembly Instructions in Augmented Reality

Massachusetts Institute of Technology(麻省理工学院);Cornell University(康奈尔大学)

Alexander Htet Kyaw, Haotian Ma, Sasa Zivkovic, Jenny Sabin

AI总结:

本文提出一种基于增强现实的自适应装配系统,通过对象识别和手部追踪实现动态指导与错误检测,提升装配效率与用户体验。

摘要:

近期增强现实(AR)的进步使能够协助用户进行物理装配任务的交互系统成为可能。本文提出了一种AR辅助的装配流程,利用对象识别和手部追踪来(1)识别定制组件,(2)显示分步骤的指导说明,(3)检测装配偏差,(4)根据用户与物理部件的互动动态更新指导说明。通过对象识别,系统能够实时检测并定位组件,从而创建工作区的数字双胞胎。对于每个装配步骤,系统在AR中叠加边界框以指示相关组件的当前位置和目标放置位置,同时手部追踪数据验证用户是否与正确的部件互动。与强制固定顺序不同,系统强调潜在的装配错误,并将用户的偏差视为迭代和创造性探索的机会。一个使用乐高积木和定制3D打印组件的案例研究展示了系统如何将数字指导与物理装配联系起来,消除了对零件手动搜索、排序或标记的需要。

英文摘要:

Recent advances in augmented reality (AR) have enabled interactive systems that assist users in physical assembly tasks. In this paper, we present an AR-assisted assembly workflow that leverages object recognition and hand tracking to (1) identify custom components, (2) display step-by-step instructions, (3) detect assembly deviations, and (4) dynamically update the instructions based on users' hands-on interactions with physical parts. Using object recognition, the system detects and localizes components in real time to create a digital twin of the workspace. For each assembly step, it overlays bounding boxes in AR to indicate both the current position and the target placement of relevant components, while hand-tracking data verifies whether the user interacts with the correct part. Rather than enforcing a fixed sequence, the system highlights potential assembly errors and interprets user deviations as opportunities for iteration and creative exploration. A case study with LEGO blocks and custom 3D-printed components demonstrates how the system links digital instructions to physical assembly, eliminating the need for manual searching, sorting, or labeling of parts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑