arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.03351cs.HC
收藏

用于维护任务的增强现实:基于ChatGPT的自动文本到动作

Augmented Reality for Maintenance Tasks with ChatGPT for Automated Text-to-Action

Fang Xu, Tri Nguyen, Jing Du

AI总结:

本研究提出一个结合增强现实、光学字符识别和GPT语言模型的系统,用于优化运营与维护任务中的用户表现,实验表明该系统能缩短任务完成时间、降低认知负荷并提升用户信任。

摘要:

传感器技术、人工智能(AI)和增强现实(AR)的进步为各个领域带来了新的机遇。AR和像GPT这样的大型语言模型已经取得了实质性进展,并越来越多地被应用于不同领域。其中一个有前景的应用是在运营与维护(O&M)中。O&M任务通常涉及复杂的程序和步骤,这些程序和步骤可能难以记忆和正确执行,特别是对于新手或在高压力情境下。通过将虚拟对象叠加到物理世界的优势与使用GPT生成类似人类文本的能力相结合,我们可以彻底改变O&M操作。本研究介绍了一个结合AR、光学字符识别(OCR)和GPT语言模型的系统,以优化用户表现,同时提供可信的交互并减轻O&M任务中的工作负荷。该系统提供了一个由Unity游戏引擎控制的交互式虚拟环境,促进了虚拟现实与物理现实之间的无缝交互。我们进行了一项案例研究(N=15)来说明研究结果并回答研究问题。结果表明,使用我们提出的AR和AI系统,用户可以在更短的时间内完成同样具有挑战性的任务。此外,收集到的数据还表明,在使用AR和AI系统执行相同操作时,认知负荷有所降低,信任度有所提高。

英文摘要:

Advancements in sensor technology, artificial intelligence (AI), and augmented reality (AR) have unlocked opportunities across various domains. AR and large language models like GPT have witnessed substantial progress and are increasingly being employed in diverse fields. One such promising application is in operations and maintenance (O&M). O&M tasks often involve complex procedures and sequences that can be challenging to memorize and execute correctly, particularly for novices or under high-stress situations. By marrying the advantages of superimposing virtual objects onto the physical world, and generating human-like text using GPT, we can revolutionize O&M operations. This study introduces a system that combines AR, Optical Character Recognition (OCR), and the GPT language model to optimize user performance while offering trustworthy interactions and alleviating workload in O&M tasks. This system provides an interactive virtual environment controlled by the Unity game engine, facilitating a seamless interaction between virtual and physical realities. A case study (N=15) is conducted to illustrate the findings and answer the research questions. The results indicate that users can complete similarly challenging tasks in less time using our proposed AR and AI system. Moreover, the collected data also suggests a reduction in cognitive load and an increase in trust when executing the same operations using the AR and AI system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑