集成大语言模型与多模态虚拟现实界面以支持协作式人机建筑施工
Integrating Large Language Models with Multimodal Virtual Reality Interfaces to Support Collaborative Human-Robot Construction Work
AI总结：
本文提出融合语音、控制器、ROS、BIM、游戏引擎与LLM聊天界面的多模态VR系统，并通过12名工人的石膏板安装实验验证其低负荷、高可用的人机协作价值。
摘要：
在建筑行业中，工作环境复杂、非结构化且往往具有危险性，人机协作（Human-Robot Collaboration, HRC）的实施正成为一项很有前景的进展。这凸显了对直观通信界面的迫切需求，以使建筑工人能够与机器人助手无缝协作。本研究提出一种对话式虚拟现实（VR）界面，该界面集成多模态交互，以增强建筑工人与机器人之间的直观通信。通过将语音和控制器输入与机器人操作系统（Robot Operating System, ROS）、建筑信息模型（Building Information Modeling, BIM）以及配备由大语言模型（Large Language Model, LLM）驱动的聊天界面的游戏引擎相集成，所提出的系统能够在VR环境中实现直观而精确的交互。该系统由12名建筑工人通过石膏板安装案例研究进行评估，结果表明其凭借简洁的命令输入具有低工作负荷和高可用性。所提出的多模态交互系统表明，此类技术集成能够显著推动机器人助手在建筑行业中的融合应用。
英文摘要：
In the construction industry, where work environments are complex, unstructured and often dangerous, the implementation of Human-Robot Collaboration (HRC) is emerging as a promising advancement. This underlines the critical need for intuitive communication interfaces that enable construction workers to collaborate seamlessly with robotic assistants. This study introduces a conversational Virtual Reality (VR) interface integrating multimodal interaction to enhance intuitive communication between construction workers and robots. By integrating voice and controller inputs with the Robot Operating System (ROS), Building Information Modeling (BIM), and a game engine featuring a chat interface powered by a Large Language Model (LLM), the proposed system enables intuitive and precise interaction within a VR setting. Evaluated by twelve construction workers through a drywall installation case study, the proposed system demonstrated its low workload and high usability with succinct command inputs. The proposed multimodal interaction system suggests that such technological integration can substantially advance the integration of robotic assistants in the construction industry.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
