多模态制造安全聊天机器人：知识库设计、基准开发及多种RAG方法的评估
A Multimodal Manufacturing Safety Chatbot: Knowledge Base Design, Benchmark Development, and Evaluation of Multiple RAG Approaches
AI总结：
本文提出了一种多模态制造安全聊天机器人，通过RAG方法实现高准确率、低延迟和低成本的安全培训，展示了其在工业5.0环境中的应用价值。
摘要：
确保工人安全仍然是现代制造环境中的关键挑战。工业5.0重新定向现有的制造范式，使其更加以人为核心。采用设计科学研究方法，我们识别出下一代安全培训系统三大必要要求：高准确性、低延迟和低成本。我们引入了一个由大型语言模型驱动的多模态聊天机器人，以满足这些设计要求。该聊天机器人使用检索增强生成技术，使其响应基于精心编纂的监管和技术文档。为了评估我们的解决方案，我们开发了一个领域特定的基准，包含三个代表性机器的专家验证的问题和答案对：Bridgeport手动铣床、Haas TL-1数控车床和Universal Robots UR5e协作机器人。我们使用全因子设计测试了24种RAG配置，并通过自动评估正确性、延迟和成本来评估它们。我们的前两名配置随后由十位行业专家和学术研究人员进行评估。我们的结果表明，检索策略和模型配置对性能有显著影响。选定的顶级配置，用于聊天机器人部署，实现了86.66%的准确率，平均成本为每查询0.005美元，平均端到端延迟为10.04秒。该延迟适用于提供完整的安全指导，并且是从查询提交到完整指导交付而非生成开始测量的。总体而言，我们的工作提供了三个贡献：一个开源、领域相关的安全培训聊天机器人；一个验证的基准，用于评估AI辅助的安全指导；以及一个系统的方法，用于设计和评估面向工业5.0环境的AI增强型指导和沉浸式安全培训系统。
英文摘要：
Ensuring worker safety remains a critical challenge in modern manufacturing environments. Industry 5.0 reorients the prevailing manufacturing paradigm toward more human-centric operations. Using a design science research methodology, we identify three essential requirements for next-generation safety training systems: high accuracy, low latency, and low cost. We introduce a multimodal chatbot powered by large language models that meets these design requirements. The chatbot uses retrieval-augmented generation to ground its responses in curated regulatory and technical documentation. To evaluate our solution, we developed a domain-specific benchmark of expert-validated question and answer pairs for three representative machines: a Bridgeport manual mill, a Haas TL-1 CNC lathe, and a Universal Robots UR5e collaborative robot. We tested 24 RAG configurations using a full-factorial design and assessed them with automated evaluations of correctness, latency, and cost. Our top 2 configurations were then evaluated by ten industry experts and academic researchers. Our results show that retrieval strategy and model configuration have a significant impact on performance. The top configuration, selected for chatbot deployment, achieved an accuracy of 86.66%, an average cost of $0.005 per query, and an average end-to-end latency of 10.04 seconds. This latency is practical for delivering a complete safety instruction and is measured from query submission to full instruction delivery rather than generation onset. Overall, our work provides three contributions: an open-source, domain-grounded safety training chatbot; a validated benchmark for evaluating AI-assisted safety instruction; and a systematic methodology for designing and assessing AI-enabled instructional and immersive safety training systems for Industry 5.0 environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
