运维智能的可信知识提取
Trusted Knowledge Extraction for Operations and Maintenance Intelligence
AI总结：
本研究针对运维智能中的知识提取挑战，分解知识图谱构建过程，评估16种NLP工具与LLMs在航空领域零样本性能，提出信任增强建议并开源数据集。
摘要：
从组织数据存储库中提取运维智能是一个关键挑战，原因在于数据保密性与数据集成目标之间的二元对立，以及自然语言处理（NLP）工具相对于运维等领域的特定知识结构的局限性。在这项工作中，我们讨论了知识图谱的构建，并将知识提取过程分解为其命名实体识别、共指消解、命名实体链接和关系提取功能组件。然后，我们评估了十六种NLP工具，与大型语言模型（LLMs）快速发展的能力进行协同或对比。我们专注于飞机工业中可信应用的运维智能用例。一个基线数据集源自美国联邦航空管理局的丰富公共领域数据集，该数据集聚焦于设备故障或维护需求。我们评估了可在受控、保密环境中运行的NLP和LLM工具的零样本性能（不向第三方发送数据）。基于我们观察到的显著性能限制，我们讨论了与可信NLP和LLM工具相关的挑战，以及它们在航空等关键任务行业中更广泛使用的技术就绪水平。最后，我们提出了增强信任的建议，并提供了我们开源的精编数据集，以支持进一步的基线测试和评估。
英文摘要：
Deriving operational intelligence from organizational data repositories is a key challenge due to the dichotomy of data confidentiality vs data integration objectives, as well as the limitations of Natural Language Processing (NLP) tools relative to the specific knowledge structure of domains such as operations and maintenance. In this work, we discuss Knowledge Graph construction and break down the Knowledge Extraction process into its Named Entity Recognition, Coreference Resolution, Named Entity Linking, and Relation Extraction functional components. We then evaluate sixteen NLP tools in concert with or in comparison to the rapidly advancing capabilities of Large Language Models (LLMs). We focus on the operational and maintenance intelligence use case for trusted applications in the aircraft industry. A baseline dataset is derived from a rich public domain US Federal Aviation Administration dataset focused on equipment failures or maintenance requirements. We assess the zero-shot performance of NLP and LLM tools that can be operated within a controlled, confidential environment (no data is sent to third parties). Based on our observation of significant performance limitations, we discuss the challenges related to trusted NLP and LLM tools as well as their Technical Readiness Level for wider use in mission-critical industries such as aviation. We conclude with recommendations to enhance trust and provide our open-source curated dataset to support further baseline testing and evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
