arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2508.02490cs.AI
收藏

PHM-Bench:面向预测与健康管理的大模型系统评估领域专用基准框架

PHM-Bench: A Domain-Specific Benchmarking Framework for Systematic Evaluation of Large Models in Prognostics and Health Management

Hangzhou International Innovation Institute, Beihang University(北京航空航天大学杭州国际创新学院);Institute of Reliability Engineering, Beihang University(北京航空航天大学可靠性工程研究所);Science & Technology on Reliability & Environmental Engineering Laboratory(可靠性与环境工程技术重点实验室);School of Reliability and Systems Engineering, Beihang University(北京航空航天大学可靠性与系统工程学院)

Puyu Yang, Laifa Tao, Zijian Huang, Haifei Liu, Wenyan Cao, Hao Ji, Jianan Qiu, Qixuan Huang, Xuanyuan Su, Yuhang Xie, Jun Zhang, Shangyu Li, Chen Lu, Zhixuan Lian

AI总结:

针对PHM领域大模型评估不足的问题,提出三维评估框架PHM-Bench,定义多层级指标,利用案例与工业数据集实现多维评估,为LLM在PHM中的系统评测奠定基础。

摘要:

随着生成式人工智能的快速发展,大语言模型(LLMs)在工业领域中的应用日益广泛,为预测与健康管理(PHM)带来了新的机遇。这些模型有助于应对开发成本高、部署周期长以及泛化能力有限等挑战。然而,尽管PHM与LLMs之间的协同作用日益增强,现有的评估方法在结构完整性、维度全面性和评估粒度方面往往存在不足。这阻碍了LLMs在PHM领域的深度融合。为解决这些局限,本研究提出了PHM-Bench,一种面向PHM大模型的新型三维评估框架。PHM-Bench基于基础能力、核心任务和全生命周期三元结构,针对PHM系统工程的特殊需求进行定制。它定义了涵盖知识理解、算法生成和任务优化的多层级评估指标。这些指标与典型PHM任务(包括状态监测、故障诊断、剩余使用寿命(RUL)预测和维护决策)保持一致。利用精选案例集和公开可用的工业数据集,本研究实现了对通用模型和领域专用模型在不同PHM任务上的多维评估。PHM-Bench为LLMs在PHM中的大规模评估奠定了方法论基础,并提供了一个关键基准,以指导从通用模型向PHM专用模型的转变。

英文摘要:

With the rapid advancement of generative artificial intelligence, large language models (LLMs) are increasingly adopted in industrial domains, offering new opportunities for Prognostics and Health Management (PHM). These models help address challenges such as high development costs, long deployment cycles, and limited generalizability. However, despite the growing synergy between PHM and LLMs, existing evaluation methodologies often fall short in structural completeness, dimensional comprehensiveness, and evaluation granularity. This hampers the in-depth integration of LLMs into the PHM domain. To address these limitations, this study proposes PHM-Bench, a novel three-dimensional evaluation framework for PHM-oriented large models. Grounded in the triadic structure of fundamental capability, core task, and entire lifecycle, PHM-Bench is tailored to the unique demands of PHM system engineering. It defines multi-level evaluation metrics spanning knowledge comprehension, algorithmic generation, and task optimization. These metrics align with typical PHM tasks, including condition monitoring, fault diagnosis, RUL prediction, and maintenance decision-making. Utilizing both curated case sets and publicly available industrial datasets, our study enables multi-dimensional evaluation of general-purpose and domain-specific models across diverse PHM tasks. PHM-Bench establishes a methodological foundation for large-scale assessment of LLMs in PHM and offers a critical benchmark to guide the transition from general-purpose to PHM-specialized models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑