arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.04358cs.SE
收藏

生成式AI在系统工程中的应用:大型语言模型风险评估框架

Generative AI in Systems Engineering: A Framework for Risk Assessment of Large Language Models

FZI Research Center for Information Technology(FZI信息技术研究中心)

Stefan Otten, Philipp Reis, Philipp Rigoll, Joshua Ransiek, Tobias Schürmann, Jacob Langner, Eric Sax

AI总结:

本文提出了一种用于评估大型语言模型在系统工程中应用风险的框架,通过自主性和影响两个维度分类,帮助组织实现安全透明的部署。

摘要:

随着大型语言模型(LLMs)的广泛应用,其在工程生命周期中的各个阶段都提供了显著的机会,包括需求工程、软件开发、过程优化和决策支持。尽管有这种潜力,但组织在评估LLM使用相关的风险方面面临重大挑战,导致集成不一致、未知的故障模式和有限的可扩展性。本文介绍了LLM风险评估框架(LRF),这是一种用于评估LLMs在系统工程(SE)环境中的应用的结构化方法。该框架将基于LLM的应用沿着两个基本维度进行分类:自主性,从支持性协助到完全自动化的决策制定,以及影响,反映错误或误导性模型输出对工程过程和系统元素的潜在严重性。通过结合这些维度,LRF能够在开发周期内一致地确定相应的风险级别。所得到的分类支持组织在识别适当的有效性策略、监督水平和所需措施以确保安全和透明的部署。该框架因此有助于将AI技术的快速演变与已建立的工程原则(可靠性、可追溯性和受控的过程整合)对齐。总体而言,LRF为在复杂工程环境中风险意识地采用LLMs提供了基础,并代表了系统工程中标准化AI保证实践的第一步。

英文摘要:

The increasing use of Large Language Models (LLMs) offers significant opportunities across the engineering lifecycle, including requirements engineering, software development, process optimization, and decision support. Despite this potential, organizations face substantial challenges in assessing the risks associated with LLM use, resulting in inconsistent integration, unknown failure modes, and limited scalability. This paper introduces the LLM Risk Assessment Framework (LRF), a structured approach for evaluating the application of LLMs within Systems Engineering (SE) environments. The framework classifies LLM-based applications along two fundamental dimensions: autonomy, ranging from supportive assistance to fully automated decision making, and impact, reflecting the potential severity of incorrect or misleading model outputs on engineering processes and system elements. By combining these dimensions, the LRF enables consistent determination of corresponding risk levels across the development lifecycle. The resulting classification supports organizations in identifying appropriate validation strategies, levels of human oversight, and required countermeasures to ensure safe and transparent deployment. The framework thereby helps align the rapid evolution of AI technologies with established engineering principles of reliability, traceability, and controlled process integration. Overall, the LRF provides a basis for risk-aware adoption of LLMs in complex engineering environments and represents a first step toward standardized AI assurance practices in systems engineering.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑