arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.09690cs.CLcs.AI
收藏

信任需自担风险:大型语言模型生成专家级系统工程制品能力的混合方法探究及失效模式表征

Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert-Like Systems Engineering Artifacts and a Characterization of Failure Modes

Virginia Tech(弗吉尼亚理工大学);Scale AI(Scale AI公司);Virginia Tech National Security Institute(弗吉尼亚理工大学国家安全研究所)

Taylan G. Topcu, Mohammed Husain, Max Ofsa, Paul Wach

AI总结:

本文以专家级SE制品为基准,通过定量NLP比较与定性分析评估多用途LLM的生成能力,发现其产物虽难被先进算法识别,却存在过早定义需求、无依据估计和过度规格化等隐蔽失效模式。

摘要:

多用途大型语言模型(LLM)是生成式人工智能(AI)的一个子集,近期已取得显著进展。尽管人们对LLM协助系统工程(SE)任务抱有极高期望;但系统的跨学科和复杂性质,加上综合深度领域知识与运行情境的需求,使人们质疑LLM生成SE制品的有效性,尤其是考虑到它们是利用互联网上广泛可得的数据训练而成。为此,我们呈现一项实证探究的结果:以人类专家生成的SE制品作为基准,对其进行解析,并通过提示工程将其输入多种LLM,以生成典型SE制品的各个片段。该过程在未进行任何微调或校准的情况下实施,以记录LLM的基线性能。随后,我们采用双重混合方法,将AI生成的制品与基准进行比较。首先,我们使用自然语言处理算法定量比较这些制品,发现当提示设计得当时,最先进的算法无法区分AI生成制品与人类专家基准。其次,我们开展定性深入分析,考察二者在质量方面有何差异。我们记录到,尽管两类材料看起来非常相似,但AI生成制品表现出可能难以察觉的严重失效模式。我们将其表征为:过早的需求定义、缺乏依据的数值估计,以及过度规格化倾向。我们认为,本研究讲述了一个警示故事,说明为何SE群体在采纳AI建议的反馈时必须更加谨慎,至少在这些反馈由多用途LLM生成时应如此。

英文摘要:

Multi-purpose Large Language Models (LLMs), a subset of generative Artificial Intelligence (AI), have recently made significant progress. While expectations for LLMs to assist systems engineering (SE) tasks are paramount; the interdisciplinary and complex nature of systems, along with the need to synthesize deep-domain knowledge and operational context, raise questions regarding the efficacy of LLMs to generate SE artifacts, particularly given that they are trained using data that is broadly available on the internet. To that end, we present results from an empirical exploration, where a human expert-generated SE artifact was taken as a benchmark, parsed, and fed into various LLMs through prompt engineering to generate segments of typical SE artifacts. This procedure was applied without any fine-tuning or calibration to document baseline LLM performance. We then adopted a two-fold mixed-methods approach to compare AI generated artifacts against the benchmark. First, we quantitatively compare the artifacts using natural language processing algorithms and find that when prompted carefully, the state-of-the-art algorithms cannot differentiate AI-generated artifacts from the human-expert benchmark. Second, we conduct a qualitative deep dive to investigate how they differ in terms of quality. We document that while the two-material appear very similar, AI generated artifacts exhibit serious failure modes that could be difficult to detect. We characterize these as: premature requirements definition, unsubstantiated numerical estimates, and propensity to overspecify. We contend that this study tells a cautionary tale about why the SE community must be more cautious adopting AI suggested feedback, at least when generated by multi-purpose LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑