面向工业规模的检索增强生成框架用于需求工程：基于汽车制造数据的实证评估
An Industrial-Scale Retrieval-Augmented Generation Framework for Requirements Engineering: Empirical Evaluation with Automotive Manufacturing Data
AI总结：
本文提出一个工业级检索增强生成框架，用于需求工程自动化，通过实证评估展示其在提高提取准确性和减少人工分析时间方面的优势。
摘要：
在工业4.0中，需求工程面临异构、非结构化文档的挑战，涵盖技术规范、供应商列表和合规标准。尽管检索增强生成（RAG）在知识密集型任务中显示出潜力，但此前没有工作在真实工业RE工作流程中评估RAG，使用全面的生产级性能指标。本文通过真实汽车制造文档对RAG进行了全面实证评估，这些文档包含2015-2023年间四个规范标准（MBN 9666-1，MBN 9666-2，BQF 9666-5，MBN 9666-9）下的669项需求，以及49个供应商资格的详细支持文档。通过与基于BERT和无地面truth LLM方法的对比，该框架实现了98.2%的提取准确率，具有完整的可追溯性，分别比基线高出24.4%和19.6%。混合语义-词汇检索实现了MRR为0.847。专家质量评估在五个维度上平均得分为4.32/5.0。评估显示，通过多提供者LLM协调，手动分析时间减少了83%，成本节省了47%。消融研究量化了各组件的贡献。纵向分析显示需求体积减少了55%，IT安全关注增加了1800%，识别出10个需要重新认证的遗留供应商（20.4%），代表潜在230万美元的避免合同罚款。
英文摘要：
Requirements engineering in Industry 4.0 faces critical challenges with heterogeneous, unstructured documentation spanning technical specifications, supplier lists, and compliance standards. While retrieval-augmented generation (RAG) shows promise for knowledge-intensive tasks, no prior work has evaluated RAG on authentic industrial RE workflows using comprehensive production-grade performance metrics. This paper presents a comprehensive empirical evaluation of RAG for industrial requirements engineering automation using authentic automotive manufacturing documentation comprising 669 requirements across four specification standards (MBN 9666-1, MBN 9666-2, BQF 9666-5, MBN 9666-9) spanning 2015-2023, plus 49 supplier qualifications with extensive supporting documentation. Through controlled comparisons with BERT-based and ungrounded LLM approaches, the framework achieves 98.2% extraction accuracy with complete traceability, outperforming baselines by 24.4% and 19.6%, respectively. Hybrid semantic-lexical retrieval achieves MRR of 0.847. Expert quality assessment averaged 4.32/5.0 across five dimensions. The evaluation demonstrates 83% reduction in manual analysis time and 47% cost savings through multi-provider LLM orchestration. Ablation studies quantify individual component contributions. Longitudinal analysis reveals a 55% reduction in requirement volume coupled with 1,800% increase in IT security focus, identifying 10 legacy suppliers (20.4%) requiring requalification, representing potential $2.3M in avoided contract penalties.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
