可解释性作为合规要求：受监管行业对用于设计制品生成的AI工具的需求
Explainability as a Compliance Requirement: What Regulated Industries Need from AI Tools for Design Artifact Generation
AI总结：
本研究通过访谈安全关键行业从业者，揭示了AI设计制品生成工具因缺乏可解释性导致的合规风险与效率损失，并提出了来源追踪、决策理由和合规验证等改进路线图。
摘要：
用于自动化设计制品生成的AI工具在需求工程（RE）中越来越多地被用于将文本需求转换为结构化图表和模型。尽管这些AI工具，特别是基于自然语言处理（NLP）的工具，有望提高效率，但它们在受监管行业的采用仍然有限，因为透明性和可追溯性在这些行业中至关重要。在本文中，我们通过对来自安全关键行业的十位从业者进行半结构化访谈，调查了AI驱动的设计制品生成中的可解释性差距。我们研究了当前基于AI的工具如何集成到工作流中，以及由于缺乏可解释性而带来的挑战。我们还探讨了缓解策略、其对项目结果的影响，以及提高可用性所需的功能。我们的发现表明，不可解释的AI输出需要大量的手动验证，降低了利益相关者的信任，难以处理领域特定术语，破坏了团队协作，并引入了监管合规风险，通常抵消了预期的效率收益。为了解决这些问题，我们确定了关键改进措施，包括来源追踪、为工具生成的决策提供清晰的理由、支持领域特定适应，以及启用合规性验证。本研究概述了提高AI工具在需求工程工作流中的透明度、可靠性和适用性的实用路线图，特别是在可解释性对于采用和认证至关重要的受监管和安全关键环境中。
英文摘要：
Artificial Intelligence (AI) tools for automating design artifact generation are increasingly used in Requirements Engineering (RE) to transform textual requirements into structured diagrams and models. While these AI tools, particularly those based on Natural Language Processing (NLP), promise to improve efficiency, their adoption remains limited in regulated industries where transparency and traceability are essential. In this paper, we investigate the explainability gap in AI-driven design artifact generation through semi-structured interviews with ten practitioners from safety-critical industries. We examine how current AI-based tools are integrated into workflows and the challenges arising from their lack of explainability. We also explore mitigation strategies, their impact on project outcomes, and features needed to improve usability. Our findings reveal that non-explainable AI outputs necessitate extensive manual validation, reduce stakeholder trust, struggle to handle domain-specific terminology, disrupt team collaboration, and introduce regulatory compliance risks, often negating the anticipated efficiency benefits. To address these issues, we identify key improvements, including source tracing, providing clear justifications for tool-generated decisions, supporting domain-specific adaptation, and enabling compliance validation. This study outlines a practical roadmap for improving the transparency, reliability, and applicability of AI tools in requirements engineering workflows, particularly in regulated and safety-critical environments where explainability is crucial for adoption and certification.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
