从法规到实施：对工业中大型语言模型（LLM）辅助法规合规性的批判性评估
From Regulation to Implementation: A Critical Evaluation of LLM-Assisted Regulatory Compliance in Industry
- School of Engineering Technology, Purdue University（普渡大学工程技术学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
该研究针对工业中LLM辅助法规合规性的研究缺口，探索数据提取指令和法规模糊性对LLM生成合规文档质量与一致性的影响，发现不同严格程度的法规指南对LLM输出的一致性和幻觉情况有不同影响。
AI中文摘要：
欧盟（EU）已成为可持续性和隐私法规制定领域的领先监管机构。尽管新的法规要求各有不同，但许多都包含用于确保合规性的文档产物。值得注意的是，《可持续产品生态设计法规》（ESPR）引入了数字产品通行证（DPPs）以实现生命周期透明度，而《通用数据保护条例》（GDPR）要求进行数据保护影响评估（DPIAs）以降低隐私风险。然而，创建这些合规性文档产物颇具挑战性：DPPs所需的工业数据通常格式异构且分散在公司及供应商系统中，难以提取为符合要求的DPP格式；此外，DPIA文档需要跨学科专业知识且无标准化格式，这使得新型系统难以开发。为应对这两项法规下合规性文档产物创建的特定复杂性，研究人员提出在生成过程中使用大型语言模型（LLMs），但上述问题对这些系统输出的影响在很大程度上未被探讨。本研究通过探索数据提取指令和法规模糊性如何影响LLM生成的合规性文档产物的质量和一致性，来调查这一现有研究缺口。通过将不同模型与人工创建的基准模式进行基准测试，对生成的文档产物进行评估。结果显示，像DPIA格式这类要求较宽松的指南需要更长的上下文提示来保持一致性和完整性；而像数字电池通行证（DBP）格式这类要求更严格的指南，无论提示上下文如何都能产生一致的结果，但可能会导致输出中出现更多幻觉。
英文摘要：
The European Union (EU) has emerged as a leading regulatory body in the development of sustainability and privacy regulations. While new regulation requirements vary, many include a documentation artifact to ensure compliance. Notably, the Ecodesign for Sustainable Products Regulation (ESPR) introduces Digital Product Passports (DPPs) for life cycle transparency, while the General Data Protection Regulation (GDPR) mandates Data Protection Impact Assessments (DPIAs) to mitigate privacy risks. Creating these compliance artifacts, however, is challenging. Industrial data, which often exists in heterogeneous formats and is scattered across company and supplier systems, is required for DPPs and can be difficult to extract into compliant DPP formatting. Furthermore, DPIA documents require interdisciplinary expertise and follow no standardized format, making development difficult for novel systems. To address the particular complexity of compliance artifact creation for both regulations, researchers have proposed the use of LLMs in the generation process; however, the impact of the aforementioned problems on the output of these systems is largely unaddressed. This work investigates the existing research gap by exploring how data extraction instructions and regulatory vagueness impact the quality and consistency of LLM-produced compliance artifacts. The resulting artifacts are evaluated by benchmarking different models against manually created ground-truth schemas. The results reveal that less strict guidelines, such as DPIA formatting, require higher context prompts to maintain consistency and completeness. Stricter guidelines, such as formatting for Digital Battery Passports (DBP), result in consistent results regardless of prompt context, but may lead to more hallucinations in the output
