ES-Trace：超越模型卡的代码生成模型伦理来源披露审计

中文摘要 AI 辅助

代码生成模型在软件开发中的应用日益广泛，但其开发过程引发了涉及知识产权、隐私、公平性、劳动实践和环境影响的伦理来源问题。尽管先前的研究已定义了代码生成的伦理来源标准，但现有模型披露了多少证据以及这些证据位于何处仍不明确。我们提出了ES-Trace，一个用于伦理来源披露审计的框架，该框架利用模型文档可追溯性图（MDTG）追踪模型卡之外的已披露证据，MDTG表示模型、版本和文档工件之间的关系。我们将ES-Trace应用于来自10个出版商的26个模型，涵盖77份文档和20个ES-CodeGen方面。仅基于模型卡的审计平均得分为1.77/5，而解析声明的引用后得分提升至2.82/5，其中大部分提升来自出版商在结构化元数据中声明的文档。我们研究的关键发现包括：（1）即使将审计扩展到完整文档范围，社会和劳动相关方面仍记录不足；（2）解析文档引用显著提高了观察到的披露水平，将平均得分从1.77/5提升至2.82/5；（3）仅基于模型卡的审计可能错误描述发布级披露变化；（4）文档不匹配可能将证据关联到错误的模型或版本，凸显了明确模型-版本绑定以及跨文档工件一致性的必要性。我们的研究呼吁进行引用感知的伦理来源披露审计、明确的模型-版本绑定和跨文档工件的一致性，以及加强对当前报告不足的社会和劳动相关方面的文档记录。

英文摘要

Code generation models have been increasingly used in software development, but their development raises ethical-sourcing concerns involving intellectual property, privacy, fairness, labour practices, and environmental impact. Although prior work has defined ethical-sourcing criteria for code generation, it remains unclear how much evidence existing models disclose and where that evidence can be found. We introduce ES-Trace, a framework for ethical-sourcing disclosure audits that traces disclosed evidence beyond model cards using the Model Documentation Traceability Graph (MDTG), which represents relationships among models, versions, and documentation artifacts. We apply ES-Trace to 26 models from 10 publishers across 77 documents and 20 ES-CodeGen aspects. Model-card-only auditing yields a mean score of 1.77/5, while resolving the declared references increases it to 2.82/5, with most of the increase arising from documents that the publisher declares in structured metadata. The key findings of our study include: (1) social and labour-related aspects remain poorly documented, even when expanding the audit to the full documentation scope, (2) resolving documentation references substantially increases observed disclosure, raising the mean score from 1.77/5 to 2.82/5, (3) model-card-only audits can mischaracterize release-level disclosure changes, and (4) documentation mismatches can associate evidence with the wrong model or version, highlighting the need for explicit model--version binding and consistency across documentation artifacts. Our study calls for reference-aware ethical-sourcing disclosure audits, explicit model--version binding and consistency across documentation artifacts, and stronger documentation of currently underreported social and labour-related aspects.

发表机构

• Concordia University（康考迪亚大学）

机构由 AI 辅助整理，请以论文原文为准。