Recent advances in Large Language Models (LLMs) have revolutionized code generation, leading to widespread adoption of AI coding tools by developers. However, LLMs can generate license-protected code without providing the necessary license information, leading to potential intellectual property violations during software production. This paper addresses the critical, yet underexplored, issue of license compliance in LLM-generated code by establishing a benchmark to evaluate the ability of LLMs to provide accurate license information for their generated code. To establish this benchmark, we conduct an empirical study to identify a reasonable standard for "striking similarity" that excludes the possibility of independent creation, indicating a copy relationship between the LLM output and certain open-source code. Based on this standard, we propose LiCoEval, to evaluate the license compliance capabilities of LLMs, i.e., the ability to provide accurate license or copyright information when they generate code with striking similarity to already existing copyrighted code. Using LiCoEval, we evaluate 14 popular LLMs, finding that even top-performing LLMs produce a non-negligible proportion (0.88% to 2.01%) of code strikingly similar to existing open-source implementations. Notably, most LLMs fail to provide accurate license information, particularly for code under copyleft licenses. These findings underscore the urgent need to enhance LLM compliance capabilities in code generation tasks. Our study provides a foundation for future research and development to improve license compliance in AI-assisted software development, contributing to both the protection of open-source software copyrights and the mitigation of legal risks for LLM users.
Can I use this publicly available dataset to build commercial AI software? -- A Case Study on Publicly Available Image Datasets
Centre for Software Excellence, Huawei Canada（华为加拿大卓越软件中心）；Lassonde School of Engineering, York University（约克大学拉松德工程学院）；Huawei China（华为中国）；University of Victoria（维多利亚大学）
Time-Series Forecasting in Safety-Critical Environments: An Open-Source Package for EU-AI-Act-Compliant Development / Zeitreihenprognose in sicherheitskritischen Umgebungen: Ein Open-Source-Paket für die KI-VO-konforme Entwicklung
Apertus: Democratizing Open and Compliant LLMs for Global Language Environments
EPFL（洛桑联邦理工学院）；ETH Zurich（苏黎世联邦理工学院）；CSCS（瑞士国家超级计算中心）；HES-SO Valais-Wallis（瑞士西部应用科学与艺术大学瓦莱州分校）；HSLU（卢塞恩应用科学与艺术大学）；IST Austria（奥地利科学技术研究所）；ZHAW（苏黎世应用科学大学）；University of Zurich（苏黎世大学）；University of Bern（伯尔尼大学）；Vischer
A Large Scale Empirical Analysis on the Adherence Gap between Standards and Tools in SBOM
Intelligent Software Research Center, Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所智能软件研究中心）；University of Chinese Academy of Sciences（中国科学院大学）；Key Laboratory of System Software (Chinese Academy of Sciences)（中国科学院系统软件重点实验室）
