深入探讨利用DL API创建恶意AI模型的滥用及如何检测
Deep Dive into the Abuse of DL APIs To Create Malicious AI Models and How to Detect Them
AI总结：
本文探讨了利用DL API创建恶意AI模型的潜在风险及检测方法，通过展示攻击方式和利用LLM构建扫描器来提高检测能力。
摘要：
根据Gartner的报告，到2025年底，超过70%的组织将已将AI模型集成到其工作流程中。为了降低成本并促进创新，通常会从像Hugging Face或TensorFlow Hub这样的模型库中获取预训练模型。然而，这种做法引入了安全风险，因为攻击者可以将恶意代码注入到他们上传到这些库中的模型中，导致各种攻击，包括远程代码执行（RCE）、敏感数据外泄和系统文件修改，当这些模型被加载或执行（预测功能）时。由于AI模型在数字化转型中起着关键作用，这将大大增加软件供应链攻击的数量。虽然已有几种努力在反序列化基于pickle保存的模型时检测恶意软件（将恶意软件隐藏在模型参数中），但利用DL API（例如TensorFlow API）的滥用风险仍被低估。具体来说，我们展示了如何利用TensorFlow API的隐藏功能，如文件读写和网络发送/接收，以及其持久化API来发起攻击。值得注意的是，Hugging Face和TensorFlow Hub等模型库中的现有扫描器无法检测到此类API的某些隐蔽滥用。这是因为扫描工具只有一组通过语法识别的可疑功能集被分析。它们通常没有对所用功能进行语义层面的理解。在展示可能的攻击后，我们展示了如何利用LLMs识别可能被滥用的隐藏API功能，并构建扫描器来检测此类滥用。
英文摘要：
According to Gartner, more than 70% of organizations will have integrated AI models into their workflows by the end of 2025. In order to reduce cost and foster innovation, it is often the case that pre-trained models are fetched from model hubs like Hugging Face or TensorFlow Hub. However, this introduces a security risk where attackers can inject malicious code into the models they upload to these hubs, leading to various kinds of attacks including remote code execution (RCE), sensitive data exfiltration, and system file modification when these models are loaded or executed (predict function). Since AI models play a critical role in digital transformation, this would drastically increase the number of software supply chain attacks. While there are several efforts at detecting malware when deserializing pickle based saved models (hiding malware in model parameters), the risk of abusing DL APIs (e.g. TensorFlow APIs) is understudied. Specifically, we show how one can abuse hidden functionalities of TensorFlow APIs such as file read/write and network send/receive along with their persistence APIs to launch attacks. It is concerning to note that existing scanners in model hubs like Hugging Face and TensorFlow Hub are unable to detect some of the stealthy abuse of such APIs. This is because scanning tools only have a syntactically identified set of suspicious functionality that is being analysed. They often do not have a semantic-level understanding of the functionality utilized. After demonstrating the possible attacks, we show how one may identify potentially abusable hidden API functionalities using LLMs and build scanners to detect such abuses.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
