arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.08890cs.LGcs.AIcs.SE
收藏

ModelDiff:基于测试的DNN相似性比较模型重用检测

ModelDiff: Testing-Based DNN Similarity Comparison for Model Reuse Detection

Microsoft Research(微软研究院);Peking University(北京大学);Institute for AI Industry Research (AIR), Tsinghua University(清华大学智能产业研究院(AIR))

Yuanchun Li, Ziqi Zhang, Bingyan Liu, Ziyue Yang, Yunxin Liu

摘要:

深度学习模型的知识可能会转移到学生模型中,导致知识产权侵权或漏洞传播。检测这种知识重用是不平凡的,因为可疑模型可能不是白盒可访问的和/或可能服务于不同的任务。在本文中,我们提出了ModelDiff,这是一种基于测试的深度学习模型相似性比较方法。我们不是直接比较两个模型的权重、激活或输出,而是在同一组测试输入上比较它们的行为模式。具体来说,模型的行为模式表示为决策距离向量(DDV),其中每个元素是模型对一对输入的反应之间的距离。模型之间的知识相似度是用它们的DDV之间的余弦相似度来度量的。为了评估ModelDiff,我们创建了一个基准测试,其中包含144对模型,涵盖了最流行的模型重用方法,包括迁移学习、模型压缩和模型窃取。在基准测试中,我们的方法达到了91.7%的正确率,这证明了使用ModelDiff进行模型重用检测的有效性。一项关于移动深度学习应用的研究表明了ModelDiff在现实世界模型上的可行性。

英文摘要:

The knowledge of a deep learning model may be transferred to a student model, leading to intellectual property infringement or vulnerability propagation. Detecting such knowledge reuse is nontrivial because the suspect models may not be white-box accessible and/or may serve different tasks. In this paper, we propose ModelDiff, a testing-based approach to deep learning model similarity comparison. Instead of directly comparing the weights, activations, or outputs of two models, we compare their behavioral patterns on the same set of test inputs. Specifically, the behavioral pattern of a model is represented as a decision distance vector (DDV), in which each element is the distance between the model's reactions to a pair of inputs. The knowledge similarity between two models is measured with the cosine similarity between their DDVs. To evaluate ModelDiff, we created a benchmark that contains 144 pairs of models that cover most popular model reuse methods, including transfer learning, model compression, and model stealing. Our method achieved 91.7% correctness on the benchmark, which demonstrates the effectiveness of using ModelDiff for model reuse detection. A study on mobile deep learning apps has shown the feasibility of ModelDiff on real-world models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑