SecretGen：基于分布判别的预训练模型隐私恢复
SecretGen: Privacy Recovery on Pre-Trained Models via Distribution Discrimination
摘要：
通过使用预训练模型的迁移学习已经成为机器学习社区的一个增长趋势。因此，许多预先训练的模型被在线发布，以促进进一步的研究。然而，这引起了广泛的关注，这些预先训练的模型是否会泄漏其训练数据的隐私敏感信息。因此，在这项工作中，我们的目标是回答以下问题：“我们能否有效地从这些预先训练的模型中恢复私人信息？检索此类敏感信息的充分条件是什么？“我们首先探索不同的统计信息，这些信息可以将私人训练分布与其他分布区分开来。根据我们的观察，我们提出了一种新的私有数据重建框架，SecretGen，有效地恢复私人信息。与以前可以通过目标恢复实例的地面真实预测来恢复私有数据的方法相比，SecretGen不需要这样的先验知识，使其更加实用。我们在不同的场景下对不同的数据集进行了广泛的实验，以将SecretGen与其他基线进行比较，并提供一个系统的基准，以更好地了解不同辅助信息和优化操作的影响。我们表明，在没有关于真实类预测的先验知识的情况下，SecretGen能够恢复与利用这种先验知识的数据相比具有类似性能的私有数据。如果给出先验知识，SecretGen将显著优于基线方法。我们还提出了几个量化指标来进一步量化预训练模型的隐私脆弱性，这将有助于隐私敏感应用的模型选择。我们的代码可在https://github.com/AI-secure/SecretGen上获得。
英文摘要：
Transfer learning through the use of pre-trained models has become a growing trend for the machine learning community. Consequently, numerous pre-trained models are released online to facilitate further research. However, it raises extensive concerns on whether these pre-trained models would leak privacy-sensitive information of their training data. Thus, in this work, we aim to answer the following questions: "Can we effectively recover private information from these pre-trained models? What are the sufficient conditions to retrieve such sensitive information?" We first explore different statistical information which can discriminate the private training distribution from other distributions. Based on our observations, we propose a novel private data reconstruction framework, SecretGen, to effectively recover private information. Compared with previous methods which can recover private data with the ground true prediction of the targeted recovery instance, SecretGen does not require such prior knowledge, making it more practical. We conduct extensive experiments on different datasets under diverse scenarios to compare SecretGen with other baselines and provide a systematic benchmark to better understand the impact of different auxiliary information and optimization operations. We show that without prior knowledge about true class prediction, SecretGen is able to recover private data with similar performance compared with the ones that leverage such prior knowledge. If the prior knowledge is given, SecretGen will significantly outperform baseline methods. We also propose several quantitative metrics to further quantify the privacy vulnerability of pre-trained models, which will help the model selection for privacy-sensitive applications. Our code is available at: https://github.com/AI-secure/SecretGen.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
