arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.04640cs.LG
收藏

域桥:基于生成模型的黑盒模型数据域取证

Domain Bridge: Generative model-based domain forensic for black-box models

School of Computing(计算机学院);National University of Singapore(新加坡国立大学)

Jiyi Zhang, Han Fang, Ee-Chien Chang

AI总结:

本文提出一种基于生成模型和迭代细化的黑盒模型数据域取证方法,利用LAION-5B和Stable Diffusion识别输入域的细粒度属性,如人脸是否戴眼镜。

摘要:

在机器学习模型的取证调查中,确定模型数据域的技术发挥着至关重要的作用,先前的工作依赖于像ImageNet这样的大规模语料库来近似目标模型的域。尽管这类方法在寻找广泛域方面有效,但它们往往难以识别这些域内更细粒度的类别。在本文中,我们引入了一种增强方法,不仅确定一般数据域(例如,人脸),还确定其特定属性(例如,戴眼镜)。我们的方法使用图像嵌入模型作为编码器,生成模型作为解码器。从粗粒度的描述开始,解码器生成一组图像,然后将这些图像呈现给未知的目标模型。模型的成功分类引导编码器细化描述,进而用于在后续迭代中生成更具体的图像集。这种迭代细化缩小了确切感兴趣的类别。我们方法的一个关键优势在于利用庞大的数据集LAION-5B,生成模型Stable Diffusion在该数据集上训练。这使我们的搜索空间超越了传统语料库,如ImageNet。实证结果展示了我们的方法在识别模型输入域特定属性方面的性能,为深度学习模型更详细的取证分析铺平了道路。

英文摘要:

In forensic investigations of machine learning models, techniques that determine a model's data domain play an essential role, with prior work relying on large-scale corpora like ImageNet to approximate the target model's domain. Although such methods are effective in finding broad domains, they often struggle in identifying finer-grained classes within those domains. In this paper, we introduce an enhanced approach to determine not just the general data domain (e.g., human face) but also its specific attributes (e.g., wearing glasses). Our approach uses an image embedding model as the encoder and a generative model as the decoder. Beginning with a coarse-grained description, the decoder generates a set of images, which are then presented to the unknown target model. Successful classifications by the model guide the encoder to refine the description, which in turn, are used to produce a more specific set of images in the subsequent iteration. This iterative refinement narrows down the exact class of interest. A key strength of our approach lies in leveraging the expansive dataset, LAION-5B, on which the generative model Stable Diffusion is trained. This enlarges our search space beyond traditional corpora, such as ImageNet. Empirical results showcase our method's performance in identifying specific attributes of a model's input domain, paving the way for more detailed forensic analyses of deep learning models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑