arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.08173cs.LGcs.CRcs.ITmath.ITstat.ML
收藏

保护多模态AI中的数据:一种差分隐私的CLIP训练方法

Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training

Harvard University(哈佛大学);Rutgers Univeristy(罗格斯大学)

Alyssa Huang, Peihan Liu, Ryumei Nakada, Linjun Zhang, Wanrong Zhang

AI总结:

针对CLIP多模态训练的隐私泄露风险,本文提出差分隐私方法Dp-CLIP,在分类和VQA等基准上保持接近非私有模型的性能,并在线性设置下分析收敛及效用-隐私权衡。

摘要:

多模态AI的成功激增引发了人们对视觉与语言任务中数据隐私的担忧。尽管CLIP通过在图像和文本上进行联合训练,彻底改变了多模态学习,但其可能在无意中泄露敏感信息,因此有必要整合隐私保护机制。我们提出了对比语言-图像预训练(Contrastive Language-Image Pretraining,CLIP)模型的一种差分隐私适配版本,可在保持准确率的同时有效解决隐私担忧。我们提出的方法Dp-CLIP在涵盖图像分类和视觉问答等多种视觉与语言任务的基准数据集上接受了严格评估。我们证明,该方法保持了与标准非私有CLIP模型相当的性能。此外,我们在线性表示设置下分析了所提算法,推导了算法的收敛率,并表明:当按批次裁剪梯度且损失函数不满足文献中分析DP-SGD时所假设的平滑性条件时,效用与隐私之间存在权衡。

英文摘要:

The surge in multimodal AI's success has sparked concerns over data privacy in vision-and-language tasks. While CLIP has revolutionized multimodal learning through joint training on images and text, its potential to unintentionally disclose sensitive information necessitates the integration of privacy-preserving mechanisms. We introduce a differentially private adaptation of the Contrastive Language-Image Pretraining (CLIP) model that effectively addresses privacy concerns while retaining accuracy. Our proposed method, Dp-CLIP, is rigorously evaluated on benchmark datasets encompassing diverse vision-and-language tasks such as image classification and visual question answering. We demonstrate that our approach retains performance on par with the standard non-private CLIP model. Furthermore, we analyze our proposed algorithm under linear representation settings. We derive the convergence rate of our algorithm and show a trade-off between utility and privacy when gradients are clipped per-batch and the loss function does not satisfy smoothness conditions assumed in the literature for the analysis of DP-SGD.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑