通过粗到细的小波建模分解私有图像生成
Decomposing Private Image Generation via Coarse-to-Fine Wavelet Modeling
AI总结：
本文提出基于小波空间的频谱DP框架，通过粗到细的两阶段方法提升图像生成的隐私与效用平衡。
摘要：
训练在敏感图像数据集上的生成模型有风险记住并复现个别训练示例，这使得强有力的隐私保证变得至关重要。虽然差分隐私（DP）为这种保证提供了一个原理性框架，但标准的DP微调（例如使用DP-SGD）往往导致图像质量严重退化，尤其是在高频纹理方面，因为噪声被 indiscriminately 添加到所有模型参数上。在本工作中，我们提出了一种基于频谱DP的框架，基于这样一个假设：图像中最敏感的部分往往是小波空间中的低频成分（例如面部特征和物体形状），而高频成分大多是通用的和公开的。基于这一假设，我们提出了一个两阶段的DP图像生成框架，使用粗略的图像中间体：（1）在敏感图像的低分辨率小波系数上对自回归频谱图像标记器模型进行DP微调；（2）使用预先训练好的公开超分辨率模型进行高分辨率上采样。通过将隐私预算限制在图像的全局结构上，并利用DP的后处理性质进行细节优化，我们实现了隐私和效用之间的有希望的权衡。在MS-COCO和MM-CelebA-HQ数据集上的实验表明，我们的方法在图像质量和风格捕捉方面比其他领先的DP图像框架有所改进。
英文摘要：
Generative models trained on sensitive image datasets risk memorizing and reproducing individual training examples, making strong privacy guarantees essential. While differential privacy (DP) provides a principled framework for such guarantees, standard DP finetuning (e.g., with DP-SGD) often results in severe degradation of image quality, particularly in high-frequency textures, due to the indiscriminate addition of noise across all model parameters. In this work, we propose a spectral DP framework based on the hypothesis that the most privacy-sensitive portions of an image are often low-frequency components in the wavelet space (e.g., facial features and object shapes) while high-frequency components are largely generic and public. Based on this hypothesis, we propose the following two-stage framework for DP image generation with coarse image intermediaries: (1) DP finetune an autoregressive spectral image tokenizer model on the low-resolution wavelet coefficients of the sensitive images, and (2) perform high-resolution upsampling using a publicly pretrained super-resolution model. By restricting the privacy budget to the global structures of the image in the first stage, and leveraging the post-processing property of DP for detail refinement, we achieve promising trade-offs between privacy and utility. Experiments on the MS-COCO and MM-CelebA-HQ datasets show that our method generates images with improved quality and style capture relative to other leading DP image frameworks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
