自监督表示引导的生成式数据集蒸馏
Self-Supervised Representation-Guided Generative Dataset Distillation
浏览论文内容
中文总结 AI 辅助
本文提出SRG框架，将SSL几何转化为扩散引导，通过分阶段引导策略实现数据集蒸馏，在多数据集和IPC设置下优于生成式基线，可跨预训练表示空间迁移。
中文摘要 AI 辅助
数据集蒸馏是将大型训练集压缩为紧凑的合成集，同时保留其下游效用的技术。现有大多数方法针对随机初始化的网络，而现代视觉系统常采用冻结的预训练编码器搭配轻量模块，因此蒸馏样本需保留预训练表示空间的判别性几何结构，这是现有生成式目标未明确考虑的。本文提出自监督表示引导的生成式数据集蒸馏（SRG）框架，该框架将自监督学习（SSL）的几何结构转化为扩散模型的引导信号。具体而言，SRG从真实图像的SSL表示中构建类别原型，并通过三个SSL空间目标实现引导：原型对齐、类别间判别和类别内分配。在扩散采样过程中，SRG采用分阶段引导策略：早期去噪锚定到与分配原型的SSL表示最接近的真实图像的隐变量，后期去噪则由SSL空间目标引导。这种划分既保留了生成先验提供的视觉真实性，又逐步将样本导向SSL表示空间中具有代表性和类别判别性的区域。SRG在多个数据集和每类样本数（IPC）设置下，始终优于所评估的生成式基线方法；跨编码器评估进一步表明其可跨预训练表示空间迁移。这些结果证明了利用表示引导生成结合预训练SSL模型进行数据集蒸馏的有效性。
英文摘要
Dataset distillation compresses a large training set into a compact synthetic set while retaining its downstream utility. Most existing methods target randomly initialized networks, whereas modern vision systems often adapt frozen pretrained encoders with lightweight modules. Distilled samples should therefore preserve the discriminative geometry of the pretrained representation space, which existing generative objectives do not explicitly consider. We propose self-supervised representation-guided generative dataset distillation (SRG), a framework that translates the SSL geometry into diffusion guidance. Specifically, SRG constructs class-wise prototypes from real-image SSL representations and performs guidance through three SSL-space objectives for prototype alignment, inter-class discrimination, and intra-class assignment. During diffusion sampling, it adopts a stage-wise guidance strategy: early denoising is anchored to the latent of the real image whose SSL representation is nearest to the assigned prototype, whereas later denoising is guided by the SSL-space objectives. This division preserves the visual realism provided by the generative prior while progressively steering samples toward representative and class-discriminative regions of the SSL representation space. SRG consistently outperforms the evaluated generative baselines across multiple datasets and IPC settings. A cross-encoder evaluation further indicates transfer across pretrained representation spaces. These results demonstrate the effectiveness of representation-guided generation for dataset distillation with pretrained SSL models.
