对比学习增强的小规模数据集蒸馏轨迹匹配方法
Contrastive Learning-Enhanced Trajectory Matching for Small-Scale Dataset Distillation
AI总结：
针对小规模数据集蒸馏中语义信息丢失问题，提出在轨迹匹配中融入对比学习以增强合成样本的判别性与多样性，实验证明该方法在极少量合成数据下显著提升模型性能。
摘要：
在资源受限的环境中部署机器学习模型，例如边缘设备或快速原型设计场景，日益需要将大型数据集蒸馏为显著更小但信息丰富的合成数据集。当前的数据集蒸馏技术，特别是轨迹匹配方法，优化合成数据以使模型在合成样本上的训练轨迹与在真实数据上的训练轨迹相一致。尽管这些方法在中等规模合成数据集上表现出有效性，但在极端样本稀缺情况下未能充分保留语义丰富性。为解决这一局限，我们提出了一种新颖的数据集蒸馏方法，在图像合成过程中整合对比学习。通过显式最大化实例级特征判别性，我们的方法即使在数据集规模受到显著限制时也能产生更具信息性和多样性的合成样本。实验结果表明，融入对比学习显著提升了在极小规模合成数据集上训练的模型的性能。这种整合不仅引导了更有效的特征表示，还显著提高了合成图像的视觉保真度。实验结果表明，我们的方法在现有蒸馏技术之上取得了显著性能提升，尤其是在合成数据极其有限的场景中。
英文摘要：
Deploying machine learning models in resource-constrained environments, such as edge devices or rapid prototyping scenarios, increasingly demands distillation of large datasets into significantly smaller yet informative synthetic datasets. Current dataset distillation techniques, particularly Trajectory Matching methods, optimize synthetic data so that the model's training trajectory on synthetic samples mirrors that on real data. While demonstrating efficacy on medium-scale synthetic datasets, these methods fail to adequately preserve semantic richness under extreme sample scarcity. To address this limitation, we propose a novel dataset distillation method integrating contrastive learning during image synthesis. By explicitly maximizing instance-level feature discrimination, our approach produces more informative and diverse synthetic samples, even when dataset sizes are significantly constrained. Experimental results demonstrate that incorporating contrastive learning substantially enhances the performance of models trained on very small-scale synthetic datasets. This integration not only guides more effective feature representation but also significantly improves the visual fidelity of the synthesized images. Experimental results demonstrate that our method achieves notable performance improvements over existing distillation techniques, especially in scenarios with extremely limited synthetic data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
