从更少样本到更少位数：将数据集蒸馏重新框架化为精度和紧凑性的联合优化
From Fewer Samples to Fewer Bits: Reframing Dataset Distillation as Joint Optimization of Precision and Compactness
AI总结：
QuADD通过联合优化数据集紧凑性和精度，在固定位预算下提升数据集蒸馏的效率和性能。
摘要：
数据集蒸馏（DD）将大规模数据集压缩成紧凑的合成数据集，以保持训练性能。然而，当前方法主要关注样本减少，对数据精度及其对效率的影响考虑有限。我们提出了量化感知的数据集蒸馏（QuADD），一个统一的框架，能够在固定位预算下联合优化数据集的紧凑性和精度。QuADD在蒸馏循环中集成了可微分的量化模块，使合成样本和量化参数的端到端联合优化成为可能。基于率失真视角，我们实证分析了样本数量和精度之间位分配如何影响学习性能。我们的框架支持均匀和自适应非均匀量化，后者从数据中学习量化级别，以更好地表示信息密集区域。在图像分类和3GPP波束管理任务上的实验表明，QuADD在每比特精度上超过了现有DD和后量化基线，在信息高效的数据集蒸馏方面建立了新标准。
英文摘要：
Dataset Distillation (DD) compresses large datasets into compact synthetic ones that maintain training performance. However, current methods mainly target sample reduction, with limited consideration of data precision and its impact on efficiency. We propose Quantization-aware Dataset Distillation (QuADD), a unified framework that jointly optimizes dataset compactness and precision under fixed bit budgets. QuADD integrates a differentiable quantization module within the distillation loop, enabling end-to-end co-optimization of synthetic samples and quantization parameters. Guided by the rate-distortion perspective, we empirically analyze how bit allocation between sample count and precision influences learning performance. Our framework supports both uniform and adaptive non-uniform quantization, where the latter learns quantization levels from data to represent information-dense regions better. Experiments on image classification and 3GPP beam management tasks show that QuADD surpasses existing DD and post-quantized baselines in accuracy per bit, establishing a new standard for information-efficient dataset distillation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
