BACON：用于数据集蒸馏的贝叶斯最优凝缩框架
BACON: Bayesian Optimal Condensation Framework for Dataset Distillation
AI总结：
针对数据集蒸馏缺乏稳健理论框架导致计算量大且大规模数据集下表现欠佳的问题，提出首个引入贝叶斯理论的凝缩框架BACON，通过最小化期望风险并推导数值下界提供近似解，在多个数据集上显著优于现有方法且可无缝集成。
摘要：
数据集蒸馏（DD）旨在将大规模数据集中的知识蒸馏到更紧凑的数据集中，同时保持测试集上的性能，从而降低存储成本和训练开销。然而，现有方法通常计算量大，特别是由于缺乏分析DD问题的稳健理论框架，在大型数据集规模下表现欠佳。为了应对这些挑战，我们提出了贝叶斯最优凝缩框架（BACON），这是首次将贝叶斯理论框架引入DD文献的工作。该框架为提升DD性能提供了理论支持。此外，BACON利用贝叶斯框架将DD问题表述为联合概率分布中期望风险函数的最小化。另外，通过分析最优凝缩的期望风险函数，我们在特定假设下推导出数值上可行的下界，为BACON提供了近似解。我们在多个数据集上验证了BACON，证明其性能优于现有的最先进方法。例如，在IPC-10设置下，BACON在CIFAR-10数据集上比IDM方法实现了3.46%的准确率提升，在TinyImageNet数据集上实现了3.10%的提升。我们的广泛实验证实了BACON的有效性及其与现有方法的无缝集成，从而提升了DD任务的性能。代码和蒸馏数据集可在BACON获取。
英文摘要：
Dataset Distillation (DD) aims to distill knowledge from extensive datasets into more compact ones while preserving performance on the test set, thereby reducing storage costs and training expenses. However, existing methods often suffer from computational intensity, particularly exhibiting suboptimal performance with large dataset sizes due to the lack of a robust theoretical framework for analyzing the DD problem. To address these challenges, we propose the BAyesian optimal CONdensation framework (BACON), which is the first work to introduce the Bayesian theoretical framework to the literature of DD. This framework provides theoretical support for enhancing the performance of DD. Furthermore, BACON formulates the DD problem as the minimization of the expected risk function in joint probability distributions using the Bayesian framework. Additionally, by analyzing the expected risk function for optimal condensation, we derive a numerically feasible lower bound based on specific assumptions, providing an approximate solution for BACON. We validate BACON across several datasets, demonstrating its superior performance compared to existing state-of-the-art methods. For instance, under the IPC-10 setting, BACON achieves a 3.46% accuracy gain over the IDM method on the CIFAR-10 dataset and a 3.10% gain on the TinyImageNet dataset. Our extensive experiments confirm the effectiveness of BACON and its seamless integration with existing methods, thereby enhancing their performance for the DD task. Code and distilled datasets are available at BACON.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
