arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.04835cs.CVcs.AIcs.LG
收藏

将数据集蒸馏到神经场中

Distilling Dataset into Neural Field

Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院)

Donghyeok Shin, HeeSun Bae, Gyuwon Sim, Wanmo Kang, Il-Chul Moon

AI总结:

本文提出了一种名为 DDiF 的数据集蒸馏新型参数化框架,利用神经场存储大规模数据集信息,在同等预算下展现出更强表达能力,并在图像、视频、音频和 3D 体素等多个基准数据集上取得卓越性能。

摘要:

利用大规模数据集对于训练高性能深度学习模型至关重要,但这也会带来巨大的计算和存储开销。为了克服这些挑战,数据集蒸馏作为一种极具前景的解决方案应运而生,它将大规模数据集压缩成一个较小的合成数据集,同时保留训练所需的必要信息。本文提出了一种用于数据集蒸馏的新型参数化框架,命名为 Distilling Dataset into Neural Field (DDiF),该框架利用神经场来存储大规模数据集的必要信息。由于神经场以坐标作为输入并输出数量,这种独特性质使得 DDiF 能够有效保留信息,并轻松生成各种形状的数据。我们从理论上证实,当单个合成实例所使用的预算相同时,DDiF 比以往的一些文献展现出更强的表达能力。通过广泛的实验,我们证明了 DDiF 在多个基准数据集上取得了卓越的性能,其应用不仅限于图像领域,还涵盖了视频、音频和 3D 体素。我们在 https://github.com/aailab-kaist/DDiF 发布了代码。

英文摘要:

Utilizing a large-scale dataset is essential for training high-performance deep learning models, but it also comes with substantial computation and storage costs. To overcome these challenges, dataset distillation has emerged as a promising solution by compressing the large-scale dataset into a smaller synthetic dataset that retains the essential information needed for training. This paper proposes a novel parameterization framework for dataset distillation, coined Distilling Dataset into Neural Field (DDiF), which leverages the neural field to store the necessary information of the large-scale dataset. Due to the unique nature of the neural field, which takes coordinates as input and output quantity, DDiF effectively preserves the information and easily generates various shapes of data. We theoretically confirm that DDiF exhibits greater expressiveness than some previous literature when the utilized budget for a single synthetic instance is the same. Through extensive experiments, we demonstrate that DDiF achieves superior performance on several benchmark datasets, extending beyond the image domain to include video, audio, and 3D voxel. We release the code at https://github.com/aailab-kaist/DDiF.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑