CEDD-optimizer：在地理分布式边缘系统上实现成本高效的数据集蒸馏

中文摘要 AI 辅助

集中式学习是现代人工智能中的一种基本范式，其中数据从分布式边缘设备收集并在中央主机处聚合以进行模型训练。然而，这一流程常常受到数据收集的大量通信开销的瓶颈限制。数据集蒸馏（DD）凭借其高压缩比，因此对于分布式数据上的集中式学习具有吸引力。然而，DD在非均匀边缘环境中的成本效率在很大程度上仍未得到探索。地理分布的边缘设备面临不同的能源和数据传输价格，而关键的DD超参数，如目标压缩比和蒸馏步数，会显著影响能源使用、传输开销和下游测试准确率。因此，这些超参数必须在本地和全局范围内进行调整。我们提出了成本高效的数据集蒸馏优化器（CEDD-optimizer），这是一个用于成本高效的分布式DD的超参数调整框架。它通过优化跨边缘设备的超参数设置并考虑环境非均匀性，在训练质量约束下最小化总成本。该框架包含两个模块：CEDD-calibrator和CEDD-solver。CEDD-calibrator识别我们的能源和训练质量模型的参数：前者通过离线校准确定，而后者通过三步调整方案在线估计。基于这些模型，CEDD-solver解决成本最小化问题，以引导和改进分布式DD工作流。在各种图像数据集上的实验表明，在相同的质量约束下，我们的方法相比基线DD方法实现了高达20.8的改进。

英文摘要

Centralized learning is a fundamental paradigm in modern AI, in which data are collected from distributed edge devices and aggregated at a central host for model training. However, this pipeline is often bottlenecked by the substantial communication overhead of data collection. Dataset Distillation (DD), with its high compression ratio, is therefore attractive for centralized learning on distributed data. Yet, the cost efficiency of DD in non-uniform edge environments remains largely unexplored. Geographically distributed edge devices face different energy and data-transfer prices, while key DD hyperparameters, such as the target compression ratio and number of distillation steps, substantially affect energy use, transfer overhead, and downstream test accuracy. These hyperparameters must therefore be tuned both locally and globally. We propose the Cost-Efficient Dataset Distillation optimizer (CEDD-optimizer), a hyperparameter-tuning framework for cost-efficient distributed DD. It minimizes total cost under a training-quality constraint by optimizing hyperparameter settings across edge devices while accounting for environmental non-uniformity. The framework comprises two modules: CEDD-calibrator and CEDD-solver. CEDD-calibrator identifies parameters for our energy and training-quality models: the former is determined through offline calibration, whereas the latter is estimated online through a three-step tuning scheme. Based on these models, CEDD-solver solves the cost-minimization problem to steer and improve the distributed DD workflow. Experiments across various image datasets show that our approach achieves up to a 20.8 improvement over the baseline DD method under the same quality constraint.

发表机构

• Technical University of Munich（慕尼黑工业大学）

机构由 AI 辅助整理，请以论文原文为准。