GLOBE：用于核心集选择的轨迹对齐梯度匹配与结构化稀疏优化
GLOBE: Trajectory-Aligned Gradient Matching with Structured SparseOptimization for Coreset Selection
浏览论文内容
中文总结 AI 辅助
本研究提出名为GLOBE的轨迹对齐核心集选择框架，通过多阶分布匹配与结构化稀疏优化提升核心集质量，在低保留率下的下游测试准确率优于现有方法，助力设备端数据高效学习。
中文摘要 AI 辅助
深度神经网络的设备端训练本质上受限于大规模数据集的计算与内存成本，核心集选择通过仅保留少量真实训练样本提供了实用解决方案。然而，现有基于梯度的方法通常依赖单一模型快照计算的梯度，并采用贪心或追踪式选择流程，限制了其捕捉优化动态变化及处理强相关样本的能力。我们提出GLOBE（Gradient Local-Balanced Extraction，梯度局部均衡提取），一种轨迹对齐的核心集选择框架，将样本选择建模为全局优化的稀疏加权问题。GLOBE通过跨多个训练检查点构建的梯度轨迹表示每个样本，从而捕捉其在优化不同阶段的影响。为保留全数据集的训练行为，我们引入多阶匹配目标，联合对齐梯度轨迹的一阶均值与投影非中心化二阶矩。GLOBE进一步结合Group LASSO、Elastic Net正则化及非负预算约束，以诱导组级和样本级稀疏性，同时稳定相关轨迹的权重。最后，类别均衡的Top-K选择在有限采样预算下维持充足的类别覆盖。在六个基准数据集和五种评估架构上的实验表明，GLOBE在下游测试准确率上始终优于现有核心集选择方法，尤其在低保留率时优势显著。这些结果凸显了结合动态梯度信息、多阶分布匹配与结构化稀疏性对数据高效学习的有效性。
英文摘要
On-device training of deep neural networks is fundamentally constrained by the computational and memory costs of large-scale datasets. Coreset selection offers a practical solution by retaining only a compact subset of real training samples. However, existing gradient-based methods commonly rely on gradients computed at a single model snapshot and employ greedy or pursuit-based selection procedures, limiting their ability to capture evolving optimization dynamics and handle strongly correlated samples. We propose GLOBE (Gradient Local-Balanced Extraction), a trajectory-aligned coreset selection framework that formulates sample selection as a globally optimized sparse weighting problem. GLOBE represents each sample by a gradient trajectory constructed across multiple training checkpoints, thereby capturing its influence throughout different stages of optimization. To preserve the training behavior of the full dataset, we introduce a multi-order matching objective that jointly aligns the first-order mean and projected uncentered second-order moments of gradient trajectories. GLOBE further combines Group LASSO, Elastic Net regularization, and nonnegative budget constraints to induce group- and sample-level sparsity while stabilizing the weights of correlated trajectories. Finally, class-balanced Top-K selection maintains adequate category coverage under limited sampling budgets. Experiments across six benchmarks and five evaluation architectures demonstrate that GLOBE consistently outperforms existing coreset selection methods in downstream test accuracy, particularly at low retention ratios. These results highlight the effectiveness of combining dynamic gradient information, multi-order distribution matching, and structured sparsity for data-efficient learning.
