SAGE：面向代表性子集选择的流式一致性驱动梯度素描
SAGE: Streaming Agreement-Driven Gradient Sketches for Representative Subset Selection
AI总结：
SAGE提出一种流式梯度素描方法，通过Frequent Directions和共识方向对齐选择代表性子集，以恒定内存实现高效训练，在多个基准上保持精度并降低计算开销。
摘要：
在现代神经网络的大规模数据集训练中，计算和能源消耗巨大。我们提出SAGE，一种流式子集选择方法，它以$O(\ell D)$内存维护梯度几何的紧凑Frequent Directions（FD）素描，并优先选择其素描梯度与共识方向一致的样本。该方法消除了$N \times N$的成对相似性计算和显式的$N \times \ell$梯度存储，形成了一个简单的两遍、GPU友好的流程。利用FD的确定性近似保证，我们分析了在主要素描子空间内一致性评分如何保留梯度能量。在多个基准测试中，SAGE以较小的保留率预算进行训练，同时保持了与全数据训练和近期子集选择基线相当的准确性，并减少了端到端计算和峰值内存。总体而言，SAGE提供了一种实用的、恒定内存的替代方案，补充了剪枝和模型压缩，以实现高效训练。
英文摘要：
Training modern neural networks on large datasets is computationally and energy intensive. We present SAGE, a streaming data-subset selection method that maintains a compact Frequent Directions (FD) sketch of gradient geometry in $O(\ell D)$ memory and prioritizes examples whose sketched gradients align with a consensus direction. The approach eliminates $N \times N$ pairwise similarities and explicit $N \times \ell$ gradient stores, yielding a simple two-pass, GPU-friendly pipeline. Leveraging FD's deterministic approximation guarantees, we analyze how agreement scoring preserves gradient energy within the principal sketched subspace. Across multiple benchmarks, SAGE trains with small kept-rate budgets while retaining competitive accuracy relative to full-data training and recent subset-selection baselines, and reduces end-to-end compute and peak memory. Overall, SAGE offers a practical, constant-memory alternative that complements pruning and model compression for efficient training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
