通过频谱动态的视角进行数据整理：静态限制、动态加速与实用预言机
Data Curation Through the Lens of Spectral Dynamics: Static Limits, Dynamic Acceleration, and Practical Oracles
AI总结：
本文通过频谱动态视角提出数据整理方法，揭示静态修剪的局限性及动态加速的潜力，强调理想预言机对学习加速的理论贡献。
摘要：
大规模神经模型越来越多地使用数据修剪、合成数据生成、跨模型蒸馏、人类反馈强化学习（RLHF）和基于难度的采样进行训练。尽管其中几种以数据为中心的策略能够可靠地提高训练效率和下游性能，其他策略却无法提供有意义的提升——最明显的是自动生成的合成数据，通常会增加数据集的体积而不会增强模型的能力。我们正式将数据整理定义为重新加权采样分布，并将其效果映射到数据诱导算子的本征结构上。我们的第一个主要结果表明，静态修剪会诱导一个有界算子，因此不能改变谱尾指数；它只能提供最多有限区域的改进，无法改变渐进神经扩展。我们的第二个结果分析了时间依赖的数据整理，表明一个能够跟踪谱残差并持续重新归一化尾部的理想预言机可以证明性地加速学习——尽管实际系统只能近似这种行为。
英文摘要：
Large-scale neural models are increasingly trained with data pruning, synthetic data generation, cross-model distillation, reinforcement learning from human feedback (RLHF), and difficulty-based sampling. While several of these data-centric strategies reliably improve training efficiency and downstream performance, others fail to provide meaningful gains -- most notably self-generated synthetic data, which often increases dataset volume without enhancing model capability. We formalize data curation as reweighting the sampling distribution and map its effect onto the eigenstructure of the data-induced operator. Our first main result shows that \textbf{static pruning induces a bounded operator and therefore cannot change the spectral tail exponent}; it provides at most finite-region improvements and cannot alter asymptotic neural scaling. Our second result analyzes \textbf{time-dependent data curation}, showing that an ideal oracle capable of tracking spectral residuals and continuously re-normalizing the tail can provably accelerate learning -- although practical systems can only approximate this behavior.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
