发表机构
University of Macau（澳门大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文证明数据排列顺序仅影响模型对约定的承诺而非能力，在约定无关度量下12.29σ效应为零，且提示标记约定可消除该效应。
AI 中文摘要
在两种不兼容但均正确的约定下，用相同的问题训练一个模型，并询问该数据的排序在参数中写入了什么。学习率调度并非该问题的背景条件。它是平均算子，并决定了答案。我们证明了一个界，其中排列和调度以独立的相乘因子进入排序效应：排列仅作为块周期，调度仅作为端点能在运行的任何时刻放置的权重。衰减调度无法在同一时刻同时放置大步长和未收缩的余项；常数调度在最后一步恰好做到这一点。衰减会缓和排序效应已在预训练中被报道；机制、分离以及对两半的受控测量是我们的贡献。同一语料库的十种排序、一个预算、除路径外一切固定，在仅lr_scheduler_type不同的族下运行两次：在常数速率下，内部跨度在分配上为0.2221，11.63对比度下限，且随排列的阻塞程度单调变化。在每篇已发表文章使用的单一余弦下，相同的十个臂占据两个可区分状态，而它们自身分辨率本可允许约十个。“顺序重要”和“顺序不重要”是同一个旋钮的两端。路径写入的是模型承诺于哪个约定，而任何精确匹配基准都无法看到它。在十二个臂中，acc_A+acc_B在9.7%以内恒定，而分配份额从0.04到0.87，因此本文测量的12.29σ排列开关在约定无关度量下恰好为零。该守恒在衰减族中被全程引用，常数速率族是读取它的更嘈杂之处。在提示中标记约定会消除该开关，并达到并集上限的87.5%。
英文摘要
"Train a model on the same problems written under two incompatible conventions, both correct, and ask what the ordering of that data writes into the parameters. The learning-rate schedule is not a background condition for that question. It is the averaging operator, and it decides the answer. We prove a bound in which the arrangement and the schedule enter the ordering effect as separate multiplied factors: the arrangement only as a block period, the schedule only as how much weight the endpoint can place on any one moment of the run. A decaying schedule cannot put a large step size and an uncontracted remainder at the same moment; a constant one does exactly that at the last step. That decay moderates ordering effects has been reported in pretraining; the mechanism, the separation, and a controlled measurement of both halves are ours. Ten orderings of one corpus, one budget, everything but the path held fixed, run twice under families differing in lr_scheduler_type and nothing else: at a constant rate the interior spans 0.2221 in allocation, 11.63 contrast floors, monotone in how blocked the arrangement is. Under the single cosine every published arm uses, the same ten arms occupy two distinguishable states where their own resolution would allow about ten. "Order matters" and "order does not matter" are the two ends of one knob. What the path writes is which convention the model commits to, and no exact-match benchmark can see it. Across twelve arms acc_A+acc_B is constant to within 9.7% while the allocation share runs 0.04 to 0.87, so the 12.29-sigma arrangement switch this paper measures is exactly zero under a convention-agnostic metric. That conservation is quoted from the decayed family throughout, the constant-rate one being a noisier place to read it. Marking the convention in the prompt collapses the switch and reaches 87.5% of the union ceiling."
Comments62 pages
