AI 中文总结
该研究发现稀疏MoE中专家约简顺序等因素会导致执行行为分歧，确定了post-mHC与完整持久状态的边界属性，为MoE运行时和硬件后端提供了数值兼容性契约。
AI 中文摘要
数学上等价的专家约简顺序会产生明显不同的稀疏MoE执行结果。我们在原生DeepSeek-V4-Flash中隔离了该效应，方法是冻结局部MoE状态，仅改变聚合语义。四种方案将操作数表示与累加器精度分离开来。在第5层分支处，720种A模式顺序产生10个延续盆地；720种B模式顺序形成360个精确结构类和11个盆地。在一个中文提示下，B类被拆分为202个“ layoffs（裁员）”、113个“ hiring（录用）”和45个其他延续。最大L∞范数B分支选择按8、16、32个 token 分别区分50个提示中的12、24、36个。在每种方案的192条持久轨迹中，P32、A和B会改变每条原生参考路径轨迹，而C则保留路径、 token 序列和文本。对192条轨迹的单独C检查与原生MoE、post-mHC、next-router和LM状态逐位匹配。对于一个受控B分支，精确的post-mHC端点重构可复现测得的下游轨迹。在下次解码边界处，对该分支完整持久状态进行精确FP64重构，在给定相同自然生成的下一个输入时，7步内的301个下游post-mHC状态、301个持久状态检查点、301条路径、预测结果和文本均达成一致。这些控制措施确定post-mHC为词元内边界，完整持久状态为跨词元延续边界。相同的 token 不一定意味着相同的自回归状态：分歧可跨越词元边界并在后续显现。这些结果使专家操作数转换、累加器精度和约简顺序成为稀疏MoE运行时和硬件后端的数值兼容性契约的一部分。它们确立了受控因果可能性，而非部署发生率；C的顺序不变性仅限于评估的6项状态和调度。
英文摘要
Mathematically equivalent expert-reduction orders can produce observably different sparse-MoE executions. We isolate this effect in native DeepSeek-V4-Flash by freezing local MoE state and varying only aggregation semantics. Four schemes separate operand representation from accumulator precision. At one layer-5 fork, 720 A-mode orders yield 10 continuation basins; 720 B-mode orders form 360 exact structural classes and 11 basins. Under one Chinese prompt, the B classes split into 202 layoffs, 113 hiring, and 45 other continuations. Maximum-L-infinity B-branch selection separates 12, 24, and 36 of 50 prompts by 8, 16, and 32 tokens. Across 192 persistent trajectories per scheme, P32, A, and B change every native-reference route trajectory, while C preserves routes, token sequences, and texts. A separate 192-trajectory C check matches native MoE, post-mHC, next-router, and LM states bitwise. For one controlled B branch, exact post-mHC endpoint reconstruction reproduces the measured downstream trajectory. At the next decode boundary, exact FP64 reconstruction of the branch's full persistent state yields agreement for 301 downstream post-mHC states, 301 persistent-state checkpoints, 301 routes, predictions, and text over seven steps, given the same naturally generated next input. These controls identify post-mHC as an intra-token boundary and full persistent state as a cross-token continuation boundary. Identical tokens need not imply identical autoregressive state: divergence can survive a token boundary and become visible later. These results make expert operand conversion, accumulator precision, and reduction order part of a numerical compatibility contract for sparse-MoE runtimes and hardware backends. They establish controlled causal possibility, not deployment incidence; C's order invariance is limited to evaluated six-term states and schedules.
Comments32 pages, 3 figures
