脱离顺序思考：当输出顺序不再反映推理顺序时扩散语言模型的表现
Thinking Out of Order: When Output Order Stops Reflecting Reasoning Order in Diffusion Language Models
AI总结：
本文提出屏蔽扩散语言模型（MDLMs）在处理输出顺序与推理顺序不一致时展现出的顺序鲁棒性，通过实验验证其在不同基准上的稳定性能。
摘要：
自回归（AR）语言模型强制采用从左到右的生成顺序，当所需输出结构与自然推理相冲突时（例如由于展示或模式约束导致需在解释之前生成答案）会形成根本性的限制。在这种情况下，AR模型必须在生成中间推理之前就确定答案，这种刚性约束迫使提前承诺。屏蔽扩散语言模型（MDLMs），它们通过并行迭代地细化所有标记，提供了一种将计算顺序与输出结构解耦的方法。我们通过GSM8K、Math500和我们引入的具有受控难度和顺序级评估的ReasonOrderQA基准验证了这一能力。当提示要求在推理之前生成答案时，AR模型与标准推理链顺序相比表现出显著的准确性差距（最高可达67%的相对下降），而MDLMs保持稳定（≤14%的相对下降），这一特性我们称之为“顺序鲁棒性”。使用ReasonOrderQA，我们提供了证据表明，MDLMs通过在扩散过程中比复杂标记（如最终答案）更早稳定简化标记（如推理步骤），使推理标记在答案承诺之前稳定下来。最后，我们识别了这种优势减弱的失败条件，概述了顺序鲁棒性的必要限制。
英文摘要：
Autoregressive (AR) language models enforce a fixed left-to-right generation order, creating a fundamental limitation when the required output structure conflicts with natural reasoning (e.g., producing answers before explanations due to presentation or schema constraints). In such cases, AR models must commit to answers before generating intermediate reasoning, and this rigid constraint forces premature commitment. Masked diffusion language models (MDLMs), which iteratively refine all tokens in parallel, offer a way to decouple computation order from output structure. We validate this capability on GSM8K, Math500, and ReasonOrderQA, a benchmark we introduce with controlled difficulty and order-level evaluation. When prompts request answers before reasoning, AR models exhibit large accuracy gaps compared to standard chain-of-thought ordering (up to 67% relative drop), while MDLMs remain stable ($\leq$14% relative drop), a property we term "order robustness". Using ReasonOrderQA, we present evidence that MDLMs achieve order robustness by stabilizing simpler tokens (e.g., reasoning steps) earlier in the diffusion process than complex ones (e.g., final answers), enabling reasoning tokens to stabilize before answer commitment. Finally, we identify failure conditions where this advantage weakens, outlining the limits required for order robustness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
