揭开状态的面纱：状态自适应何时对掩码扩散语言模型重要

中文摘要 AI 辅助

掩码扩散语言模型（MDMs）允许灵活的解码顺序，使得解掩码策略成为一种推理决策。现有方法在如何优先处理位置、控制并行性、限制选择区域、修正预测或规划未来去噪方面各不相同，但仍不清楚这些选择在生成过程中何时应发生变化。我们通过策略反转来研究这个问题，即在反转中，替代行动变得比固定选择更可取。我们将MDM推理组织为五个轴——评分、基数、区域、承诺和规划——并将自适应机会定义为最佳候选行动相对于验证集选择的固定行动的一步效用优势。这一观点表明，自适应价值既取决于此类反转的频率，也取决于其幅度。在三个MDM和十个任务中，自适应机会高度异质，某些情况表现出集中且可预测的一步增益。这激发了选择性自适应：在验证提示上校准的轻量级检测器识别高机会状态，例如，在LLaDA-8B受约束JSON填充中，仅自适应前10%的状态即可捕获候选集预言机机会的56.9%。我们的转换级结果表明，状态自适应在选择性应用时最为有用，而非均匀应用。

英文摘要

Masked diffusion language models (MDMs) admit flexible generation orders, making the unmasking strategy an inference decision. Existing methods vary in how they prioritize positions, control parallelism, restrict selection regions, revise predictions, or plan future denoising, yet it remains unclear when these choices should change during generation. We study this question through strategy reversals, where an alternative action becomes preferable to a fixed choice. We organize MDM inference into five axes--score, cardinality, region, commitment, and planning--and define adaptation opportunity as the one-step utility advantage of the best candidate action over a validation-selected fixed action. This view shows that adaptation value depends on both the frequency and magnitude of such reversals. Across three MDMs and ten tasks, adaptation opportunities are highly heterogeneous, with some regimes exhibiting concentrated and predictable one-step gains. This motivates selective adaptation: lightweight detectors calibrated on validation prompts identify high-opportunity states, capturing, for example, 56.9 percent of the candidate-set oracle opportunity by adapting only the top 10 percent of states on LLaDA-8B constrained JSON filling. Our transition-level results suggest that state adaptation is most useful when applied selectively rather than uniformly.

发表机构

• Seoul National University（首尔国立大学）

机构由 AI 辅助整理，请以论文原文为准。