DGPO：基于强化学习的图扩散用于神经架构生成
DGPO: RL-Steered Graph Diffusion for Neural Architecture Generation
AI总结：
DGPO通过拓扑节点排序和位置编码扩展强化学习微调，用于生成有向无环图的神经架构搜索，验证结果表明其在NAS-Bench-201任务中达到最优性能，且模型能学习可转移的结构先验。
摘要：
强化学习微调已被证明在图像和分子领域有效，用于引导生成扩散模型向期望属性发展。图扩散模型已应用于组合结构生成，包括神经架构搜索（NAS）。然而，神经架构是有向无环图（DAG），其中边的方向编码功能语义如数据流信息，而现有图扩散方法设计用于无向结构，因此会丢弃此类信息。我们提出有向图策略优化（DGPO），通过拓扑节点排序和位置编码扩展强化学习微调离散图扩散模型至DAG。在NAS-Bench-101和NAS-Bench-201上验证，DGPO在所有三个NAS-Bench-201任务中达到基准最优（91.61%、73.49%、46.77%）。核心发现是模型学习了可转移的结构先验：预训练仅在7%的搜索空间上，微调后生成接近 oracle 的架构，在0.32个百分点内达到全数据模型性能，并在训练上限外扩展7.3个百分点。双向控制实验确认了真实的奖励驱动引导，反向优化达到接近随机猜测准确率（9.5%）。这些结果表明，一旦扩展到处理方向性，强化学习引导的离散扩散提供了可控的生成框架用于有向组合结构。
英文摘要：
Reinforcement learning fine-tuning has proven effective for steering generative diffusion models toward desired properties in image and molecular domains. Graph diffusion models have similarly been applied to combinatorial structure generation, including neural architecture search (NAS). However, neural architectures are directed acyclic graphs (DAGs) where edge direction encodes functional semantics such as data flow-information that existing graph diffusion methods, designed for undirected structures, discard. We propose Directed Graph Policy Optimization (DGPO), which extends reinforcement learning fine-tuning of discrete graph diffusion models to DAGs via topological node ordering and positional encoding. Validated on NAS-Bench-101 and NAS-Bench-201, DGPO matches the benchmark optimum on all three NAS-Bench-201 tasks (91.61%, 73.49%, 46.77%). The central finding is that the model learns transferable structural priors: pretrained on only 7% of the search space, it generates near-oracle architectures after fine-tuning, within 0.32 percentage points of the full-data model and extrapolating 7.3 percentage points beyond its training ceiling. Bidirectional control experiments confirm genuine reward-driven steering, with inverse optimization reaching near random-chance accuracy (9.5%). These results demonstrate that reinforcement learning-steered discrete diffusion, once extended to handle directionality, provides a controllable generative framework for directed combinatorial structures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
