通过马尔可夫链视角控制GFlowNets中的探索-利用
Controlling Exploration-Exploitation in GFlowNets via Markov Chain Perspectives
AI总结:
通过引入可调参数$α$,$α$-GFNs改进了GFlowNets的探索-利用平衡,提升了模式发现能力。
摘要:
生成流网络(GFlowNet)目标隐式地固定了前向和后向策略的等量混合,这可能会在训练过程中限制探索-利用权衡。通过进一步探索GFlowNets与马尔可夫链之间的联系,我们建立了GFlowNet目标与马尔可夫链可逆性之间的等价关系,从而揭示了此类约束的来源,并提供了一种将马尔可夫链性质适应到GFlowNets的框架。基于这些理论发现,我们提出了$α$-GFNs,通过可调参数$α$扩展了混合。这种扩展使能够直接控制探索-利用动态以增强模式发现能力,同时确保收敛到唯一的流。在各种基准测试中,包括Set、Bit序列和分子生成,$α$-GFN目标 consistently 超过之前的GFlowNet目标,实现了发现模式数量的10倍增加。
英文摘要:
Generative Flow Network (GFlowNet) objectives implicitly fix an equal mixing of forward and backward policies, potentially constraining the exploration-exploitation trade-off during training. By further exploring the link between GFlowNets and Markov chains, we establish an equivalence between GFlowNet objectives and Markov chain reversibility, thereby revealing the origin of such constraints, and provide a framework for adapting Markov chain properties to GFlowNets. Building on these theoretical findings, we propose $α$-GFNs, which generalize the mixing via a tunable parameter $α$. This generalization enables direct control over exploration-exploitation dynamics to enhance mode discovery capabilities, while ensuring convergence to unique flows. Across various benchmarks, including Set, Bit Sequence, and Molecule Generation, $α$-GFN objectives consistently outperform previous GFlowNet objectives, achieving up to a $10 \times$ increase in the number of discovered modes.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。