arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.15092cs.LGstat.ML
收藏

用路径正则化改进生成流网络

Improving Generative Flow Networks with Path Regularization

University of Texas, Austin(德克萨斯大学奥斯汀分校);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);FPT Software AI Center(FPT软件人工智能中心)

Anh Do, Duy Dinh, Tan Nguyen, Khuong Nguyen, Stanley Osher, Nhat Ho

摘要:

生成流网络(GFlowNets)是最近提出的用于学习随机策略的模型,该随机策略通过具有与给定奖励函数成比例的概率的动作序列来生成组合对象。GFlowNets的核心问题是提高其探索和泛化能力。在这项工作中,我们提出了一种新的路径正则化方法的基础上,最优传输理论的GFlowNets的底层结构上的先验约束。先验被设计成帮助GFlowNets更好地发现目标分布的潜在结构或增强其在主动学习背景下探索环境的能力。路径正则化通过最大化两个前向策略之间的最优传输距离来控制GFlowNets中的流,以生成更多样和新颖的候选项,或者通过最小化最优传输距离来提高泛化能力。此外,我们通过在特定情况下找到其封闭形式的解决方案和一个有意义的上限,可以作为一个近似值,以最小化正则化项,从而获得一个有效的实现的正则化。我们经验证明了我们的路径正则化在广泛的任务上的优势,包括合成超网格环境建模,离散概率建模和生物序列设计。

英文摘要:

Generative Flow Networks (GFlowNets) are recently proposed models for learning stochastic policies that generate compositional objects by sequences of actions with the probability proportional to a given reward function. The central problem of GFlowNets is to improve their exploration and generalization. In this work, we propose a novel path regularization method based on optimal transport theory that places prior constraints on the underlying structure of the GFlowNets. The prior is designed to help the GFlowNets better discover the latent structure of the target distribution or enhance its ability to explore the environment in the context of active learning. The path regularization controls the flow in GFlowNets to generate more diverse and novel candidates via maximizing the optimal transport distances between two forward policies or to improve the generalization via minimizing the optimal transport distances. In addition, we derive an efficient implementation of the regularization by finding its closed form solutions in specific cases and a meaningful upper bound that can be used as an approximation to minimize the regularization term. We empirically demonstrate the advantage of our path regularization on a wide range of tasks, including synthetic hypergrid environment modeling, discrete probabilistic modeling, and biological sequence design.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑