回顾：面向目标条件GFlowNets的回顾式反向合成
Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNets
AI总结：
针对目标条件GFlowNets训练中稀疏奖励和轨迹覆盖不足的问题，提出回顾式反向合成（RBS）方法，通过合成反向轨迹丰富训练数据，显著提升样本效率并超越多个基准上的强基线。
摘要：
生成流网络（GFlowNets）是一类新型的概率采样器，与通常收敛于单一最优解的标准回报最大化方法（如强化学习）相比，已展现出生成多样化高回报候选集的卓越能力。近期研究聚焦于开发目标条件GFlowNets，旨在训练一个能够根据任务指定实现不同结果的单一GFlowNet。然而，由于奖励极其稀疏，特别是在高维问题中，训练此类模型颇具挑战性。此外，先前方法在训练过程中面临探索轨迹覆盖范围有限的问题，当仅有离线数据可用时，这一挑战更为突出。在本工作中，我们提出了一种名为回顾式反向合成（RBS）的新方法来解决这些关键问题。具体而言，RBS在目标条件GFlowNets中合成新的反向轨迹，以丰富训练轨迹的质量和多样性，从而引入大量可学习信号，有效应对稀疏奖励问题。大量实证结果表明，我们的方法大幅提升了样本效率，并在多个标准评估基准上超越了强基线方法。
英文摘要：
Generative Flow Networks (GFlowNets), a new family of probabilistic samplers, have demonstrated remarkable capabilities to generate diverse sets of high-reward candidates, in contrast to standard return maximization approaches (e.g., reinforcement learning) which often converge to a single optimal solution. Recent works have focused on developing goal-conditioned GFlowNets, which aim to train a single GFlowNet capable of achieving different outcomes as the task specifies. However, training such models is challenging due to extremely sparse rewards, particularly in high-dimensional problems. Moreover, previous methods suffer from the limited coverage of explored trajectories during training, which presents more pronounced challenges when only offline data is available. In this work, we propose a novel method called \textbf{R}etrospective \textbf{B}ackward \textbf{S}ynthesis (\textbf{RBS}) to address these critical problems. Specifically, RBS synthesizes new backward trajectories in goal-conditioned GFlowNets to enrich training trajectories with enhanced quality and diversity, thereby introducing copious learnable signals for effectively tackling the sparse reward problem. Extensive empirical results show that our method improves sample efficiency by a large margin and outperforms strong baselines on various standard evaluation benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
