重申程序化策略的来源：程序化空间与潜在空间
Reclaiming the Source of Programmatic Policies: Programmatic versus Latent Spaces
AI总结：
本文指出无需训练的程序化空间在行为损失上与潜在空间相似，且其优化拓扑结构更有利于局部搜索，因而在POMDPs程序化策略学习上显著优于LEAPS和HPRL。
摘要：
近期的研究引入了LEAPS和HPRL，这两个系统学习领域特定语言的潜在空间，用于为部分可观察马尔可夫决策过程（POMDPs）定义程序化策略。这些系统在优化诸如行为损失等损失函数时诱导出潜在空间，行为损失旨在实现程序行为的局部性，这意味着潜在空间中相近的向量应对应行为相似的程序。在本文中，我们证明了由领域特定语言诱导且无需训练的程序化空间，其行为损失值与先前工作中提出的潜在空间相似。此外，在程序化空间中搜索的算法显著优于在LEAPS和HPRL中的算法。为了解释我们的结果，我们测量了这两种空间对局部搜索算法的“友好度”。我们发现，与在程序化空间中搜索相比，算法在潜在空间中搜索时更容易停滞在局部极大值处。这表明，由奖励函数结合邻域函数诱导的程序化空间的优化拓扑结构比潜在空间更有利于搜索。这一结果解释了程序化空间中性能优越的原因。
英文摘要：
Recent works have introduced LEAPS and HPRL, systems that learn latent spaces of domain-specific languages, which are used to define programmatic policies for partially observable Markov decision processes (POMDPs). These systems induce a latent space while optimizing losses such as the behavior loss, which aim to achieve locality in program behavior, meaning that vectors close in the latent space should correspond to similarly behaving programs. In this paper, we show that the programmatic space, induced by the domain-specific language and requiring no training, presents values for the behavior loss similar to those observed in latent spaces presented in previous work. Moreover, algorithms searching in the programmatic space significantly outperform those in LEAPS and HPRL. To explain our results, we measured the "friendliness" of the two spaces to local search algorithms. We discovered that algorithms are more likely to stop at local maxima when searching in the latent space than when searching in the programmatic space. This implies that the optimization topology of the programmatic space, induced by the reward function in conjunction with the neighborhood function, is more conducive to search than that of the latent space. This result provides an explanation for the superior performance in the programmatic space.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
