通用基准低估了程序化策略的泛化能力
Common Benchmarks Undervalue the Generalization Power of Programmatic Policies
AI总结：
该研究指出常用基准低估了程序化策略的泛化能力，通过调整神经策略训练流程（如用稀疏观测的简单架构、安全奖励函数）可让神经策略达到与程序化策略相当的OOD泛化效果，并提议构建契合程序化表示特性的基准任务。
摘要：
针对序列决策问题学习程序化表示的算法通常在分布外（OOD）问题上进行评估，普遍结论是程序化策略在OOD问题上的泛化能力优于神经策略。在这篇立场论文中，我们认为常用的基准低估了程序化表示的泛化能力。我们分析了文献中四篇论文的实验，结果表明，此前被认为不具备泛化能力的神经策略，在OOD问题上可以达到与程序化策略相当的泛化效果。这一结果通过对神经策略训练流程进行简单调整即可实现。具体而言，我们证明，采用与程序化策略相同类型的稀疏观测的更简单神经架构，有助于实现OOD泛化。另一个被证实有效的改进是使用能生成更安全策略的奖励函数（例如，行驶缓慢的智能体泛化能力更强）。此外，我们主张构建能够凸显OOD泛化所需核心概念的基准问题，这类问题可能对神经策略构成挑战，但与程序化表示相契合，例如需要栈等算法结构的任务。
英文摘要：
Algorithms for learning programmatic representations for sequential decision-making problems are often evaluated on out-of-distribution (OOD) problems, with the common conclusion that programmatic policies generalize better than neural policies on OOD problems. In this position paper, we argue that commonly used benchmarks undervalue the generalization capabilities of programmatic representations. We analyze the experiments of four papers from the literature and show that neural policies, which were shown not to generalize, can generalize as effectively as programmatic policies on OOD problems. This is achieved with simple changes in the neural policies training pipeline. Namely, we show that simpler neural architectures with the same type of sparse observation used with programmatic policies can help attain OOD generalization. Another modification we have shown to be effective is the use of reward functions that allow for safer policies (e.g., agents that drive slowly can generalize better). Also, we argue for creating benchmark problems highlighting concepts needed for OOD generalization that may challenge neural policies but align with programmatic representations, such as tasks requiring algorithmic constructs like stacks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
