InnateCoder：基于基础模型学习程序化选项
InnateCoder: Learning Programmatic Options with Foundation Models
AI总结：
针对强化学习从零开始学习技能效率低的问题，提出InnateCoder系统，零样本利用基础模型的人类知识学习程序化选项并组合成复杂策略，在MicroRTS和Karel the Robot上验证了更高的样本效率。
摘要：
在迁移学习场景之外，强化学习智能体均从零开始启动学习过程。因此，这类智能体即便要学习解决问题所需的最基础技能，也必须经历漫长的过程。本文提出InnateCoder系统，该系统利用基础模型中编码的人类知识，提供以时间扩展动作（即选项）形式编码“先天技能”的程序化策略。与现有的选项学习方法不同，InnateCoder在零样本设置下从基础模型编码的通用人类知识中学习选项，而非从智能体与环境交互获得的知识中学习。随后，InnateCoder通过将编码这些选项的程序组合为更大、更复杂的程序，来搜索程序化策略。我们假设，InnateCoder的选项学习与使用方式能够提升现有程序化策略学习方法的采样效率。在MicroRTS和Karel the Robot上的实验结果支持了这一假设，结果表明InnateCoder的样本效率高于不使用选项或从经验中学习选项的系统版本。
英文摘要：
Outside of transfer learning settings, reinforcement learning agents start their learning process from a clean slate. As a result, such agents have to go through a slow process to learn even the most obvious skills required to solve a problem. In this paper, we present InnateCoder, a system that leverages human knowledge encoded in foundation models to provide programmatic policies that encode "innate skills" in the form of temporally extended actions, or options. In contrast to existing approaches to learning options, InnateCoder learns them from the general human knowledge encoded in foundation models in a zero-shot setting, and not from the knowledge the agent gains by interacting with the environment. Then, InnateCoder searches for a programmatic policy by combining the programs encoding these options into larger and more complex programs. We hypothesized that InnateCoder's way of learning and using options could improve the sampling efficiency of current methods for learning programmatic policies. Empirical results in MicroRTS and Karel the Robot support our hypothesis, since they show that InnateCoder is more sample efficient than versions of the system that do not use options or learn them from experience.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
