将我情境化——强化学习中情境的理由
Contextualize Me -- The Case for Context in Reinforcement Learning
AI总结：
本文提出情境强化学习框架与基准库 CARL，通过情境扩展环境实证说明情境信息对零样本泛化和通用智能体研究的关键作用。
摘要：
尽管强化学习（Reinforcement Learning, RL）在解决日益复杂的问题方面已取得长足进步，但许多算法即使面对微小的环境变化仍然十分脆弱。情境强化学习（Contextual Reinforcement Learning, cRL）提供了一个以有原则的方式对此类变化进行建模的框架，从而实现灵活、精确且可解释的任务规范与生成。我们的目标是展示 cRL 框架如何通过有意义的基准以及关于泛化任务的结构化推理，促进改善 RL 中的零样本泛化。我们证实了如下洞见：与部分可观测性的其他相关领域一样，cRL 中的最优行为需要情境信息。为了在 cRL 框架中对其进行实证验证，我们提供了常见 RL 环境的多种情境扩展版本。它们构成首个基准库 CARL 的一部分；CARL 基于流行基准的 cRL 扩展、专为泛化而设计，我们提出将其作为进一步研究通用智能体的测试平台。我们表明，在情境化设定下，即使是简单的 RL 环境也会变得具有挑战性——而且朴素解决方案不足以跨复杂情境空间实现泛化。
英文摘要：
While Reinforcement Learning ( RL) has made great strides towards solving increasingly complicated problems, many algorithms are still brittle to even slight environmental changes. Contextual Reinforcement Learning (cRL) provides a framework to model such changes in a principled manner, thereby enabling flexible, precise and interpretable task specification and generation. Our goal is to show how the framework of cRL contributes to improving zero-shot generalization in RL through meaningful benchmarks and structured reasoning about generalization tasks. We confirm the insight that optimal behavior in cRL requires context information, as in other related areas of partial observability. To empirically validate this in the cRL framework, we provide various context-extended versions of common RL environments. They are part of the first benchmark library, CARL, designed for generalization based on cRL extensions of popular benchmarks, which we propose as a testbed to further study general agents. We show that in the contextual setting, even simple RL environments become challenging - and that naive solutions are not enough to generalize across complex context spaces.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
