基于潜在类比的组合转导用于离线目标条件强化学习
Compositional Transduction with Latent Analogies for Offline Goal-Conditioned Reinforcement Learning
AI总结：
本文提出了一种基于潜在类比的组合转导方法，用于解决离线目标条件强化学习中面对新情境时的目标泛化问题，通过引入新的类比表示方法，提升了在不同情境下的目标达到能力。
摘要：
组合泛化对于在新颖的上下文变化中达到未见过的目标在离线目标条件强化学习（GCRL）中至关重要，其中必须从有限的数据中学习一个通用的目标达到智能体。大多数先前的方法通过在时间连续的片段上进行轨迹缝合来实现这一点，这限制了在不同上下文中组合行为的能力。为了克服这一限制，我们正式将类比转导定义为通过组合任务内固有的类比与给定的上下文来合成新的计划，并提出了一个针对此目的的新型类比表示。基于我们的理论，这种类比表示捕捉了在最优任务执行下发生变化的内容，对上下文变化保持不变，并且足以实现最优的目标达到。我们进一步认为，对未见过的类比-上下文对的泛化是类比转导中的实际障碍，并引入了一种新的离线GCRL方法，使类比转导能够超越已见过的对到未见的组合。我们通过在OGBench操纵环境中实验证明了我们方法的有效性，显著优于不进行类比转导的先前方法。项目页面：https://rllab-snu.github.io/projects/CTA/
英文摘要：
Compositional generalization is essential for reaching unseen goals under novel contextual variations in offline goal-conditioned reinforcement learning (GCRL), where a generalist goal-reaching agent must be learned from limited data. Most prior approaches pursue this via trajectory stitching over temporally contiguous segments, which limits composing behaviors across varying contexts. To overcome this limitation, we formalize analogy transduction as synthesizing new plans by composing task-endogenous analogies with given contexts and propose a novel analogy representation tailored for it. Grounded in our theory, this analogy representation captures what changes under optimal task execution, remains invariant to contextual variations, and is sufficient for optimal goal reaching. We further contend that generalization to unseen analogy-context pairs is a practical obstacle in analogy transduction, and introduce a new approach for offline GCRL that enables analogy transduction beyond seen pairs to unseen combinations. We empirically demonstrate the effectiveness of our approach on OGBench manipulation environments, substantially outperforming prior methods that do not perform analogy transduction. Project page: https://rllab-snu.github.io/projects/CTA/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
