在度量空间上基于目标的强化学习从次优数据中学习最优行为
Goal-Conditioned Reinforcement Learning from Sub-Optimal Data on Metric Spaces
AI总结：
MetricRL通过结合度量学习和加权模仿学习，在度量空间上有效学习最优行为，优于现有方法。
摘要：
我们研究了在稀疏奖励、可逆动作和确定性转移下，从次优数据集中学习最优行为的问题。为缓解分布偏移的影响，我们提出了MetricRL，一种结合度量学习进行价值函数近似和加权模仿学习进行策略估计的方法。MetricRL避免了保守或行为克隆约束，即使在严重次优情况下也能实现有效学习。我们引入了距离单调性作为将度量表示与最优性联系起来的关键性质，并设计了一个明确促进它的目标函数。实验表明，MetricRL在从次优离线数据中恢复近最优行为方面，始终优于先前最先进的基于目标的强化学习方法。
英文摘要：
We study the problem of learning optimal behavior from sub-optimal datasets for goal-conditioned offline reinforcement learning under sparse rewards, invertible actions and deterministic transitions. To mitigate the effects of \emph{distribution shift}, we propose MetricRL, a method that combines metric learning for value function approximation with weighted imitation learning for policy estimation. MetricRL avoids conservative or behavior-cloning constraints, enabling effective learning even in severely sub-optimal regimes. We introduce distance monotonicity as a key property linking metric representations to optimality and design an objective that explicitly promotes it. Empirically, MetricRL consistently outperforms prior state-of-the-art goal-conditioned RL methods in recovering near-optimal behavior from sub-optimal offline data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
