DAGR：通过差异感知目标交叉注意力实现的状态条件目标表示

中文摘要 AI 辅助

目标条件强化学习取决于目标的编码方式。对比、度量、时间距离和信息理论编码器在目标上有所不同，但都有一个共同特点，即都不考虑当前状态。这种与状态无关的嵌入无法标记目标中仍需采取行动的部分，策略必须通过反转两个编码器来恢复该线索。我们提出了DAGR，它通过多尺度门控交叉注意力将任何后期融合编码器的静态嵌入细化为状态条件嵌入。近恒等门控残差保留了基础表示。差异感知目标交叉注意力然后使用每个令牌的状态-目标差异图来偏向注意力分数。在OGBench上，DAGR改善了导航。我们的消融实验将收益追溯到门控残差，而不是命名该方法的差异偏差。在操纵和拼图任务中，它与基础模型匹配或低于基础模型。DAGR是一种结构化细化，而不是普遍改进。

英文摘要

Goal-conditioned reinforcement learning hinges on how the goal is encoded. Contrastive, metric, temporal-distance and information-theoretic encoders disagree on the objective. They agree on one thing. None of them sees the current state, so the embedding cannot mark which part of the goal still needs action, and the policy must recover that cue by inverting both encoders. We propose DAGR, which refines the static embedding of any late-fusion encoder into a state-conditioned one through multi-scale gated cross-attention. A gated residual holds the refinement near the base, and a difference-aware attention rule biases the scores by a per-token state-goal mismatch. A single condition decides what such a refinement can guarantee, namely whether the block returns its input at closed gates. We prove that the usual post-norm placement violates it, measure the consequence on frozen checkpoints, and recover part of the resulting loss by restoring the condition. On OGBench DAGR improves navigation and matches or trails the base elsewhere. Our ablations trace the gain to the gated residual rather than to the difference bias that names the method. Code is available at https://github.com/leixingxing1/DAGR

发表机构

• Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所）
• Department of Electrical Engineering and Automation, Aalto University（阿尔托大学电气工程与自动化系）
• School of Engineering, Westlake University（西湖大学工学院）

机构由 AI 辅助整理，请以论文原文为准。