局部动态感知领域自适应用于离线强化学习
Localized Dynamics-Aware Domain Adaption for Off-Dynamics Offline Reinforcement Learning
AI总结：
LoDADA通过局部动态不匹配改进离线强化学习，有效利用源数据并提升领域适应性能。
摘要：
离线强化学习（RL）旨在利用有限的目标数据和大量在不同转移动态下收集的源数据来学习目标领域的策略。现有方法通常通过全局状态空间或点wise数据过滤来处理动态不匹配；这些方法可能遗漏跨领域局部相似性或导致计算成本高。我们提出了局部动态感知领域自适应（LoDADA），该方法利用局部动态不匹配来更有效地重用源数据。LoDADA对源和目标数据集的转换进行聚类，并通过领域判别估计集群级动态差异。来自差异小的集群的源转换被保留，而差异大的则被过滤掉。这产生了一种细粒度且可扩展的数据选择策略，避免了过于粗略的全局假设和昂贵的每样本过滤。我们提供了理论见解，并在具有多样化全局和局部动态偏移的环境中进行了广泛的实验。结果表明，LoDADA在利用局部分布不匹配方面优于最先进的离线强化学习方法。
英文摘要：
Off-dynamics offline reinforcement learning (RL) aims to learn a policy for a target domain using limited target data and abundant source data collected under different transition dynamics. Existing methods typically address dynamics mismatch either globally over the state space or via pointwise data filtering; these approaches can miss localized cross-domain similarities or incur high computational cost. We propose Localized Dynamics-Aware Domain Adaptation (LoDADA), which exploits localized dynamics mismatch to better reuse source data. LoDADA clusters transitions from source and target datasets and estimates cluster-level dynamics discrepancy via domain discrimination. Source transitions from clusters with small discrepancy are retained, while those from clusters with large discrepancy are filtered out. This yields a fine-grained and scalable data selection strategy that avoids overly coarse global assumptions and expensive per-sample filtering. We provide theoretical insights and extensive experiments across environments with diverse global and local dynamics shifts. Results show that LoDADA consistently outperforms state-of-the-art off-dynamics offline RL methods by better leveraging localized distribution mismatch.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
