在离线强化学习中整合领域知识以处理有限数据
Integrating Domain Knowledge for handling Limited Data in Offline RL
AI总结：
本文针对离线强化学习在有限且状态缺失数据下性能下降的问题，提出基于领域知识的自适应正则化方法，以抑制稀疏和未观测状态下的错误动作，实验显示平均性能提升至少27%。
摘要：
离线强化学习（Offline Reinforcement Learning, RL）由于能够从静态数据集中学习，成为现实世界应用中一条引人注目的途径。然而，当面对局限于状态空间内特定区域的有限数据时，最先进的离线RL算法表现欠佳。性能下降归因于离线RL算法无法为罕见或未见过的观测学习合适的动作。本文提出了一种新颖的基于领域知识的正则化技术，并自适应地细化初始领域知识，以在部分状态缺失的有限数据中显著提升性能。关键见解在于，正则化项能够减轻由领域知识覆盖的稀疏样本和未观测状态下的错误动作。在标准离散环境数据集上的实证评估表明，与在有限数据上运行的现有离线RL算法相比，平均性能至少提升了27%。
英文摘要：
With the ability to learn from static datasets, Offline Reinforcement Learning (RL) emerges as a compelling avenue for real-world applications. However, state-of-the-art offline RL algorithms perform sub-optimally when confronted with limited data confined to specific regions within the state space. The performance degradation is attributed to the inability of offline RL algorithms to learn appropriate actions for rare or unseen observations. This paper proposes a novel domain knowledge-based regularization technique and adaptively refines the initial domain knowledge to considerably boost performance in limited data with partially omitted states. The key insight is that the regularization term mitigates erroneous actions for sparse samples and unobserved states covered by domain knowledge. Empirical evaluations on standard discrete environment datasets demonstrate a substantial average performance increase of at least 27% compared to existing offline RL algorithms operating on limited data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
