对比扩散器：通过对比学习规划朝向高回报状态
Contrastive Diffuser: Planning Towards High Return States via Contrastive Learning
AI总结：
针对离线强化学习中低回报轨迹占比高导致策略学习困难的问题，本文提出对比扩散器CDiffuser，通过对比机制将状态分为高回报正样本和低回报负样本，引导智能体远离低回报区域，并在14个D4RL基准上验证了有效性。
摘要：
离线强化学习（RL）的性能对离线数据集中高回报轨迹的比例很敏感。然而，在许多仿真环境和现实场景中，低回报轨迹的比例很大，而不是高回报轨迹，这使得学习一个高效的策略具有挑战性。在本文中，我们提出了一种称为对比扩散器（CDiffuser）的方法，以充分利用低回报轨迹并提高离线RL算法的性能。具体来说，CDiffuser将离线数据集中轨迹的状态分为高回报状态和低回报状态，并将它们分别视为正样本和负样本。然后，它设计了一种对比机制，将智能体的轨迹拉向高回报状态，并将其推离低回报状态。通过对比机制，低回报轨迹可以作为策略学习的负样本，引导智能体避开与低回报相关的区域，并取得更好的性能。在14个常用的D4RL基准上的实验证明了我们提出的方法的有效性。我们的代码公开在\url{https://anonymous.4open.science/r/CDiffuser}。
英文摘要：
The performance of offline reinforcement learning (RL) is sensitive to the proportion of high-return trajectories in the offline dataset. However, in many simulation environments and real-world scenarios, there are large ratios of low-return trajectories rather than high-return trajectories, which makes learning an efficient policy challenging. In this paper, we propose a method called Contrastive Diffuser (CDiffuser) to make full use of low-return trajectories and improve the performance of offline RL algorithms. Specifically, CDiffuser groups the states of trajectories in the offline dataset into high-return states and low-return states and treats them as positive and negative samples correspondingly. Then, it designs a contrastive mechanism to pull the trajectory of an agent toward high-return states and push them away from low-return states. Through the contrast mechanism, trajectories with low returns can serve as negative examples for policy learning, guiding the agent to avoid areas associated with low returns and achieve better performance. Experiments on 14 commonly used D4RL benchmarks demonstrate the effectiveness of our proposed method. Our code is publicly available at \url{https://anonymous.4open.science/r/CDiffuser}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
