MinMaxMin $Q$ -学习
MinMaxMin $Q$-learning
摘要：
MinMaxMin $Q$ -学习是一种新的乐观Actor-Critic算法，它解决了保守RL算法中固有的高估偏差（$Q$ -估计高估了真实的$Q$ -值）问题。其核心公式依赖于$Q$ -网络之间的不一致性，以min-batch MaxMin $Q$ -网络距离的形式添加到$Q$ -目标中，并用作优先级经验重放采样规则。我们在TD 3和TD 7之上实现MinMaxMin，并在流行的MuJoCo和Bullet环境中对最先进的连续空间算法DDPG、TD 3和TD 7进行严格测试。结果显示，在所有测试任务中，MinMaxMin的性能都比DDPG、TD 3和TD 7有一致的提高。
英文摘要：
MinMaxMin $Q$-learning is a novel optimistic Actor-Critic algorithm that addresses the problem of overestimation bias ($Q$-estimations are overestimating the real $Q$-values) inherent in conservative RL algorithms. Its core formula relies on the disagreement among $Q$-networks in the form of the min-batch MaxMin $Q$-networks distance which is added to the $Q$-target and used as the priority experience replay sampling-rule. We implement MinMaxMin on top of TD3 and TD7, subjecting it to rigorous testing against state-of-the-art continuous-space algorithms-DDPG, TD3, and TD7-across popular MuJoCo and Bullet environments. The results show a consistent performance improvement of MinMaxMin over DDPG, TD3, and TD7 across all tested tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
