arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.00877cs.LGcs.AI
收藏

马尔可夫流策略——深度MC

Markov flow policy -- deep MC

Nitsan Soffair, Gilad Katz

AI总结:

本文提出马尔可夫流策略(MFP),利用非负神经网络流进行前向视图预测,解决折扣算法的训练-测试偏差问题,并在MuJoCo基准上取得显著性能提升。

摘要:

折扣算法由于依赖短期估计而经常遇到评估误差,这会妨碍它们处理简单的短期任务,并引入不期望的时间折扣(γ)。有趣的是,这些算法通常在测试时不应用折扣,我们将这一现象称为训练-测试偏差。为应对这些挑战,我们提出了马尔可夫流策略(Markov Flow Policy),它利用非负神经网络流来实现全面的前向视图预测。通过将其集成到TD7代码库中并使用MuJoCo基准进行评估,我们观察到显著的性能提升,使MFP成为平均奖励算法领域中一种直接、实用且易于实现的解决方案。

英文摘要:

Discounted algorithms often encounter evaluation errors due to their reliance on short-term estimations, which can impede their efficacy in addressing simple, short-term tasks and impose undesired temporal discounts (\(γ\)). Interestingly, these algorithms are often tested without applying a discount, a phenomenon we refer as the \textit{train-test bias}. In response to these challenges, we propose the Markov Flow Policy, which utilizes a non-negative neural network flow to enable comprehensive forward-view predictions. Through integration into the TD7 codebase and evaluation using the MuJoCo benchmark, we observe significant performance improvements, positioning MFP as a straightforward, practical, and easily implementable solution within the domain of average rewards algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑