表示学习实现可扩展的多任务深度强化学习
Representation Learning Enables Scalable Multitask Deep Reinforcement Learning
AI总结：
本文提出一种结合预测性表示学习与高容量值函数近似的无模型算法MR.Q，在无需规划的情况下，在多任务连续控制任务中超越基于世界模型的方法和多种深度强化学习基线，并显著降低计算开销。
摘要：
将强化学习扩展到多样化的多任务设置仍然是一个核心挑战。虽然基于模型的强化学习的最新进展取得了强劲的性能，但它们依赖于规划和复杂的训练流程，使得不清楚哪些组件对可扩展性至关重要。我们重新审视这个问题，并认为可扩展多任务强化学习的主要驱动力不是基于模型的控制，而是\emph{表示学习}。特别地，我们表明，将预测性的、基于模型的表示与高容量值函数逼近相结合，即使没有规划，也足以实现强劲的性能。我们评估了一种简单的无模型算法MR.Q，将辅助预测目标与可扩展的actor-critic架构相结合。这种方法在多样化的多任务连续控制任务套件中优于最近基于世界模型的方法和一系列深度强化学习基线，同时显著降低了计算开销并提高了实际时间效率。我们观察到随着模型容量的增加而持续改进，并通过消融实验表明预测性表示学习对性能至关重要。
英文摘要：
Scaling reinforcement learning (RL) to diverse multitask settings remains a central challenge. While recent advances in model-based RL achieve strong performance, they rely on planning and complex training pipelines, making it unclear which components are essential for scalability. We revisit this question and argue that the primary driver of scalable multitask RL is not model-based control, but \emph{representation learning}. In particular, we show that combining predictive, model-based representations with high-capacity value function approximation is sufficient to achieve strong performance, even without planning. We evaluate a simple model-free algorithm, MR.Q, coupled with auxiliary predictive objectives into a scalable actor-critic architecture. This approach outperforms a recent world-model-based method and a range of deep RL baselines across a diverse suite of multitask continuous control tasks, while significantly reducing computational overhead and improving wall-clock efficiency. We observe consistent improvements with increased model capacity and show through ablations that predictive representation learning is critical for performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
