arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.08418cs.LG
收藏

具有共享表示的元强化学习实现能源系统中的快速适应

Meta-RL with Shared Representations Enables Fast Adaptation in Energy Systems

Université Sorbonne Paris Nord(索邦巴黎北大学);INSA Rouen(里昂大学 Rouen 分校);Université Lumière Lyon 2, Université Claude Bernard Lyon 1, ERIC(里昂大学二分校、里昂大学一分校、ERIC)

Théo Zangato, Aomar Osmani, Pegah Alizadeh

AI总结:

本文提出了一种具有共享表示的元强化学习框架,通过改进的知识转移和参数共享机制,实现能源系统中的快速适应与高效泛化。

摘要:

元强化学习通过在多任务和非平稳环境中实现快速策略适应和改进的泛化能力,解决了传统强化学习的关键限制。我们介绍了一种新的元强化学习框架,该框架结合了双层优化方案和一种专门设计的混合策略-批评者架构,以提高样本效率和跨任务适应性。为提高知识转移,我们元学习了一个共享状态特征提取器,该提取器在策略和批评者网络之间联合优化,提供高效的表示学习,并限制对个别任务或主导模式的过拟合。此外,我们提出了一种在外部循环和内部循环策略网络之间的参数共享机制,以减少冗余学习并加速任务重访时的适应。该方法在覆盖近十年时间与结构变化的现实世界建筑能源管理系统数据集上进行了验证,并提出了一个任务准备方法以促进泛化。实验表明,该方法在任务适应方面表现出色,比传统强化学习和元强化学习方法性能更好。

英文摘要:

Meta-Reinforcement Learning addresses the critical limitations of conventional Reinforcement Learning in multi-task and non-stationary environments by enabling fast policy adaptation and improved generalization. We introduce a novel Meta-RL framework that integrates a bi-level optimization scheme with a hybrid actor-critic architecture specially designed to enhance sample efficiency and inter-task adaptability. To improve knowledge transfer, we meta-learn a shared state feature extractor jointly optimized across actor and critic networks, providing efficient representation learning and limiting overfitting to individual tasks or dominant profiles. Additionally, we propose a parameter-sharing mechanism between the outer- and inner-loop actor networks, to reduce redundant learning and accelerate adaptation during task revisitation. The approach is validated on a real-world Building Energy Management Systems dataset covering nearly a decade of temporal and structural variability, for which we propose a task preparation method to promote generalization. Experiments demonstrate effective task adaptation and better performance compared to conventional RL and Meta-RL methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑