arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.08045cs.NI
收藏

面向数字孪生同步优化的持续强化学习

Continual Reinforcement Learning for Digital Twin Synchronization Optimization

Haonan Tong, Mingzhe Chen, Jun Zhao, Ye Hu, Zhaohui Yang, Yuchen Liu, Changchuan Yin

AI总结:

针对动态无线网络中数字孪生同步需消耗大量无线资源的问题,提出基于持续强化学习的资源分配方案,在相同资源块数量下可将物理与虚拟状态的归一化均方根误差降低最多55.2%。

摘要:

本文研究动态无线网络中面向数字孪生(DT)同步优化的自适应资源分配方案。在我们考虑的模型中,基站(BS)持续从无线设备收集工厂物理对象状态数据,以构建用于工厂事件分析的实时虚拟DT系统。由于数据传输是持续进行的,维持DT同步需要消耗大量无线资源。为解决这一问题,本文选择一部分设备传输其传感数据,并对资源块(RB)分配进行优化。该问题被建模为一个带约束的马尔可夫过程(CMDP)问题,目标是最小化物理系统与虚拟系统之间的长期失配。为求解该CMDP,我们首先将问题转化为一个对偶问题,以细化RB约束对设备调度策略的影响;随后提出一种持续强化学习(CRL)算法来求解该对偶问题。CRL算法基于历史经验学习稳定策略,以快速适应物理状态和网络容量的动态变化。仿真结果表明,在使用与传统方法相同数量RB的情况下,CRL能够快速适应网络容量变化,并且可将物理状态与虚拟状态之间的归一化均方根误差(NRMSE)降低高达55.2%。

英文摘要:

This article investigates the adaptive resource allocation scheme for digital twin (DT) synchronization optimization over dynamic wireless networks. In our considered model, a base station (BS) continuously collects factory physical object state data from wireless devices to build a real-time virtual DT system for factory event analysis. Due to continuous data transmission, maintaining DT synchronization must use extensive wireless resources. To address this issue, a subset of devices is selected to transmit their sensing data, and resource block (RB) allocation is optimized. This problem is formulated as a constrained Markov process (CMDP) problem that minimizes the long-term mismatch between the physical and virtual systems. To solve this CMDP, we first transform the problem into a dual problem that refines RB constraint impacts on device scheduling strategies. We then propose a continual reinforcement learning (CRL) algorithm to solve the dual problem. The CRL algorithm learns a stable policy across historical experiences for quick adaptation to dynamics in physical states and network capacity. Simulation results show that the CRL can adapt quickly to network capacity changes and reduce normalized root mean square error (NRMSE) between physical and virtual states by up to 55.2%, using the same RB number as traditional methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑