arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.05199eess.SPcs.ITmath.IT
收藏

用于无线通信的Decision Transformers:资源管理的新范式

Decision Transformers for Wireless Communications: A New Paradigm of Resource Management

Jie Zhang, Jun Li, Long Shi, Zhe Wang, Shi Jin, Wen Chen, H. Vincent Poor

AI总结:

针对DRL在无线资源管理中样本效率低、泛化差且收敛慢的问题,提出基于Decision Transformer的云边协同自适应决策架构,在智能反射面与无人机场景下实现3至6倍收敛加速及更优性能。

摘要:

随着下一代移动系统的演进,人工智能(AI)有望与无线通信深度融合,以实现可变环境下的资源管理。特别是,深度强化学习(DRL)是解决资源分配随机优化问题的重要工具。然而,一旦状态和动作空间发生变化,DRL就必须从头开始每个新的训练过程,导致样本效率低和泛化能力差。此外,每个DRL训练过程可能需要大量的epochs才能收敛,这对于时间敏感的场景是不可接受的。在本文中,我们采用一种替代的AI技术,即Decision Transformer(DT),并提出了一种基于DT的无线资源管理自适应决策架构。该架构通过在云端构建预训练模型,然后在边缘微调个性化模型进行创新。通过利用在离线数据集上学习到的DT模型的能力,与DRL相比,所提出的架构有望在不同状态和动作空间的新场景中,以更少的训练epochs实现快速收敛并获得更高的性能。然后,我们为两个典型的通信场景设计了DT框架:智能反射面辅助通信和无人机辅助移动边缘计算。仿真表明,与经典的DRL方法(即近端策略优化)相比,所提出的DT框架在收敛速度上实现了3至6倍的加速,并获得了更好的性能。

英文摘要:

As the next generation of mobile systems evolves, artificial intelligence (AI) is expected to deeply integrate with wireless communications for resource management in variable environments. In particular, deep reinforcement learning (DRL) is an important tool for addressing stochastic optimization issues of resource allocation. However, DRL has to start each new training process from the beginning once the state and action spaces change, causing low sample efficiency and poor generalization ability. Moreover, each DRL training process may take a large number of epochs to converge, which is unacceptable for time-sensitive scenarios. In this paper, we adopt an alternative AI technology, namely, Decision Transformer (DT), and propose a DT-based adaptive decision architecture for wireless resource management. This architecture innovates through constructing pre-trained models in the cloud and then fine-tuning personalized models at the edges. By leveraging the power of DT models learned over offline datasets, the proposed architecture is expected to achieve rapid convergence with many fewer training epochs and higher performance in new scenarios with different state and action spaces, compared with DRL. We then design DT frameworks for two typical communication scenarios: intelligent reflecting surfaces-aided communications and unmanned aerial vehicle-aided mobile edge computing. Simulations demonstrate that the proposed DT frameworks achieve over $3$-$6$ times speedup in convergence and better performance relative to the classic DRL method, namely, proximal policy optimization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑