面向无线资源管理的混合离线-在线多智能体决策变换器
Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management
浏览论文内容
中文总结 AI 辅助
本文提出混合离线-在线多智能体决策变换器框架，通过分布式训练实现无线资源管理，性能可媲美集中式方法，低质量数据集预训练后在线微调仍能提升效果。
中文摘要 AI 辅助
本文开发了一种基于决策变换器的混合离线-在线多智能体强化学习框架。首先通过对现有策略生成的轨迹进行监督序列建模，对策略进行离线预训练，提供安全且样本高效的初始化；随后使用包含评论家引导梯度的混合目标函数进行在线微调，实现超出离线策略的性能提升。为促进稳定的离线到在线迁移及有效的多智能体协同，该框架引入了回报加权采样、基于邻居动作的评论家以及与邻居相关的探索。该方法完全分布式，训练和执行均仅依赖局部观测及相邻智能体间的有限信息交换。在两种场景下针对动态流量到达进行评估：（i）联合调度与功率分配，（ii）协调波束成形，结果表明所提方法的服务质量（QoS）性能可与集中式方法媲美。此外，在低质量数据集上预训练时，在线微调也被观察到能超越初始离线策略。这些结果为无线资源管理展示了一种有前景的基于学习的替代方案。
英文摘要
This paper develops a hybrid offline-online multi-agent reinforcement learning framework based on decision transformers. The policy is first pretrained offline via supervised sequence modeling of trajectories generated by existing policies, providing a safe and sample-efficient initialization. It is then fine-tuned online using a hybrid objective that incorporates critic-guided gradients, enabling performance improvements beyond the offline policy. To facilitate stable offline-to-online transfer and effective multi-agent coordination, the framework incorporates return-weighted sampling, a critic conditioned on neighbors' actions, and neighborhood-correlated exploration. The approach is fully distributed: both training and execution rely only on local observations and limited information exchange among neighboring agents. Evaluations with dynamic traffic arrivals in two settings: (i) joint scheduling and power allocation and (ii) coordinated beamforming, show that the proposed method achieves quality-of-service (QoS) performance comparable to centralized methods. Moreover, when pretrained on lower-quality datasets, online fine-tuning is also observed to surpass the initial offline policy. These results demonstrate a promising learning-based alternative for wireless resource management.
发表机构
- Northwestern University（西北大学）
- University of Virginia（弗吉尼亚大学）
机构由 AI 辅助整理，请以论文原文为准。
