基于Transformer的Actor-Critic用于高效的 freshness-aware 资源分配
Transformer Actor-Critic for Efficient Freshness-Aware Resource Allocation
AI总结：
本文提出基于Transformer的Actor-Critic框架，用于高效实现 freshness-aware 资源分配，通过注意力机制提升策略性能和可扩展性。
摘要：
新兴应用如自动驾驶和工业自动化要求超可靠低时延通信(URLLC)，其中保持信息新鲜及时至关重要。此类系统的关键性能指标是信息年龄(AoI)。本文针对多用户上行无线网络中使用非正交多址接入(NOMA)的AoI最小化问题，其中用户将任务卸载至基站。系统必须处理用户在任务大小、AoI阈值和惩罚敏感性上的异质性，同时遵守NOMA对用户调度的约束。我们提出了一种基于近端策略优化(PPO)的深度强化学习(DRL)框架，并辅以Transformer编码器。注意力机制使智能体能够聚焦于关键用户状态并捕捉用户间依赖关系，从而提升策略性能和可扩展性。大量仿真显示，我们的方法在平均AoI方面优于基线方法。我们还分析了训练过程中注意力权重的演变，并发现模型逐渐学会优先考虑高重要性用户。注意力图揭示了有意义的结构：早期阶段的策略表现出均匀的注意力，而后期阶段则表现出与用户优先级和NOMA约束对齐的聚焦模式。这些结果突显了注意力驱动的DRL在下一代无线系统中实现智能、优先级感知资源分配的潜力。
英文摘要：
Emerging applications such as autonomous driving and industrial automation demand ultra-reliable and low-latency communication (URLLC), where maintaining fresh and timely information is critical. A key performance metric in such systems is the age of information (AoI). This paper addresses AoI minimization in a multi-user uplink wireless network using non-orthogonal multiple access (NOMA), where users offload tasks to a base station. The system must handle user heterogeneity in task sizes, AoI thresholds, and penalty sensitivities, while adhering to NOMA constraints on user scheduling. We propose a deep reinforcement learning (DRL) framework based on proximal policy optimization (PPO), enhanced with a Transformer encoder. The attention mechanism allows the agent to focus on critical user states and capture inter-user dependencies, improving policy performance and scalability. Extensive simulations show that our method reduces average AoI compared to baselines. We also analyze the evolution of attention weights during training and observe that the model progressively learns to prioritize high-importance users. Attention maps reveal meaningful structure: early-stage policies exhibit uniform attention, while later stages show focused patterns aligned with user priority and NOMA constraints. These results highlight the promise of attention-driven DRL for intelligent, priority-aware resource allocation in next-generation wireless systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
