具有智能反射器的毫米波通信：性能优化和分布式强化学习
Millimeter Wave Communications with an Intelligent Reflector: Performance Optimization and Distributional Reinforcement Learning
摘要：
提出了一种新的毫米波基站下行多用户通信优化框架，该框架由可重构智能反射器（IR）辅助。特别地，开发了一种信道估计方法来实时测量信道状态信息（CSI）。首先，对于理想的CSI场景，通过迭代方法联合优化BS的预编码传输和IR的反射系数，以便最大化朝向多个用户的下行链路速率之和。接下来，在不完美的CSI场景中，提出了一种分布式强化学习（DRL）方法来学习最佳IR反射并最大化下行链路容量的期望。为了对下行链路传输速率的概率分布进行建模，提出了一种基于分位数回归（QR）的学习算法，并证明了所提出的QR-DRL方法收敛于一个稳定的下行链路传输速率分布。仿真结果表明，在信道状态信息无差错的情况下，与固定红外反射方案和直接传输方案相比，该方法的下行链路和速率分别提高了30%和2倍以上。仿真结果还表明，通过部署更多的IR元素，可以显着提高下行链路的和速率。然而，随着IR分量的数量增加，信道估计需要更多的时间，并且IR辅助传输速率的增加的斜率将变得更小。此外，在有限的CSI知识下，仿真结果表明，所提出的QR-DRL方法，学习下行链路速率的全分布，产生更好的预测精度，并提高了10%的下行链路速率在线部署，相比Q学习基线。
英文摘要：
In this paper, a novel framework is proposed to optimize the downlink multi-user communication of a millimeter wave base station, which is assisted by a reconfigurable intelligent reflector (IR). In particular, a channel estimation approach is developed to measure the channel state information (CSI) in real-time. First, for a perfect CSI scenario, the precoding transmission of the BS and the reflection coefficient of the IR are jointly optimized, via an iterative approach, so as to maximize the sum of downlink rates towards multiple users. Next, in the imperfect CSI scenario, a distributional reinforcement learning (DRL) approach is proposed to learn the optimal IR reflection and maximize the expectation of downlink capacity. In order to model the transmission rate's probability distribution, a learning algorithm, based on quantile regression (QR), is developed, and the proposed QR-DRL method is proved to converge to a stable distribution of downlink transmission rate. Simulation results show that, in the error-free CSI scenario, the proposed approach yields over 30% and 2-fold increase in the downlink sum-rate, compared with a fixed IR reflection scheme and direct transmission scheme, respectively. Simulation results also show that by deploying more IR elements, the downlink sum-rate can be significantly improved. However, as the number of IR components increases, more time is required for channel estimation, and the slope of increase in the IR-aided transmission rate will become smaller. Furthermore, under limited knowledge of CSI, simulation results show that the proposed QR-DRL method, which learns a full distribution of the downlink rate, yields a better prediction accuracy and improves the downlink rate by 10% for online deployments, compared with a Q-learning baseline.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
