面向大规模MIMO预编码的量化Transformer与自动分辨率调优
Quantized Transformers for Massive MIMO Precoding with Automatic Resolution Tuning
- Polytechnique Montréal（蒙特利尔理工学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对大规模MIMO预编码，提出可微分的精度学习与神经架构搜索方法，联合优化量化参数，实现紧凑Transformer预编码器，能耗效率较经典算法提升最高288倍。
AI中文摘要：
基于深度学习的预编码器，尤其是基于Transformer架构的预编码器，为大规模MIMO系统提供了卓越的频谱效率，但可能带来较高的计算成本。虽然现有的针对卷积神经网络的混合精度量化研究建立了重要的能耗基线，但传统的离散搜索方法无法扩展到现代Transformer的大参数空间。为弥合这一差距，我们将一种可微分的精度学习技术应用于大规模MIMO预编码。该方法在单一训练循环中自主且联合地优化权重、激活、量化步长以及逐层位宽，直接以能耗效率为优化目标。我们进一步扩展了该方法，将神经架构搜索纳入不同规模的Transformer模型，并评估随机初始化与浮点训练初始化对分辨率调优的影响。最终，我们证明了所提出的训练方法能够部署高度紧凑的Transformer预编码器，在密集市中心环境中，与经典的加权最小均方误差算法相比，在相同总速率性能下，能耗效率最高可提升288倍。
英文摘要：
Deep learning precoders, particularly those based on Transformer architectures, offer superior spectral efficiency for Massive MIMO systems but may incur high computational cost. While existing mixed-precision quantization studies on convolutional neural networks establish important energy baselines, traditional discrete search methods fail to scale to the large parameter spaces of modern Transformers. To bridge this gap, we apply a differentiable precision learning technique to massive MIMO precoding. This approach autonomously and jointly optimizes weights, activations, quantization step sizes, and layer-wise bit-widths in a single training loop, directly optimizing for energy efficiency. We further expand this approach by incorporating Neural Architecture Search across various Transformer model sizes and evaluating the impact of random versus floating-point trained initializations on resolution tuning. Ultimately, we demonstrate that the proposed training method enables the deployment of highly compact Transformer precoders, improving energy efficiency by up to 288\(\times\) compared to the classical Weighted Minimum Mean Square Error algorithm at equal sum-rate performance in a dense downtown environment.
