Accurate beam prediction is a key enabler for next-generation wireless communication systems. In this paper, we propose a multimodal large language model (LLM)-based beam prediction framework that effectively utilizes contextual information, provided by sensory data including RGB camera images and LiDAR point clouds. To effectively fuse heterogeneous modalities, we design specialized modality encoders together with a beam-guided attention masking mechanism and a high-frequency temporal alignment strategy, enabling robust cross-modal feature integration under dynamic environments. Furthermore, we construct a large-scale multimodal dataset for communication, named Multimodal-Wireless, which covers diverse weather and traffic conditions with high-fidelity ray-tracing labels. Extensive simulation results demonstrate that the proposed approach significantly reduces the reliance on oracle angle-of-departure knowledge and consistently outperforms state-of-the-art multimodal LLM-based beam prediction methods in terms of beam accuracy and communication performance, improving the average Top-1 accuracy to 80.8% and the average normalized gain to 89.1%.
Structure-Aware Multimodal LLM Framework for Trustworthy Near-Field Beam Prediction
School of Information Science and Engineering, Southeast University(信息科学与工程学院,东南大学);Institute of Communications Engineering, National Sun Yat-sen University(通讯工程学院,国立中山大学)
Knowledge Distillation for Lightweight Multimodal Sensing-Aided mmWave Beam Tracking
Centre for Wireless Communications (CWC), University of Oulu(奥卢大学无线通信中心);School of Electrical, Computer, and Energy Engineering, Arizona State University(亚利桑那州立大学电气、计算机与能源工程学院);School of Electrical Engineering & Computer Science, University of California, Irvine(加州大学欧文分校电气与计算机工程学院)