The hyper-parameter optimization (HPO) process is imperative for finding the best-performing Convolutional Neural Networks (CNNs). The automation process of HPO is characterized by its sizable computational footprint and its lack of transparency; both important factors in a resource-constrained Internet of Things (IoT) environment. In this paper, we address these problems by proposing a novel approach that combines transformer architecture and actor-critic Reinforcement Learning (RL) model, TRL-HPO, equipped with multi-headed attention that enables parallelization and progressive generation of layers. These assumptions are founded empirically by evaluating TRL-HPO on the MNIST dataset and comparing it with state-of-the-art approaches that build CNN models from scratch. The results show that TRL-HPO outperforms the classification results of these approaches by 6.8% within the same time frame, demonstrating the efficiency of TRL-HPO for the HPO process. The analysis of the results identifies the main culprit for performance degradation attributed to stacking fully connected layers. This paper identifies new avenues for improving RL-based HPO processes in resource-constrained environments.
Enhancing IoT Intelligence: A Transformer-based Reinforcement Learning Methodology
Aqaba University of Technology(阿卡巴理工大学);Concordia University(康考迪亚大学);Khalifa University(哈利法大学);The University of Jordan(约旦大学);Applied Science Private University(应用科学私立大学);King Fahd University of Petroleum & Minerals(法赫德国王石油与矿业大学)
Attention-based Adversarial Robust Distillation in Radio Signal Classifications for Low-Power IoT Devices
Wolfson School of Mechanical, Electrical and Manufacturing Engineering, Loughborough University(洛辛姆机械、电气与制造工程学院,劳德斯大学);School of Engineering, University of Warwick(工程学院,沃里克大学);School of Electronic Engineering, Xidian University(电子工程学院,西安电子科技大学);Department of Electrical Engineering, King Saud University(电气工程系,沙特国王大学);Department of Informatics, Bioengineering, Robotics, and Systems Engineering, University of Genova(信息学、生物工程、机器人学和系统工程系,热那亚大学)
Enhancing Automatic Modulation Recognition for IoT Applications Using Transformers
The University of North Carolina at Charlotte(北卡罗来纳大学夏洛特分校);Kostas Research Institute at Northeastern University(东北大学科斯塔斯研究所);Northeastern University(东北大学)
Reliability-Calibrated Edge-IoT Early Fault Warning for Rotating Machinery with a Physics-Guided Tiny-Mamba Transformer
Great Bay University(大亚湾大学);Huizhou University(惠州大学);National University of Singapore(国立新加坡大学);Shenzhen University(深圳大学);James Cook University(詹姆斯库克大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
UBiGTLoc: A Unified BiLSTM-Graph Transformer Localization Framework for IoT Sensor Networks
Department of Computer Science, The University of Vermont(计算机科学系,佛蒙特大学);Department of Computer Science and Engineering, American University of Sharjah(计算机科学与工程系,阿曼大学沙迦分校);Department of Electrical Engineering, The University of Vermont(电气工程系,佛蒙特大学);Department of Mechanical Engineering, The University of Vermont(机械工程系,佛蒙特大学)
Easz: An Agile Transformer-based Image Compression Framework for Resource-constrained IoTs
Mohamed bin Zayed University of Artificial Intelligence(Mohamed bin Zayed大学人工智能学院);The Hang Seng University of Hong Kong(恒生大学);City University of Hong Kong(城市大学);The Chinese University of Hong Kong(香港中文大学);National Taiwan University(台湾国立台湾大学)
Hyper-STTN: Hypergraph Augmented Spatial-Temporal Transformer Network for Trajectory Prediction
School of Applied and Creative Computing, Purdue University(普渡大学应用与创意计算学院);Department of Computer Science and Department of Intelligent Systems Engineering, Indiana University Bloomington(印第安纳大学布卢明顿分校计算机科学系和智能系统工程系);Department of Applied Artificial Intelligence, Sungkyunkwan University(成均馆大学应用人工智能系)
TFormer: A Transmission-Friendly ViT Model for IoT Devices
Sun Yat-sen University(中山大学);Beijing Jiaotong University(北京交通大学);Meta AI;New York University(纽约大学);Michigan State University(密歇根州立大学);Beijing University of Posts and Telecommunications(北京邮电大学);Swinburne University of Technology(斯威本科技大学)