On the Comparison of Reinforcement Learning and Adaptive Control for Linear Systems under Packet Loss and Uncertainty
Northeastern University（东北大学）
Moh Kamalul Wafi
备注：
Extended and revised version of the paper published in the Proceedings of the SIAM Conference on Control and Its Applications (CT23). doi.org/10.1137/1.9781611977745
This paper presents a comparative study between Adaptive Quantized Control (AQC) and Deep Deterministic Policy Gradient (DDPG) reinforcement learning for uncertain linear systems with input quantization over communication channels subject to packet loss. The considered setting also includes dynamic switching from a nominal unstable system to a more unstable one during operation. The AQC is designed for unknown system dynamics using acknowledgment messages to compensate for packet losses, whereas the DDPG controller is trained using the nominal system model without acknowledgment messages. Numerical results show that the DDPG controller achieves faster transient responses and improved damping within its training environment. However, under model uncertainty, packet loss, and dynamic switching, the AQC consistently demonstrates superior robustness owing to its rigorous Lyapunov stability guarantees. These results highlight the trade-off between data-driven performance and model-based robustness, and provide insight into the applicability of reinforcement learning and adaptive control for networked uncertain systems.
Power Allocation for Delay Optimization in Device-to-Device Networks: A Graph Reinforcement Learning Approach
National Mobile Communications Research Laboratory, Southeast University（东南大学国家移动通信研究中心）；College of Electronics and Information Engineering, Shenzhen University（深圳大学电子与信息工程学院）；Department of Electrical and Computer Engineering, University of California, Santa Cruz（加州大学圣克鲁兹分校电子与计算机工程系）
All Models are Wrong, Knowing Where is Useful: On Model Uncertainty in Reinforcement Learning
German Federal Ministry of Research, Technology and Space (BMFTR)（德国联邦研究、技术和空间部）；Robotics Institute Germany (RIG)（德国机器人研究所）；Institute for Data Science in Mechanical Engineering, RWTH Aachen University（机械工程数据科学研究所，亚琛工业大学）；NHR Center NHR4CES at RWTH Aachen University（亚琛工业大学NHR4CES中心）
