基于状态空间模型的强化学习用于EXL-50U上的磁构型控制
State-Space Model-Enabled Reinforcement Learning for Magnetic Configuration Controlon EXL-50U
浏览论文内容
中文总结 AI 辅助
针对EXL-50U球形托卡马克的磁构型控制难题，本文提出基于状态空间模型的强化学习控制器，实现了等离子体电流与质心位置的稳定调节，维持放电达650毫秒，为磁控提供了新方向。
中文摘要 AI 辅助
等离子体电流（$I_p$）与质心位置（$R_c,Z_c$）的精确反馈控制对球形托卡马克（ST）等离子体的稳定运行至关重要。传统的比例-积分-微分（PID）控制器需要大量手动调参，且难以应对等离子体性能提升时出现的快速强耦合动力学过程。强化学习（RL）近来成为这类复杂磁控问题的有前景替代方案，但其在ST装置上的实际部署仍具挑战性。本文提出一种适用于EXL-50U ST的实用RL控制器，该控制器在刚性RZIP状态空间模型（SSM）内训练，可实现高效的离线策略学习。研究开发了一种轻量级等离子体位形重构器，用于在实时控制周期内从磁探针信号中估算$R_c,Z_c$。训练后的策略被无缝部署到EXL-50U等离子体控制系统，实现了$I_p$与$R_c,Z_c$的稳定调节，在RL控制下维持放电时长可达650毫秒。这些结果证明了模型辅助RL用于ST装置磁构型控制的可行性与实际潜力，为超越传统PID方案提供了有前景的方向。
英文摘要
Accurate feedback control of the plasma current ($I_p$) and centroid position $(R_c,Z_c)$ is essential for the stable operation of spherical torus (ST) plasmas. Conventional proportional-integral-derivative (PID) controllers require extensive manual tuning and struggle with the fast, strongly coupled dynamics that arise as plasma performance improves. Reinforcement learning (RL) has recently emerged as a promising alternative to such complex magnetic control problems, yet its practical deployment on ST devices remains challenging. This paper presents a practical RL controller for the EXL-50U ST, trained within a rigid RZIP state-space model (SSM) that enables efficient offline policy learning. A lightweight plasma position reconstructor is developed to estimate $(R_c,Z_c)$ from magnetic probe signals within the real-time control cycle. The trained policy is seamlessly deployed on the EXL-50U plasma control system, achieving stable regulation of $I_p$ and $(R_c,Z_c)$ and sustaining discharges up to 650 ms under RL control. These results demonstrate the feasibility and practical potential of model-informed RL for magnetic configuration control in ST devices, offering a promising direction beyond conventional PID-based schemes.
