自组织双缓冲自适应聚类经验回放（SODACER）用于最优控制中的安全强化学习
Self-Organizing Dual-Buffer Adaptive Clustering Experience Replay (SODACER) for Safe Reinforcement Learning in Optimal Control
AI总结：
本文提出SODACER框架，结合自组织自适应聚类机制与控制屏障函数，实现非线性系统的安全高效控制，通过Sophia优化器提升收敛性和稳定性，验证了其在非线性HPV传播模型中的有效性。
摘要：
本文提出了一种名为自组织双缓冲自适应聚类经验回放（SODACER）的新型强化学习框架，旨在实现非线性系统的安全和可扩展最优控制。SODACER机制包含一个快速缓冲区用于快速适应近期经验，以及一个配备自组织自适应聚类机制的慢速缓冲区，以保持多样且非冗余的历史经验。自适应聚类机制动态修剪冗余样本，优化内存效率同时保留关键环境模式。该方法将SODACER与控制屏障函数（CBFs）结合，通过整个学习过程强制状态和输入约束以保证安全。为了增强收敛性和稳定性，框架结合Sophia优化器，实现自适应二阶梯度更新。所提出的SODACER-Sophia架构确保在动态、安全关键环境中可靠、有效和稳健的学习，为机器人、医疗保健和大规模系统优化提供通用解决方案。所提出的方法在具有多个控制输入和安全约束的非线性HPV传播模型上进行了验证。与随机和基于聚类的经验回放方法的比较评估显示，SODACER在收敛速度、样本效率和偏误-方差权衡方面均优于现有方法，同时保持安全系统轨迹，通过Friedman检验验证。
英文摘要：
This paper proposes a novel reinforcement learning framework, named Self-Organizing Dual-buffer Adaptive Clustering Experience Replay (SODACER), designed to achieve safe and scalable optimal control of nonlinear systems. The proposed SODACER mechanism consisting of a Fast-Buffer for rapid adaptation to recent experiences and a Slow-Buffer equipped with a self-organizing adaptive clustering mechanism to maintain diverse and non-redundant historical experiences. The adaptive clustering mechanism dynamically prunes redundant samples, optimizing memory efficiency while retaining critical environmental patterns. The approach integrates SODACER with Control Barrier Functions (CBFs) to guarantee safety by enforcing state and input constraints throughout the learning process. To enhance convergence and stability, the framework is combined with the Sophia optimizer, enabling adaptive second-order gradient updates. The proposed SODACER-Sophia's architecture ensures reliable, effective, and robust learning in dynamic, safety-critical environments, offering a generalizable solution for applications in robotics, healthcare, and large-scale system optimization. The proposed approach is validated on a nonlinear Human Papillomavirus (HPV) transmission model with multiple control inputs and safety constraints. Comparative evaluations against random and clustering-based experience replay methods demonstrate that SODACER achieves faster convergence, improved sample efficiency, and a superior bias-variance trade-off, while maintaining safe system trajectories, validated via the Friedman test.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
