arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.16613cs.LGcs.AI
收藏

用于现实世界发动机控制的安全强化学习

Safe Reinforcement Learning for Real-World Engine Control

Teaching and Research Area Mechatronics in Mobile Propulsion(移动传动机械研究与教学领域);RWTH Aachen University(亚琛工业大学)

Julian Bedei, Lucas Koch, Kevin Badalian, Alexander Winkler, Patrick Schaber, Jakob Andert

AI总结:

本文提出了一种安全强化学习方法,用于在高热效率和低排放的HCCI模式下实现发动机控制,通过实时安全监控和实验验证,有效解决了传统控制方法的局限性。

摘要:

本文介绍了一套工具链,用于将强化学习(RL),特别是深度确定性策略梯度(DDPG)算法应用于安全关键的现实环境。作为示例应用,在同质燃料压缩点火(HCCI)模式下的单缸内燃机测试台上展示了瞬态负载控制。HCCI模式具有高热效率和低排放,但其非线性、自回归和随机性质使得传统控制方法面临挑战。RL提供了一种可行的解决方案,但将其应用于HCCI时必须解决安全问题,如过高的压力上升率。单个不恰当的控制输入可能会严重损坏发动机或导致点火失败和停机。此外,操作限制并非事先已知,必须通过实验确定。为缓解这些风险,采用基于k-最近邻算法的实时安全监控,使测试台能够安全交互。该方法的可行性通过RL智能体与测试台的交互学习控制策略来验证。对于指示均有效压力,均方根误差达到0.1374 bar,与文献中基于神经网络的控制器相当。该工具链的灵活性进一步通过调整智能体的策略来增加乙醇能量份额得到体现,从而促进可再生能源的使用,同时保持安全。这种RL方法解决了长期以来将RL应用于安全关键现实环境的挑战。所开发的工具链,具有适应性和安全机制,为未来RL在发动机测试台和其他安全关键设置中的应用铺平了道路。

英文摘要:

This work introduces a toolchain for applying Reinforcement Learning (RL), specifically the Deep Deterministic Policy Gradient (DDPG) algorithm, in safety-critical real-world environments. As an exemplary application, transient load control is demonstrated on a single-cylinder internal combustion engine testbench in Homogeneous Charge Compression Ignition (HCCI) mode, that offers high thermal efficiency and low emissions. However, HCCI poses challenges for traditional control methods due to its nonlinear, autoregressive, and stochastic nature. RL provides a viable solution, however, safety concerns, such as excessive pressure rise rates, must be addressed when applying to HCCI. A single unsuitable control input can severely damage the engine or cause misfiring and shut down. Additionally, operating limits are not known a priori and must be determined experimentally. To mitigate these risks, real-time safety monitoring based on the k-nearest neighbor algorithm is implemented, enabling safe interaction with the testbench. The feasibility of this approach is demonstrated as the RL agent learns a control policy through interaction with the testbench. A root mean square error of 0.1374 bar is achieved for the indicated mean effective pressure, comparable to neural network-based controllers from the literature. The toolchain's flexibility is further demonstrated by adapting the agent's policy to increase ethanol energy shares, promoting renewable fuel use while maintaining safety. This RL approach addresses the longstanding challenge of applying RL to safety-critical real-world environments. The developed toolchain, with its adaptability and safety mechanisms, paves the way for future applicability of RL in engine testbenches and other safety-critical settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑