arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.11259cond-mat.mtrl-scics.LG
收藏

基于深度强化学习的节能热处理微结构设计计算发现

Computational Discovery of Energy-Efficient Heat Treatment for Microstructure Design using Deep Reinforcement Learning

Max-Planck-Institut für Eisenforschung(马克斯·普朗克钢铁研究所)

Jaber R. Mianroodi, Nima H. Siboni, Dierk Raabe

摘要:

深度强化学习(DRL)用于开发自主优化和定制设计的热处理工艺,这些工艺既对微观结构敏感又节能。与传统的监督机器学习不同,DRL不依赖于仅从数据中进行静态神经网络训练,而是由学习代理基于奖励和惩罚元素自主开发最佳解决方案,减少或不进行监督。在我们的方法中,一个温度依赖的艾伦-卡恩模型相变被用作环境的DRL代理,作为模型世界中,它获得的经验,并采取自主决策。DRL算法的代理是控制系统的温度,作为合金热处理的模型炉。根据所需的相微观结构来定义试剂的微观结构目标。在训练之后,代理可以生成各种初始微观结构状态的温度-时间曲线,以达到最终期望的微观结构状态。详细研究了该热处理剂的性能和生成的热处理曲线的物理意义。特别地,该试剂能够控制温度以从各种初始条件开始达到所需的微结构。该试剂在处理各种条件方面的这种能力为使用这种方法铺平了道路,该方法还用于回收导向的热处理工艺设计,其中由于杂质侵入,初始组成可以在批次之间变化,并且还用于节能热处理的设计。为了检验这一假设,一个代理人没有总消耗的能量的罚款相比,考虑能源成本。能源成本的惩罚施加作为一个额外的标准,对代理寻找最佳的温度-时间曲线。

英文摘要:

Deep Reinforcement Learning (DRL) is employed to develop autonomously optimized and custom-designed heat-treatment processes that are both, microstructure-sensitive and energy efficient. Different from conventional supervised machine learning, DRL does not rely on static neural network training from data alone, but a learning agent autonomously develops optimal solutions, based on reward and penalty elements, with reduced or no supervision. In our approach, a temperature-dependent Allen-Cahn model for phase transformation is used as the environment for the DRL agent, serving as the model world in which it gains experience and takes autonomous decisions. The agent of the DRL algorithm is controlling the temperature of the system, as a model furnace for heat-treatment of alloys. Microstructure goals are defined for the agent based on the desired microstructure of the phases. After training, the agent can generate temperature-time profiles for a variety of initial microstructure states to reach the final desired microstructure state. The agent's performance and the physical meaning of the heat-treatment profiles generated are investigated in detail. In particular, the agent is capable of controlling the temperature to reach the desired microstructure starting from a variety of initial conditions. This capability of the agent in handling a variety of conditions paves the way for using such an approach also for recycling-oriented heat treatment process design where the initial composition can vary from batch to batch, due to impurity intrusion, and also for the design of energy-efficient heat treatments. For testing this hypothesis, an agent without penalty on the total consumed energy is compared with one that considers energy costs. The energy cost penalty is imposed as an additional criterion on the agent for finding the optimal temperature-time profile.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑