自适应网格加密的多智能体强化学习
Multi-Agent Reinforcement Learning for Adaptive Mesh Refinement
摘要:
自适应网格细化(AMR)对于复杂物理现象的高效有限元模拟是必要的,因为它根据对随空间和时间变化的更高或更低分辨率的需求分配有限的计算预算。我们提出了一种新的配方AMR作为一个完全合作的马尔可夫博弈,其中每个元素是一个独立的代理人,使细化和去细化的选择,根据当地的信息。我们设计了一种新的深度多智能体强化学习(MARL)算法,称为值分解图网络(VDGN),它解决了AMR为MARL带来的两个核心挑战:由于代理创建和删除而导致的死后信用分配,以及由于网格几何形状的多样性而导致的非结构化观察。这是第一次,我们表明,MARL使预期细化的区域,将遇到复杂的功能在未来的时间,从而解锁全新的区域的错误成本的客观景观是不可访问的传统方法的基础上局部误差估计。综合实验表明,VDGN策略的全局错误和成本指标显着优于基于错误阈值的政策。我们表明,学习的政策推广到测试问题的物理特征,网格几何形状,和更长的模拟时间,在训练中没有看到。我们还扩展了VDGN的多目标优化功能,以找到成本和误差之间的权衡的帕累托前沿。
英文摘要:
Adaptive mesh refinement (AMR) is necessary for efficient finite element simulations of complex physical phenomenon, as it allocates limited computational budget based on the need for higher or lower resolution, which varies over space and time. We present a novel formulation of AMR as a fully-cooperative Markov game, in which each element is an independent agent who makes refinement and de-refinement choices based on local information. We design a novel deep multi-agent reinforcement learning (MARL) algorithm called Value Decomposition Graph Network (VDGN), which solves the two core challenges that AMR poses for MARL: posthumous credit assignment due to agent creation and deletion, and unstructured observations due to the diversity of mesh geometries. For the first time, we show that MARL enables anticipatory refinement of regions that will encounter complex features at future times, thereby unlocking entirely new regions of the error-cost objective landscape that are inaccessible by traditional methods based on local error estimators. Comprehensive experiments show that VDGN policies significantly outperform error threshold-based policies in global error and cost metrics. We show that learned policies generalize to test problems with physical features, mesh geometries, and longer simulation times that were not seen in training. We also extend VDGN with multi-objective optimization capabilities to find the Pareto front of the tradeoff between cost and error.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。