基于策略对齐的非分层多保真模型设计策略自适应学习
Adaptive Learning of Design Strategies over Non-Hierarchical Multi-Fidelity Models via Policy Alignment
AI总结：
提出多保真强化学习框架ALPHA，通过策略对齐自适应利用非分层异构低保真模型学习高保真策略，消除了模型调度需求并展现出更优收敛性。
摘要：
多保真强化学习（RL）框架通过利用具有不同精度和计算成本的分析模型，显著提升了工程设计的效率。现有主流方法涵盖迁移学习、人类启发策略、控制变量技术和自适应采样，主要依赖于结构化的模型分层。然而，这种对模型分层的依赖忽略了模型在设计空间中超越单一保真度水平的异构误差分布。本研究提出了ALPHA（Adaptively Learned Policy with Heterogeneous Analyses），这是一种新颖的多保真RL框架，通过自适应地利用任意一组非分层、异构的低保真模型与一个高保真模型，来高效学习高保真策略。具体而言，低保真策略及其经验数据在与其与高保真策略的对齐指导下，被动态用于高效的定向学习。ALPHA的有效性在解析测试优化和八旋翼飞行器设计问题中得到了验证，实验中使用了两个低保真模型和一个高保真模型。结果突显了ALPHA在时间和设计空间上动态利用模型的自适应能力，消除了分层框架中所需的模型调度需求。此外，自适应智能体找到了通往高性能解决方案的更直接路径，展现出优于分层智能体的收敛行为。
英文摘要：
Multi-fidelity Reinforcement Learning (RL) frameworks significantly enhance the efficiency of engineering design by leveraging analysis models with varying levels of accuracy and computational costs. The prevailing methodologies, characterized by transfer learning, human-inspired strategies, control variate techniques, and adaptive sampling, predominantly depend on a structured hierarchy of models. However, this reliance on a model hierarchy overlooks the heterogeneous error distributions of models across the design space, extending beyond mere fidelity levels. This work proposes ALPHA (Adaptively Learned Policy with Heterogeneous Analyses), a novel multi-fidelity RL framework to efficiently learn a high-fidelity policy by adaptively leveraging an arbitrary set of non-hierarchical, heterogeneous, low-fidelity models alongside a high-fidelity model. Specifically, low-fidelity policies and their experience data are dynamically used for efficient targeted learning, guided by their alignment with the high-fidelity policy. The effectiveness of ALPHA is demonstrated in analytical test optimization and octocopter design problems, utilizing two low-fidelity models alongside a high-fidelity one. The results highlight ALPHA's adaptive capability to dynamically utilize models across time and design space, eliminating the need for scheduling models as required in a hierarchical framework. Furthermore, the adaptive agents find more direct paths to high-performance solutions, showing superior convergence behavior compared to hierarchical agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
