arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.08406cs.LG
收藏

基于马尔可夫决策过程的转移约束贝叶斯优化

Transition Constrained Bayesian Optimization via Markov Decision Processes

Imperial College London(帝国理工学院);BASF SE(巴斯夫公司);ETH Zurich(苏黎世联邦理工学院)

Jose Pablo Folch, Calvin Tsay, Robert M Lee, Behrang Shafei, Weronika Ormaniec, Andreas Krause, Mark van der Wilk, Ruth Misener, Mojmír Mutný

AI总结:

针对现实中贝叶斯优化查询受转移约束的问题,提出基于马尔可夫决策过程框架的方法,用强化学习求解效用函数线性化形式得到规划策略,在多场景验证了有效性。

摘要:

贝叶斯优化是一种用于优化黑盒函数的方法。传统上,它聚焦于可任意查询搜索空间的场景。但许多现实问题并不具备这种灵活性;尤其是下一次查询的搜索空间可能取决于之前的查询。物理科学中存在这类挑战,例如局部移动约束、部分变量需满足单调性,以及状态转移会影响测量精度。总体而言,这类转移约束需要某种形式的规划。本研究通过马尔可夫决策过程(Markov Decision Processes)框架对经典贝叶斯优化进行扩展。我们使用强化学习迭代求解效用函数的可处理线性化形式,得到可在整个决策时域提前规划的策略,这相当于在策略空间中优化采集函数。所得策略可能依赖历史信息且具有非马尔可夫性。我们在化学反应器优化、信息路径规划、机器校准及其他合成示例中展示了该方法的应用。

英文摘要:

Bayesian optimization is a methodology to optimize black-box functions. Traditionally, it focuses on the setting where you can arbitrarily query the search space. However, many real-life problems do not offer this flexibility; in particular, the search space of the next query may depend on previous ones. Example challenges arise in the physical sciences in the form of local movement constraints, required monotonicity in certain variables, and transitions influencing the accuracy of measurements. Altogether, such transition constraints necessitate a form of planning. This work extends classical Bayesian optimization via the framework of Markov Decision Processes. We iteratively solve a tractable linearization of our utility function using reinforcement learning to obtain a policy that plans ahead for the entire horizon. This is a parallel to the optimization of an acquisition function in policy space. The resulting policy is potentially history-dependent and non-Markovian. We showcase applications in chemical reactor optimization, informative path planning, machine calibration, and other synthetic examples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑