arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.06259cs.LGcs.AI
收藏

基于Meta-强化学习和贝叶斯优化的生物序列设计

Designing Biological Sequences via Meta-Reinforcement Learning and Bayesian Optimization

Mila(米拉研究所);Université de Montréal(蒙特利尔大学)

Leo Feng, Padideh Nouri, Aneri Muni, Yoshua Bengio, Pierre-Luc Bacon

摘要:

加速生物序列设计的能力可以对医学领域的进步产生重大影响。该问题可以被构造为全局优化问题,其中目标是昂贵的黑盒函数,使得我们可以查询受限于低轮数限制的大批量。贝叶斯优化是解决这个问题的一种原则性方法。然而,生物序列的天文数字般大的状态空间使得暴力迭代所有可能的序列是不可行的。在本文中,我们提出MetaRLBO,其中我们通过Meta-强化学习训练自回归生成模型,以通过贝叶斯优化提出有希望的序列进行选择。我们提出这个问题,找到一个最佳的政策,在分布的MDP引起的采样子集的数据采集在前几轮。我们的计算机模拟实验表明,这种集成的元学习提供了对奖励错误指定的鲁棒性,并实现了与现有强基线相比具有竞争力的结果。

英文摘要:

The ability to accelerate the design of biological sequences can have a substantial impact on the progress of the medical field. The problem can be framed as a global optimization problem where the objective is an expensive black-box function such that we can query large batches restricted with a limitation of a low number of rounds. Bayesian Optimization is a principled method for tackling this problem. However, the astronomically large state space of biological sequences renders brute-force iterating over all possible sequences infeasible. In this paper, we propose MetaRLBO where we train an autoregressive generative model via Meta-Reinforcement Learning to propose promising sequences for selection via Bayesian Optimization. We pose this problem as that of finding an optimal policy over a distribution of MDPs induced by sampling subsets of the data acquired in the previous rounds. Our in-silico experiments show that meta-learning over such ensembles provides robustness against reward misspecification and achieves competitive results compared to existing strong baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑