在函数空间中使用NEON进行复合贝叶斯优化
Composite Bayesian Optimization In Function Spaces Using NEON -- Neural Epistemic Operator Networks
AI总结:
本文提出NEON网络,通过单一运算符网络 backbone 实现预测不确定性,用于解决复合贝叶斯优化问题,展示其在toy和现实场景中优于其他方法。
摘要:
运算学习是科学计算中的新兴领域,其中机器学习模型的输入或输出是在无限维空间中定义的函数。本文介绍NEON(神经认知运算网络),一种通过单一运算符网络骨架生成带有不确定性的预测的架构,其可训练参数数量比性能相当的深度集合少几个数量级。我们通过考察复合贝叶斯优化(BO)问题来展示该方法在顺序决策中的效用,其中目标是优化函数f=g∘h,其中h:X→C(Y,R^{d_s})是未知映射,输出函数空间中的元素,而g:C(Y,R^{d_s})→R是已知且易于计算的函数。通过在玩具和现实场景中将我们的方法与其他最先进的方法进行比较,我们证明NEON在要求可训练参数数量级更少的同时实现了最先进的性能。
英文摘要:
Operator learning is a rising field of scientific computing where inputs or outputs of a machine learning model are functions defined in infinite-dimensional spaces. In this paper, we introduce NEON (Neural Epistemic Operator Networks), an architecture for generating predictions with uncertainty using a single operator network backbone, which presents orders of magnitude less trainable parameters than deep ensembles of comparable performance. We showcase the utility of this method for sequential decision-making by examining the problem of composite Bayesian Optimization (BO), where we aim to optimize a function $f=g\circ h$, where $h:X\to C(\mathcal{Y},\mathbb{R}^{d_s})$ is an unknown map which outputs elements of a function space, and $g: C(\mathcal{Y},\mathbb{R}^{d_s})\to \mathbb{R}$ is a known and cheap-to-compute functional. By comparing our approach to other state-of-the-art methods on toy and real world scenarios, we demonstrate that NEON achieves state-of-the-art performance while requiring orders of magnitude less trainable parameters.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。