GRAPE：面向查询高效性的高维贝叶斯优化的梯度细化与感知进展利用

中文摘要 AI 辅助

优化昂贵的高维黑箱函数仍是现代机器学习与科学发现中的核心挑战。尽管局部贝叶斯优化可缓解维度灾难，但现有技术常将下降概率置于进展幅度之上，导致过于保守的步长，仅产生可忽略的改进，在几乎确定会下降但降幅极小的方向上浪费查询。我们提出GRAPE（Gradient Refinement and Progress-Aware Exploitation），这一两阶段框架先通过闭式采集函数细化局部梯度后验，再基于下降条件最大化期望降幅以选择更新方向。理论分析证明，该梯度细化阶段可单调最小化局部不确定性，且随着后验细化，感知进展方向会收敛至真实最速下降。实验表明，GRAPE在高维任务中展现出更优的查询效率：在黑箱对抗攻击中，其平均速度较基线提升5.4倍；在大语言模型提示优化任务中，它在最终平均遗憾值上较次优方法降低3.8个对数单位。

英文摘要

Optimizing expensive, high-dimensional black-box functions remains a central challenge in modern machine learning and scientific discovery. While local Bayesian optimization mitigates the curse of dimensionality, existing techniques often prioritize the probability of descent over the magnitude of progress. This leads to overly conservative steps that yield negligible improvement, wasting queries on directions that are nearly certain to descend but offer little decrease. We introduce Gradient Refinement and Progress-Aware Exploitation (GRAPE), a two-stage framework that first sharpens the local gradient posterior via a closed-form acquisition function, then selects update directions by maximizing the expected decrease conditional on descent. Theoretical analysis proves that this gradient refinement stage monotonically minimizes local uncertainty and that the progress-aware direction converges to true steepest descent as the posterior sharpens. Empirically, GRAPE demonstrates superior query efficiency across high-dimensional tasks: in black-box adversarial attacks, it achieves an average 5.4$\times$ speedup over baselines, and on large language model prompt optimization tasks, it outperforms the second best method by a reduction of 3.8 log-units in the final average regret.

发表机构

• School of Artificial Intelligence（人工智能学院）
• The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））

机构由 AI 辅助整理，请以论文原文为准。