发表机构
Missouri University of Science & Technology; OpsCanvas（密苏里科技大学; OpsCanvas公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出自适应划分方案的乐观优化算法，利用神经网络等灵活替代模型，在多指标函数上实现更优的遗憾界，并应用于AWQ量化提升OPT-1.3B模型性能约10%。
AI 中文摘要
诸如工程设计等应用常常需要我们优化一个黑盒函数，即一个内部处理过程在分析上未知且梯度不可用的系统。实践者通常对函数评估的次数有固定预算，优化算法的性能通过其简单遗憾来衡量。在本文中，我们研究用于黑盒优化的“乐观优化”算法类别，这些算法使用域的划分方案。我们开发了学习良好划分方案并在优化过程中使用灵活替代模型（如神经网络）的算法。对于在$d$维空间中$m$维子空间上的多指标函数，我们的算法实现了$\ ilde{O}(n^{-\eta / d})$的遗憾，其中$\eta = 1 + \ rac{d-m}{2m-1}$，而SequOOL（一种最先进的乐观优化算法）的遗憾为$\ ilde{O}(n^{-1/d})$。我们使用我们的方法提高了OPT-1.3B模型的激活感知权重量化（AWQ）的质量，相对于最佳可能的未量化模型，性能提升了约10%。
英文摘要
Applications such as engineering design often require us to optimize a black-box function, i.e., a system whose inner processing is not analytically known and whose gradients are not available. Practitioners often have a fixed budget for the number of function evaluations and the performance of an optimization algorithm is measured by its simple regret. In this paper, we study the class of "Optimistic Optimization" algorithms for black-box optimization that use a partitioning scheme for the domain. We develop algorithms that learn a good partitioning scheme and use flexible surrogate models such as neural networks in the optimization procedure. For multi-index functions on an $m$-dimensional subspace within $d$ dimensions, our algorithm attains $\tilde{O}(n^{-β/ d})$ regret, where $β= 1 + \frac{d-m}{2m-1}$, as opposed to $\tilde{O}(n^{-1/d})$ for SequOOL, a state-of-the-art optimistic optimization algorithm. We use our approach to improve the quality of Activation-aware Weight Quantization (AWQ) of the OPT-1.3B model, achieving $\sim10\%$ improvement in performance relative to the best possible unquantized model.
CommentsAccepted at ICML 2025
Journal refProceedings of the 42nd International Conference on Machine Learning, PMLR 267:58029-58064, 2025
