重新思考离线数据驱动优化中的可学习性
Rethinking Learnability in Offline Data-driven Optimization
- Nanjing University（南京大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出算法依赖型可学习性，设计轨迹学习框架，提出UGTL方法，在五个Design-Bench任务上的25种方法中取得最佳综合平均排名3.1/25，验证了轨迹构建的重要作用。
AI中文摘要：
黑盒优化（BBO）已得到广泛应用，但随着现实世界BBO问题日益复杂，进化算法和贝叶斯优化面临效率挑战。数据驱动优化通过从数据中学习来提高BBO算法的效率，离线数据驱动优化仅使用固定的先前评估集寻求高质量解决方案，无需额外在线评估，因此受到大量关注。尽管已提出多种离线优化方法，但一个基本问题仍未得到解答：离线优化需要何种可学习性才足够？先前理论研究表明，可能近似正确（PAC）可学习性并不充分，因为即使大部分区域已被良好学习，最优区域仍可能学习不佳。本文中，我们提出算法依赖型可学习性，该可学习性仅要求在优化器的轨迹上具有准确性。我们证明，其值查询形式对于代表性离散设置是充分的，包括子模最大化的贪心算法和局部搜索，而其一阶类似形式对于凸最小化的投影梯度下降是充分的。受该概念启发，我们将轨迹学习框架形式化，该框架包括轨迹构建、轨迹建模和候选生成，并在此框架下分析现有基于轨迹的方法。我们进一步提出不确定性感知梯度引导轨迹学习（UGTL），该方法构建反映合理搜索路径的局部一致改进轨迹，用条件扩散模型对其进行建模，并选择多样化的候选集。在五个Design-Bench任务上，UGTL在25种方法中取得了最佳的综合平均排名：3.1/25。受控轨迹分析和跨架构替换证实，我们的轨迹构建在性能提升中发挥重要作用。
英文摘要：
Black-Box Optimization (BBO) has broad applications, while traditional algorithms such as evolutionary algorithms and Bayesian optimization face efficiency challenges as real-world BBO problems grow increasingly complex. Data-driven optimization has been the most popular paradigm to improve the efficiency of BBO, by learning from data. Offline data-driven optimization seeks high-quality solutions using only a fixed set of previous evaluations, attracting substantial attention because it requires no additional online evaluations. Many offline optimization methods have been proposed, but a fundamental question remains unanswered: what learnability is sufficient for offline optimization? Prior theoretical studies show that Probably Approximately Correct (PAC) learnability is insufficient, as the optimal region may remain poorly learned even when most regions are well learned. In this paper, we propose algorithm-dependent learnability, which requires accuracy only on the optimizer's trajectory. We prove that its value-query form is sufficient for representative discrete settings, including greedy and local search for submodular maximization, while its first-order analogue is sufficient for projected gradient descent on convex minimization. Motivated by this notion, we formalize a trajectory-learning framework comprising trajectory construction, trajectory modeling, and candidate generation, and analyze existing trajectory-based methods under it. We further propose Uncertainty-aware Gradient-guided Trajectory Learning (UGTL), which constructs locally coherent improvement trajectories reflecting plausible search paths, models them with conditional diffusion, and selects a diverse candidate set. Our experiments show that UGTL achieves the best average rank, 3.1/25, among 25 methods on Design-Bench tasks, and confirm that our trajectory construction plays a significant role in the improvement.
