神经网络为何及何时能优化局部近似效果
Why and When Neural Networks Improve Local Approximation in Optimization
浏览论文内容
中文总结 AI 辅助
本文明确了影响神经替代模型在无导数优化中作用的三个关键因素，通过117个基准实例等实验验证了各因素对求解性能的影响，化解了该领域的应用矛盾。
中文摘要 AI 辅助
无导数优化中神经替代模型的应用经验存在矛盾：同一类模型可减少某求解器的评估次数，却对另一求解器无影响甚至使其性能变差。本文表明，明确三个因素可化解该矛盾，且这些因素而非训练曲线报告的拟合精度，决定了学习到的局部模型何时能发挥作用。角色：提出候选解的替代模型仍需经真实目标函数验证，此类模型有帮助；而替代求解器依赖的梯度的模型则有害。半径：拟合于优化路径的模型仅在有界邻域内可靠，其误差既不随邻域缩小而消失，也不随邻域扩大而持续存在。空间：替代模型仅能加速基础方法仍能取得的进展。本文将感知半径的局部泛化形式化，将其与经典全线性条件关联，并在替代模型类、训练流程和基础方法固定的情况下对每个因素进行测试。在117个基准实例中，保障帮助作用可将高准确率求解的实例从67个提升至84个，而梯度替代则降至65个；从训练损失中移除梯度项使替代模型的接受度从0.703降至0.148；在10个噪声水平下的1000次配对比较显示，不存在噪声阈值，仅存在提前停止的基础方法。相同因素限制了增益：基于模型的信赖域求解器几乎无空间，附加相同替代模型后从88降至86，而释放插值软件保持在103，在蒙特卡洛库存模型上，修复接受接口可获得10.40成本单位的价值，而替代模型仅为0.00。
英文摘要
Published experience with neural surrogates in derivative-free optimisation is contradictory: the same family of models that cuts the evaluation count of one solver leaves another unchanged, or makes it worse. We show that the contradiction dissolves once three factors are stated, and that these, rather than the fit accuracy a training curve reports, are what delimit when a learned local model pays. Role: a surrogate that proposes candidates the true objective must still approve helps, while one that replaces a gradient the solver depends on hurts. Radius: a model fitted to an optimisation path is reliable only inside a bounded neighbourhood, and its error neither vanishes as that neighbourhood shrinks nor survives its growth. Room: a surrogate can only accelerate progress the base method is still able to make. We formalise radius-aware local generalisation, relate it to the classical fully linear condition, and test each factor with the surrogate class, training pipeline and base method held fixed. Over 117 benchmark instances safeguarded assistance raises the instances solved to high accuracy from 67 to 84 while gradient replacement lowers them to 65; removing the gradient term from the training loss cuts surrogate acceptance from 0.703 to 0.148; and 1000 paired comparisons over ten noise levels show no noise threshold, only a base method that stops early. The same factors bound the gain: a model-based trust-region solver, which leaves little room, drops from 88 to 86 when the identical surrogate is attached, and released interpolation software stays ahead at 103, and on a Monte-Carlo inventory model repairing the acceptance interface is worth 10.40 cost units against 0.00 for the surrogate.
发表机构
- University of California, Berkeley（加州大学伯克利分校）
- Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
机构由 AI 辅助整理，请以论文原文为准。
