不等式约束随机最优化SPSA算法中的开关更新
Switch Updating in SPSA Algorithm for Stochastic Optimization with Inequality Constraints
摘要：
同时扰动随机逼近（SPSA）由于其高效性、渐近稳定性和减少所需损失函数测量次数等优点，在随机最优化中得到了广泛的应用。然而，标准的SPSA算法需要修改，以处理约束问题。近年来，人们对基于序列二次规划（SQP）的投影思想和惩罚思想进行了分析。这两种方法都具有收敛性和广泛的应用前景，但在实际应用中存在一些限制，如计算时间、复杂性和可行性保证。我们提出了一种基于SPSA的开关更新算法，它更新的损失函数或不等式约束的基础上，根据当前的可行性在每次迭代。我们证明了算法的收敛性，并分析了其相对于其他方法的性能。我们还数值比较开关更新算法与罚函数方法的两个约束的例子。
英文摘要：
Simultaneous perturbation stochastic approximation (SPSA) is widely used in stochastic optimization due to its high efficiency, asymptotic stability, and reduced number of required loss function measurements. However, the standard SPSA algorithm needs to be modified to deal with constrained problems. In recent years, sequential quadratic programming (SQP)-based projection ideas and penalty ideas have been analyzed. Both ideas have convergence results and a potentially wide range of applications, but with some limitations in practical consideration, such as computation time, complexity, and feasibility guarantee. We propose an SPSA-based switch updating algorithm, which updates based on the loss function or the inequality constraints, depending on current feasibility in each iteration. We show convergence results for the algorithm, and analyze its properties relative to other methods. We also numerically compare the switch updating algorithm with the penalty function approach for two constrained examples.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
