Jiawang Nie, Liu Yang, Suhan Zhong
This paper studies stochastic optimization problems with polynomials. We propose an optimization model with sample averages and perturbations. The Lasserre type Moment-SOS relaxations are used to solve the sample average optimization. Properties of the optimization and its relaxations are studied. Numerical experiments are presented.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
Recent results of Ye and Hansen, Miltersen and Zwick show that policy iteration for one or two player (perfect information) zero-sum stochastic games, restricted to instances with a fixed discount rate, is strongly polynomial. We show that policy iteration for mean-payoff zero-sum stochastic games is also strongly polynomial when restricted to instances with bounded first mean return time to a given state. The proof is based on methods of nonlinear Perron-Frobenius theory, allowing us to reduce the mean-payoff problem to a discounted problem with state dependent discount rate. Our analysis als…
This paper reviews local and global optimality conditions in polynomial optimization. We summarize the relationship between them.
In this paper, we use stochastic polynomial optimization to derive high-performance operating strategies for heating networks with uncertain or variable demand. The heat flow in district heating networks can be regulated by varying the supply temperature, the mass flow rate, or both simultaneously, leading to different operating strategies. The task of choosing the set-points within each strategy that minimize the network losses for a range of demand conditions can be cast as a two-stage stochastic optimization problem with polynomial objective and polynomial constraints. We derive a generaliz…
This paper discusses a method enabling optimal control of nonlinear systems that are subject to parametric uncertainty. A stochastic optimal tracking problem is formulated that can be expressed in function of the first two stochastic moments of the state. The proposed formulation allows to penalize system performance and system robustness independently. The use of polynomial chaos expansions is investigated to arrive at a computationally tractable formulation expressing the stochastic moments in function of the polynomial expansion coefficients rigorously. It is then demonstrated how the stoch…
This paper studies bilevel polynomial optimization problems. To solve them, we give a method based on polynomial optimization relaxations. Each relaxation is obtained from the Kurash-Kuhn-Tucker (KKT) conditions for the lower level optimization and the exchange technique for semi-infinite programming. For KKT conditions, Lagrange multipliers are represented as polynomial or rational functions. The Moment-SOS relaxations are used to solve the polynomial optimizattion relaxations. Under some general assumptions, we prove the convergence of the algorithm for solving bilevel polynomial optimizatio…
In this chapter we present the sums of Hermitian squares approach to noncommutative polynomial optimization problems. This is an extension of the sums of squares approach for polynomial optimization arising from real algebraic geometry. We provide a gentle introduction to the underlying theory of this methodology and highlight its importance.
In this paper, we present a new stochastic algorithm, namely the stochastic block mirror descent (SBMD) method for solving large-scale nonsmooth and stochastic optimization problems. The basic idea of this algorithm is to incorporate the block-coordinate decomposition and an incremental block averaging scheme into the classic (stochastic) mirror-descent method, in order to significantly reduce the cost per iteration of the latter algorithm. We establish the rate of convergence of the SBMD method along with its associated large-deviation results for solving general nonsmooth and stochastic opti…
In this paper we consider the solvability of a non-convex regular polynomial vector optimization problem on a nonempty closed set. We introduce regularity conditions for the polynomial vector optimization problem and study properties and characterizations of the regularity conditions. Under the regularity conditions, we study nonemptiness and boundedness of the solution sets of the problem. As a consequence, we establish two Frank-Wolfe type theorems for the non-convex polynomial vector optimization problem. Finally, we investigate the solution stability of the non-convex regular polynomial ve…
In this paper, we examine linear programming (LP) relaxations based on Bernstein polynomials for polynomial optimization problems (POPs). We present a progression of increasingly more precise LP relaxations based on expressing the given polynomial in its Bernstein form, as a linear combination of Bernstein polynomials. The well-known bounds on Bernstein polynomials over the unit box combined with linear inter-relationships between Bernstein polynomials help us formulate "Bernstein inequalities" which yield tighter lower bounds for POPs in bounded rectangular domains. The results can be easily …
We consider stochastic convex optimization with a strongly convex (but not necessarily smooth) objective. We give an algorithm which performs only gradient updates with optimal rate of convergence.
We consider polynomials of a few linear forms and show how exploit this type of sparsity for optimization on some particular domains like the Euclidean sphere or a polytope. Moreover, a simple procedure allows to detect this form of sparsity and also allows to provide an approximation of any polynomial by such sparse polynomials.
We study how to solve semidefinite programming relaxations for large scale polynomial optimization. When interior-point methods are used, typically only small or moderately large problems could be solved. This paper studies regularization methods for solving polynomial optimization problems. We describe these methods for semidefinite optimization with block structures, and then apply them to solve large scale polynomial optimization problems. The performance is tested on various numerical examples. By regularization methods, significantly bigger problems could be solved on a regular computer, …
This paper proposes tight semidefinite relaxations for polynomial optimization. The optimality conditions are investigated. We show that generally Lagrange multipliers can be expressed as polynomial functions in decision variables over the set of critical points. The polynomial expressions can be determined by linear equations. Based on these expressions, new Lasserre type semidefinite relaxations are constructed for solving polynomial optimization. We show that the hierarchy of new relaxations has finite convergence, or equivalently, the new relaxations are tight for a finite relaxation order…
A new Simheuristics procedure for stochastic combinatorial optimization
本文提出一种基于统计模型动态切换确定性/随机优化重点的新型仿真启发式方法，在解空间探索与利用间取得平衡，并在现实随机调度问题上验证了其优于标准仿真启发式策略的求解效率。
Utopia point method based robust vector polynomial optimization scheme
本文针对无凸性假设的鲁棒向量多项式优化问题，提出结合乌托邦点标量化与“联合+边际”松弛的求解方案，并兼顾理论与计算分析。
This paper studies convex Generalized Nash Equilibrium Problems (GNEPs) that are given by polynomials. We use rational and parametric expressions for Lagrange multipliers to formulate efficient polynomial optimization for computing Generalized Nash Equilibria (GNEs). The Moment-SOS hierarchy of semidefinite relaxations are used to solve the polynomial optimization. Under some general assumptions, we prove the method can find a GNE if there exists one, or detect nonexistence of GNEs. Numerical experiments are presented to show the efficiency of the method.
Polynomial Optimization Relaxations for Generalized Semi-Infinite Programs
本文针对多项式形式的广义半无限规划，提出基于拉格朗日乘子表达式和多项式扩张的多项式优化松弛层级方法，证明其收敛性，并给出凸无穷约束情形下的精确求解条件，数值实验验证了计算效率。
We propose a novel stochastic optimization algorithm called STOchastic Recursive Momentum for Compositional (STORM-Compositional) optimization that minimizes the composition of expectations of two stochastic functions, the latter being an optimization problem arising in various important machine learning applications. By introducing the momentum term in the compositional gradient updates, STORM-Compositional operates the stochastic recursive variance-reduced compositional gradients in an exponential-moving average way. This leads to an $O(\varepsilon^{-3})$ complexity upper bound for STORM-Com…
In this paper we study stochastic quasi-Newton methods for nonconvex stochastic optimization, where we assume that only stochastic information of the gradients of the objective function is available via a stochastic first-order oracle (SFO). Firstly, we propose a general framework of stochastic quasi-Newton methods for solving nonconvex stochastic optimization. The proposed framework extends the classic quasi-Newton methods working in deterministic settings to stochastic settings, and we prove its almost sure convergence to stationary points. Secondly, we propose a general framework for a clas…
Mirror Descent (MD) is a well-known method of solving non-smooth convex optimization problems. This paper analyzes the stochastic variant of MD with adaptive stepsizes. Its convergence on average is shown to be faster than with the fixed stepsizes and optimal in terms of lower bounds.
Recursive McCormick relaxations have been among the most popular convexification techniques for binary polynomial optimization problems. It is well-understood that both the quality and the size of these relaxations depend on the recursive sequence, and finding an optimal recursive sequence amounts to solving a difficult combinatorial optimization problem. In this paper, we prove that any recursive McCormick relaxation is implied by the extended flower relaxation, a linear programming relaxation that is a natural generalization of the flower relaxation introduced by Del Pia and Khajavirad 2018,…
Higher-Order Stochastic Dominance Constraints in Optimization
本文针对含不可数随机占优约束的优化问题，提出通过有限测试点缩减约束的方法，并引入期望算子和风险度量两种表述，构建优化框架，数值验证其在投资组合优化中的高效稳健性。
This paper addresses risk awareness of stochastic optimization problems. Nested risk measures appear naturally in this context, as they allow beneficial reformulations for algorithmic treatments. The reformulations presented extend usual Hamilton-Jacobi-Bellman equations in dynamic optimization by involving risk awareness in the problem formulation. Nested risk measures are built on risk measures, which originate by conditioning on the history of a stochastic process. We derive martingale properties of these risk measures and use them to prove continuity. It is demonstrated that stochastic opt…
Polynomial Optimization via Random Projection and Consensus
提出一种基于随机线性子空间投影的黑盒方法，将大型半定规划约简为多个小规模半定规划，并在多项式优化问题上验证了其可扩展性的提升。
In this work we introduce the stochastic nonlinear constrained derivative-free optimization method (S)NOWPAC (Stochastic Nonlinear Optimization With Path-Augmented Constraints). The method extends the derivative-free optimizer NOWPAC to be applicable for optimization under uncertainty. It is based on a trust-region framework, utilizing local fully quadratic surrogate models combined with Gaussian process surrogates to mitigate the noise in the objective function and constraint evaluations. We show the performance of our algorithm on a variety of robust optimization problems from the CUTEst ben…
Consistency of sample-based stationary points for infinite-dimensional stochastic optimization
本文研究Banach空间中非光滑随机优化问题的样本近似方法，证明了样本近似Clarke平稳点的一致性，并将其应用于风险规避和风险中性的PDE约束优化。
This paper features expectiles in dynamic and stochastic optimization. Expectiles are a family of risk functionals characterized as minimizers of optimization problems. For this reason, they enjoy various unique stability properties, which can be exploited in risk averse management, in stochastic optimization and in optimal control. The paper provides tight relates of expectiles to other risk functionals and addresses their properties in regression. Further, we extend expectiles to a dynamic framework. As such, they allow incorporating a risk averse aspect in continuous-time dynamic optimizati…
We show the existence of an FPTAS for the problem of maximizing a non-negative polynomial over mixed-integer sets in convex polytopes, when the number of variables is fixed.
We study constrained nested stochastic optimization problems in which the objective function is a composition of two smooth functions whose exact values and derivatives are not available. We propose a single time-scale stochastic approximation algorithm, which we call the Nested Averaged Stochastic Approximation (NASA), to find an approximate stationary point of the problem. The algorithm has two auxiliary averaged sequences (filters) which estimate the gradient of the composite objective function and the inner function value. By using a special Lyapunov function, we show that NASA achieves th…
In this paper, we describe a stochastic adaptive fast gradient descent method based on the mirror variant of similar triangles method. To our knowledge, this is the first attempt to use adaptivity in stochastic method. Additionally, a main result was proved in terms of probabilities of large deviations.
In this note, we prove that for homogeneous polynomial optimization on the sphere, if the objective $f$ is generic in the input space, all feasible points satisfying the first order and second order necessary optimality conditions are local minimizers, which addresses an issue raised in the recent work by Lasserre (Optimization Letters, 2021). As a corollary, this implies that Lasserre's hierarchy has finite convergence when $f$ is generic.
This paper considers distributed stochastic optimization, in which a number of agents cooperate to optimize a global objective function through local computations and information exchanges with neighbors over a network. Stochastic optimization problems are usually tackled by variants of projected stochastic gradient descent. However, projecting a point onto a feasible set is often expensive. The Frank-Wolfe (FW) method has well-documented merits in handling convex constraints, but existing stochastic FW algorithms are basically developed for centralized settings. In this context, the present w…
We present a new theoretical framework for designing linear parameter varying controllers in the polynomial chaos framework. We assume the scheduling variable to be random and apply polynomial chaos approach to synthesize the controller for the resulting linear stochastic dynamical system. Two algorithms are presented that minimize the performance objective with respect to the stochastic system. The first algorithm is based on Galerkin projection and the second algorithm is based on stochastic collocation. LPV controllers from both the algorithms are shown to outperform classical LPV designs w…
S-SOS: Stochastic Sum-Of-Squares for Parametric Polynomial Optimization
本文提出随机平方和（S-SOS）算法，通过构造半定规划层级求解随机参数化多项式优化的严格下界与全局极小点候选解，并利用 n 体先验诱导稀疏性，成功处理高达 40 个变量、矩阵规模超过 800×800 的传感器网络定位问题。
Simulation-based optimization of acoustic liner design in a turbofan engine nacelle for noise reduction purposes can dramatically reduce the cost and time needed for experimental designs. Because uncertainties are inevitable in the design process, a stochastic optimization algorithm is posed based on the conditional value-at-risk measure so that an ideal acoustic liner impedance is determined that is robust in the presence of uncertainties. A parallel reduced-order modeling framework is developed that dramatically improves the computational efficiency of the stochastic optimization solver for …
Many optimization problems in power transmission networks can be formulated as polynomial problems with complex variables. A polynomial optimization problem with complex variables consists in optimizing a real-valued polynomial whose variables and coefficients are complex numbers subject to some complex polynomial equality or inequality constraints. These problems are usually directly converted to real variables, either using the polar form or the rectangular form. In this work, we propose a Julia module allowing the representation of polynomial problems in their original complex formulation. …
This paper studies the dynamic programming principle for general convex stochastic optimization problems introduced by Rockafellar and Wets in [30]. We extend the applicability of the theory by relaxing compactness and boundedness assumptions. In the context of financial mathematics, the relaxed assumption are satisfied under the well-known no-arbitrage condition and the reasonable asymptotic elasticity condition of the utility function. Besides financial mathematics, we obtain several new results in linear and nonlinear stochastic programming and stochastic optimal control.
A polynomial-time solvable class of sparse box-constrained polynomial optimization problems
本文研究在单位超立方体上最小化多项式函数的问题，通过将多项式表示为超图并利用其稀疏性，提出了一种多项式时间求解的新条件。识别出在最优解中取二值的变量子集，并通过消除小且弱耦合的连续变量块，将问题转化为可高效求解的结构化二元优化问题。
This paper considers a class of constrained convex stochastic composite optimization problems whose objective function is given by the summation of a differentiable convex component, together with a nonsmooth but convex component. The nonsmooth component has an explicit max structure that may not easy to compute its proximal mapping. In order to solve these problems, we propose a mini-batch stochastic Nesterov's smoothing (MSNS) method. Convergence and the optimal iteration complexity of the method are established. Numerical results are provided to illustrate the efficiency of the proposed MSN…
A novel exact approach to polynomial optimization
本文提出一种基于线性乘凸函数分解的多项式优化精确方法，通过构造最紧凸松弛下界提升分支定界效率，实验表明其优于BARON和SOS，可在一小时内求解40变量3次及20变量4次问题。
Exploiting Sparsity in Complex Polynomial Optimization
本文提出稀疏自适应复moment-HSOS层级，用于复多项式优化，通过数值实验证明其在计算成本与界质量间提供有效权衡。
Block coordinate descent methods and stochastic subgradient methods have been extensively studied in optimization and machine learning. By combining randomized block sampling with stochastic subgradient methods based on dual averaging, we present stochastic block dual averaging (SBDA)---a novel class of block subgradient methods for convex nonsmooth and stochastic optimization. SBDA requires only a block of subgradients and updates blocks of variables and hence has significantly lower iteration cost than traditional subgradient methods. We show that the SBDA-based methods exhibit the optimal c…
This paper considers a general problem of convex stochastic optimization in a relatively low-dimensional space (e.g., 100 variables). It is known that for deterministic convex optimization problems of small dimensions, the fastest convergence is achieved by the center of gravity type methods (e.g., Vaidya's cutting plane method). For stochastic optimization problems, the question of whether Vaidya's method can be used comes down to the question of how it accumulates inaccuracy in the subgradient. The recent result of the authors states that the errors do not accumulate on iterations of Vaidya'…
Trust-Region Stochastic Optimization with Variance Reduction Technique
本文提出TR-SVR算法，将信赖域、SQP与方差缩减梯度估计相结合，以提升无约束随机优化的稳定性、计算效率和收敛稳健性。
Stochastic variance reduced optimization methods are known to be globally convergent while they suffer from slow local convergence, especially when moderate or high accuracy is needed. To alleviate this problem, we propose an optimization algorithm -- which we refer to as a hybrid acceleration scheme -- for a class of proximal variance reduced stochastic optimization algorithms. The proposed optimization scheme combines a fast locally convergent algorithm, such as a quasi--Newton method, with a globally convergent variance reduced stochastic algorithm, for instance SAGA or L--SVRG. Our global …
This paper presents a general class of dynamic stochastic optimization problems we refer to as Stochastic Depletion Problems. A number of challenging dynamic optimization problems of practical interest are stochastic depletion problems. Optimal solutions for such problems are difficult to obtain, both from a pragmatic computational perspective as also from a theoretical perspective. As such, simple heuristics are highly desirable. We isolate two simple properties that, if satisfied by a problem within this class, guarantee that a myopic policy incurs a performance loss of at most 50 % relative…
This paper describes an approximate method for global optimization of polynomial programming problems with bounded variables. The method uses a reformulation and linearization technique to transform the original polynomial optimization problem into a pair of mixed binary-linear programs. The solutions to these two integer-linear reformulations provide upper and lower bounds on the global solution to the original polynomial program. The tightness of these bounds, the error in approximating each polynomial expression, and the number of constraints that must be added in the process of reformulati…
The goal of this paper is to solve a class of high-order polynomial benchmark optimization problems, including the Goldstein-Price problem and the Three Hump Camel Back problem. By using a generalized canonical duality theory, we are able to transform the nonconvex primal problems to concave dual problems over convex domain(without duality gap), which can be solved easily to obtain global solutions.
Symmetries in polynomial optimization
本章研究利用对称性降低多项式优化计算复杂度，重点探讨矩-SOS层级中表示论与不变量理论的应用，并介绍通用对称性约简技术。
Our work focuses on stochastic gradient methods for optimizing a smooth non-convex loss function with a non-smooth non-convex regularizer. Research on this class of problem is quite limited, and until recently no non-asymptotic convergence results have been reported. We present two simple stochastic gradient algorithms, for finite-sum and general stochastic optimization problems, which have superior convergence complexities compared to the current state-of-the-art. We also compare our algorithms' performance in practice for empirical risk minimization.
Towards Global Solutions for Nonconvex Two-Stage Stochastic Programs: A Polynomial Lower Approximation Approach
本文提出一种两阶段多项式下逼近方法，通过半定松弛构造补偿函数的下界，求解替代问题以获得非凸两阶段随机规划的全局解，并验证其有效性。
This paper considers a class of constrained stochastic composite optimization problems whose objective function is given by the summation of a differentiable (possibly nonconvex) component, together with a certain non-differentiable (but convex) component. In order to solve these problems, we propose a randomized stochastic projected gradient (RSPG) algorithm, in which proper mini-batch of samples are taken at each iteration depending on the total budget of stochastic samples allowed. The RSPG algorithm also employs a general distance function to allow taking advantage of the geometry of the f…
This paper introduces and investigates a regularity condition in the asymptotic sense for optimization problems whose objective functions are polynomial. Under this regularity condition, the normalization argument in asymptotic analysis enables us to see the solution existence as well as the solution stability of these problems. We prove a Frank-Wolfe type theorem for regular optimization problems and an Eaves type theorem for non-regular pseudoconvex optimization problems. Moreover, we show results on the stability such as upper semicontinuity and local upper-Hölder stability of the solution …
A Certificate of Unboundedness for Polynomial Optimization Problems
本文提出一种预处理算法，用于判断多项式优化问题是否从下无界，从而在求解前提供关于问题渐进行几何的信息。
We introduce bounds on the finite-time performance of Markov chain Monte Carlo algorithms in approaching the global solution of stochastic optimization problems over continuous domains. A comparison with other state-of-the-art methods having finite-time guarantees for solving stochastic programming problems is included.
We consider nonlinear multistage stochastic optimization problems in the spaces of integrable functions. We allow for nonlinear dynamics and general objective functionals, including dynamic risk measures. We study causal operators describing the dynamics of the system and derive the Clarke subdifferential for a penalty function involving such operators. Then we introduce the concept of subregular recourse in nonlinear multistage stochastic optimization and establish subregularity of the resulting systems in two formulations: with built-in nonanticipativity and with explicit nonanticipativity c…
On Approximation Algorithms for Commutative Quaternion Polynomial Optimization
本文提出一种多项式时间随机近似算法，解决交换四元数域上受球面约束的齐次多项式优化问题，理论证明了算法的近似比和最坏情况性能保证。
Global optimization of low-rank polynomials
本研究提出LRPOP方法，通过低秩张量分解解决大规模低秩多项式优化问题，显著提升求解能力与数值稳定性。
Stochastic nonconvex optimization problems with nonlinear constraints have a broad range of applications in intelligent transportation, cyber-security, and smart grids. In this paper, first, we propose an inexact-proximal accelerated gradient method to solve a nonconvex stochastic composite optimization problem where the objective is the sum of smooth and nonsmooth functions, the constraint functions are assumed to be deterministic and the solution to the proximal map of the nonsmooth part is calculated inexactly at each iteration. We demonstrate an asymptotic sublinear rate of convergence for…
Stochastic dynamical systems often contain nonlinearities which make it hard to compute probability density functions or statistical moments of these systems. For the moment computations, nonlinearities in the dynamics lead to unclosed moment dynamics; in particular, the time evolution of a moment of a specific order may depend both on moments of order higher than it and on some nonlinear function of other moments. The moment closure techniques are used to find an approximate, close system of equations the moment dynamics. In this work, we extend a moment closure technique based on derivative …
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。