Asymptotic Properties of Optimal Trajectories in Dynamic Programming
Sylvain Sorin, Xavier Venel, Guillaume Vigeral
我们证明了在一个动态规划的框架，一致收敛性的有限水平值意味着渐近的平均累积收益是恒定的最佳轨迹。我们分析和讨论了两人对策的几种可能的扩展。
We prove in a dynamic programming framework that uniform convergence of the finite horizon values implies that asymptotically the average accumulated payoff is constant on optimal trajectories. We analyze and discuss several possible extensions to two-person games.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
Shrink-Wrapping trajectories for Linear Programming
研究线性规划的收缩包裹轨迹的几何性质，分析其与中心路径的异同，并给出双曲锥凸性的初等证明。
Indirect trajectory optimization methods such as Differential Dynamic Programming (DDP) have found considerable success when only planning under dynamic feasibility constraints. Meanwhile, nonlinear programming (NLP) has been the state-of-the-art approach when faced with additional constraints (e.g., control bounds, obstacle avoidance). However, a na$ï$ve implementation of NLP algorithms, e.g., shooting-based sequential quadratic programming (SQP), may suffer from slow convergence -- caused from natural instabilities of the underlying system manifesting as poor numerical stability within the o…
Weak sharp minima at infinity and solution stability in mathematical programming via asymptotic analysis
该研究利用渐近锥与广义渐近函数推导非光滑优化问题无穷远弱尖锐极小值的充分条件，证明其可用于非凸优化线性扰动下的解稳定性分析，并给出在一类含凸函数的拟凸函数子类中的应用。
Sequential Convex Programming with Filtering-Based Warm-Starting for Continuous-Time Multiagent Quadrotor Trajectory Optimization
针对多四旋翼连续时间轨迹优化中离散点间约束违反及迭代求解收敛慢问题，提出结合序列凸规划与滤波预热启动的框架，通过转化约束、贝叶斯近似预热加速收敛，实现全轨迹约束满足且计算时间减少两个数量级。
This paper investigates an efficient algorithm for trajectory planning problem of autonomous unmanned aerial vehicles which fly over three-dimensional terrains. The proposed algorithm combines convex optimization with disjunctive programming and receding horizon concept, which has many advantages, such as a high computational speed. Disjunctive programming is applied in order to relax the non-convex constraints of the problem. Moreover, the B-spline curves are employed to represent the trajectories which should be generated in the optimization process.
Asymptotic KKT Conditions for Continuous-Time Nonlinear Programming
本文研究连续时间非线性规划问题的渐近KKT条件，提出序列最优性条件作为终止数值方法的工具，并提出增广拉格朗日方法求解连续时间优化问题。
Empirical Asymptotic Runtime Analysis of Linear Programming Algorithms
本文通过六个优化应用领域的大规模难题LP模型，实证分析了最常用线性规划算法的渐近运行时间增长速率，发现简单回归模型在同类模型中预测运行时间准确，但不同算法的渐近行为差异显著，可能影响未来大规模LP问题求解算法的选择。
Inexact subgradient algorithm with a non-asymptotic convergence guarantee for copositive programming problems
本文提出一种近似子梯度算法，用于求解copositive规划问题，通过非渐近收敛保证在O(ε^{-2})次迭代内得到近似解，且讨论了标准二次规划问题的精确与近似解法，并通过实验验证了算法在检测完全非负矩阵不完全正性方面的有效性。
Asymptotic stationarity and regularity conditions turned out to be quite useful to study the qualitative properties of numerical solution methods for standard nonlinear and complementarity-constrained programs. In this paper, we first extend these notions to nonlinear optimization problems with nonsmooth but Lipschitzian data functions in order to find reasonable notions of asymptotic stationarity and regularity in terms of Clarke's and Mordukhovich's subdifferential construction. Particularly, we compare the associated novel asymptotic constraint qualifications with already existing ones. The…
Solving Underdetermined Boundary Value Problems By Sequential Quadratic Programming
对技术系统建模的常微分方程通常包含被认为对系统是危险的状态。达到这种状态的轨迹通常表明设计中存在缺陷。在本文中，我们提出并研究了一种算法，找到这样的轨迹的属性。也就是说，对于给定的常微分方程，该算法找到起源于一组状态并到达另一组状态的轨迹。该算法是基于序列二次规划适用于正则化最优化问题得到的多次射击。
On the Asymptotic Superlinear Convergence of the Augmented Lagrangian Method for Semidefinite Programming with Multiple Solutions
求解大规模凸半定规划问题一直是一个具有挑战性的数值任务。幸运的是，几个强大的求解器，包括SDPNAL，SDPNAL+和QSDPNAL最近已经开发出解决线性和凸二次SDP问题，以高精度成功。这些求解器是基于增广拉格朗日方法（ALM）的对偶问题的子问题被解决的半光滑牛顿CG方法。值得注意的是，由于Rockafellar关于邻近点算法的一般理论，由ALM生成的原始迭代序列在原问题的二阶充分条件下具有渐近Q-超线性收敛速度.这个二阶充分条件意味着原始问题有唯一的解，这在许多应用中可能是限制性的。为了获得更有见地的解释，这些求解器的高效率，在本文中，我们进行了渐近超线性收敛分析的ALM凸SDP时，原始问题有多个解决方案（可以是无界的）。在一个较弱的二阶增长条件下，我们证明了由ALM生成的原始迭代序列渐近Q-超线性收敛，而对偶可行性和对偶目标函数值渐近R-超线性收敛.此外，通过研究Karush-Kuhn-Tucker解映射的度量次正则性，我们还给出了对偶映射渐近R-超线性收敛的充分条件.
On the asymptotic behavior of a higher-order extrapolation primal-dual interior-point method for nonlinear programming
本文研究非线性规划原始-对偶内点法中利用高阶导数信息进行轨迹外推的渐近收敛行为，证明足够高阶时仅靠外推即可达到障碍参数减小速率，并给出二次规划数值验证。
This paper studies finite-horizon robust tracking control for discrete-time linear systems, based on input-output data. We leverage behavioral theory to represent system trajectories through a set of noiseless historical data, instead of using an explicit system model. By assuming that recent output data available to the controller are affected by noise terms verifying a quadratic bound, we formulate an optimization problem with a linear cost and LMI constraints for solving the robust tracking problem without any approximations. Our approach hinges on a parameterization of noise trajectories c…
The goal of this paper is to develop data-driven control design and evaluation strategies based on linear matrix inequalities (LMIs) and dynamic programming. We consider deterministic discrete-time LTI systems, where the system model is unknown. We propose efficient data collection schemes from the state-input trajectories together with data-driven LMIs to design state-feedback controllers for stabilization and linear quadratic regulation (LQR) problem. In addition, we investigate theoretically guaranteed exploration schemes to acquire valid data from the trajectories under different scenarios…
Control Design along Trajectories with Sums of Squares Programming
提出一种通过平方和规划最大化不变漏斗尺寸的控制设计方法，以形式化保证机器人控制任务的稳定性和安全性。
Motivated by a constrained minimization problem, it is studied the gradient flows with respect to Hessian Riemannian metrics induced by convex functions of Legendre type. The first result characterizes Hessian Riemannian structures on convex sets as those metrics that have a specific integration property with respect to variational inequalities, giving a new motivation for the introduction of Bregman-type distances. Then, the general evolution problem is introduced and a differential inclusion reformulation is given. A general existence result is proved and global convergence is established un…
A New Constraint Qualification for Continuous-Time Nonlinear Programming Based on Asymptotic KKT Conditions
提出一种新的约束规格AKKT-正则性，使得局部最优解满足KKT条件，并证明该规格是最弱的。
To guarantee the safety of flight operations, decision-support systems for air traffic control must be able to improve the usage of airspace capacity and handle increasing demand. In this study, we address the aircraft conflict avoidance and trajectory recovery problem. The problem of finding least deviation conflict-free aircraft trajectories that guarantee the return to a target waypoint is highly complex due to the nature of the nonlinear trajectories that are sought. We present a two-stage iterative algorithm that first solves initial conflict by manipulating their speed and heading contro…
On the Relation between the Minimum Principle and Dynamic Programming for Classical and Hybrid Control Systems
研究了一类混合系统的混合最优控制问题，在这类混合系统中，除了终端和运行成本外，在离散状态之间的切换也会产生成本，在切换时刻允许自治和受控状态的跳跃。在此框架下给出了混合最小值原理和混合动态规划的表述，并证明了在一定的假设条件下，混合最小值原理中的伴随过程和混合动态规划中的值函数的梯度由同一组微分方程控制，具有相同的边界条件，因此沿最优轨迹几乎处处相同.分析的例子来说明的结果，特别是，线性二次混合跟踪问题的Riccati形式主义。
The paper provides a connection between Commutative Algebra and Integer Programming and contains two parts. The first one is devoted to the asymptotic behavior of integer programs with a fixed cost linear functional and the constraint sets consisting of a finite system of linear equations or inequalities with integer coefficients depending linearly on $n$. An integer $N_*$ is determined such that the optima of these integer programs are a quasi-linear function of $n$ for all $n\ge N_*$. Using results in the first part, one can bound in the second part the indices of stability of the Castelnuov…
We introduce an extension of Dual Dynamic Programming (DDP) to solve linear dynamic programming equations. We call this extension IDDP-LP which applies to situations where some or all primal and dual subproblems to be solved along the iterations of the method are solved with a bounded error (inexactly). We provide convergence theorems both in the case when errors are bounded and for asymptotically vanishing errors. We extend the analysis to stochastic linear dynamic programming equations, introducing Inexact Stochastic Dual Dynamic Programming for linear programs (ISDDP-LP), an inexact variant…
Linear Programming Formulations of Deterministic Infinite Horizon Optimal Control Problems in Discrete Time
本文研究离散时间中具有时间折扣和时间平均准则的无限时域最优控制问题。我们建立，这些问题涉及到某些无限维线性规划（IDLP）问题。我们还建立了时间折扣和长期平均准则下问题的最优值之间的渐近关系。
In this work, we approach the minimization of a continuously differentiable convex function under linear equality constraints by a second-order dynamical system with an asymptotically vanishing damping term. The system under consideration is a time rescaled version of another system previously found in the literature. We show fast convergence of the primal-dual gap, the feasibility measure, and the objective function value along the generated trajectories. These convergence rates now depend on the rescaling parameter, and thus can be improved by choosing said parameter appropriately. When the …
This paper studies the dynamic programming principle for general convex stochastic optimization problems introduced by Rockafellar and Wets in [30]. We extend the applicability of the theory by relaxing compactness and boundedness assumptions. In the context of financial mathematics, the relaxed assumption are satisfied under the well-known no-arbitrage condition and the reasonable asymptotic elasticity condition of the utility function. Besides financial mathematics, we obtain several new results in linear and nonlinear stochastic programming and stochastic optimal control.
In this work we consider algorithms for reconstructing time-varying data into a finite sum of discrete trajectories, alternatively, an off-the-grid sparse-spikes decomposition which is continuous in time. Recent work showed that this decomposition was possible by minimising a convex variational model which combined a quadratic data fidelity with dynamical Optimal Transport. We generalise this framework and propose new numerical methods which leverage efficient classical algorithms for computing shortest paths on directed acyclic graphs. Our theoretical analysis confirms that these methods conv…
Incremental Peak-to-Peak Gain Computation and Regulation of Discrete-Time Positive Luré Systems using Linear Programming
本文针对Luré反馈正线性系统，提出用线性规划计算增量峰间增益上界，并设计状态反馈控制器调节增量$\ell_\infty$增益，数值算例验证了方法的有效性。
Nonasymptotic and asymptotic linear convergence of an almost cyclic SHQP Dykstra's algorithm for polyhedral problems
我们证明了一个几乎循环（或广义Gauss-Seidel）Dykstra算法，它结合了SHQP（supporting halfspace-二次规划）策略，可以实现多面体问题的非渐近和渐近线性收敛.
Asymptotic Analysis of Empirical Dynamic Programming in Infinite-Horizon Stochastic Optimal Control
研究无限时域随机最优控制问题，推导基于样本近似的统计极限定理，包括在不同条件下的中心极限定理，比较不同方法的渐近结果，还给出非唯一最优策略模型的极限定理，并通过实例说明理论。
Dissipativity in infinite horizon optimal control and dynamic programming
本文扩展动态规划技术研究紧致控制不变集上具有状态无关最佳渐近平均成本的离散时间无限时域最优控制问题，通过分析耗散性与最优控制的相互作用，提出求解 shifted Bellman Equations 的新型递归方法。
In this paper, we develop approximate dynamic programming methods for stochastic systems modeled as Markov Decision Processes, given both soft performance criteria and hard constraints in a class of probabilistic temporal logic called Probabilistic Computation Tree Logic (PCTL). Our approach consists of two steps: First, we show how to transform a class of PCTL formulas into chance constraints that can be enforced during planning in stochastic systems. Second, by integrating randomized optimization and entropy-regulated dynamic programming, we devise a novel trajectory sampling-based approxima…
On Asymptotic Stability of Non-Demolition Quantum Trajectories with Measurement Imperfections
本文研究了在测量不完美条件下量子非破坏轨迹的渐近稳定性，给出了确保估计轨迹向真实轨迹收敛的条件，并通过数值仿真验证，同时探讨了退相干下由新Kraus算子描述的更一般情况。
The optimal control problem for the kinematic bicycle model is considered where the trajectories are required to satisfy the safety constraints in the continuous-time sense. Based on the differential flatness property of the model, necessary and sufficient conditions in the flat space are provided to guarantee safety in the state space. The optimal control problem is relaxed to the problem of solving three second-order cone programs (SOCPs) sequentially, which find the safe path, the trajectory duration, and the speed profile, respectively. Solutions of the three SOCPs together provide a sub-o…
Stability and Robustness Analysis of Nonlinear Systems via Contraction Metrics and SOS Programming
收缩分析是非线性系统的稳定性理论，其中稳定性在两个任意轨迹之间增量定义。它提供了一个替代框架，在其中研究不确定的互连或系统与外部输入，它提供了几个显着的优势，与传统的李雅普诺夫分析相比。基于收缩的方法对于分析具有不确定参数的系统和证明非线性振子的同步性质特别有用。对于给定的系统，收缩度量的存在性是系统轨迹全局指数收敛的充要条件。对于具有多项式或有理动态的系统，通过使用凸优化和平方和（SOS）编程，可以使对收缩度量的搜索完全算法化。通过放松矩阵的确定性约束，其可行性表明存在一个收缩度量，到SOS约束多项式矩阵的搜索过程是计算上容易处理的。我们通过文献中的例子来说明结果，强调收缩方法和传统的李雅普诺夫技术之间的优势和对比的差异。
Geometric and asymptotic properties associated with linear switched systems
考虑R^n上的连续时间线性切换系统与矩阵的紧凸集相关联。当系统不可约且最大李雅普诺夫指数等于0时，总存在一个Barabanov范数（即沿着线性切换系统的轨迹以及从每一点开始的极值轨迹（即具有常范数的线性切换系统的轨迹）不增加的范数）.本文讨论两组问题：（a）Barabanov范数的性质，如齐次唯一性和严格凸性;（b）线性切换系统极值解的渐近性态。关于问题（a），我们提供了部分答案，并提出了四个开放的问题，由适当的例子。对于问题（b），当n = 3时，我们在定义系统的矩阵集合的正则性假设下建立了一个庞加莱-本迪克松定理。此外，我们回顾了N. E. Barabanov [5]关于R^3上线性切换系统与一对Hurwitz矩阵{A，A + bcT }相关的值得注意的结果。我们首先指出一个致命的差距在巴拉巴诺夫的参数与几何特征与巴拉巴诺夫规范。然后，我们提供了部分答案相对于这个线性切换系统的渐近行为。
We present an algorithm called Tropical Dynamic Programming (TDP) which builds upper and lower approximations of the Bellman value functions in risk-neutral Multistage Stochastic Programming (MSP), with independent noises of finite supports. To tackle the curse of dimensionality, popular parametric variants of Approximate Dynamic Programming approximate the Bellman value function as linear combinations of basis functions. Here, Tropical Dynamic Programming builds upper (resp. lower) approximations of a given value function as min-plus linear (resp. max-plus linear) combinations of "basic funct…
This paper deals with a second order dynamical system with vanishing damping that contains a Tikhonov regularization term, in connection to the minimization problem of a convex Fréchet differentiable function $g$. We show that for appropriate Tikhonov regularization parameters the value of the objective function in a generated trajectory converges fast to the global minimum of the objective function and a trajectory generated by the dynamical system converges weakly to a minimizer of the objective function. We also obtain the fast convergence of the velocities towards zero and some integral es…
Multistage stochastic programming deals with operational and planning problems that involve a sequence of decisions over time while responding to realizations that are uncertain. Algorithms designed to address multistage stochastic linear programming (MSLP) problems often rely upon scenario trees to represent the underlying stochastic process. When this process exhibits stagewise independence, sampling-based techniques, particularly the stochastic dual dynamic programming (SDDP) algorithm, have received wide acceptance. However, these sampling-based methods still operate with a deterministic r…
Stochastic Differential Dynamic Programming for Trajectory Optimization under Partial Observability
提出一种随机微分动态规划算法，用于在部分可观测条件下联合优化标称控制序列和反馈增益，显式考虑协方差传播对标称轨迹的依赖，生成导航感知且鲁棒的轨迹。
Value function for regional control problems via dynamic programming and Pontryagin maximum principle
在本文中，我们专注于区域确定性最优控制问题，即，动力学和成本泛函在状态空间的几个区域中可能不同，并且在它们的界面处呈现不连续性。假设最优轨迹具有局部有限的切换次数，（无Zeno现象），我们利用复制技术证明了区域最优控制问题的值函数是固定结构上经典最优控制问题的值函数轨迹的所有可能结构上的最小值，其中每一个都是对高维经典最优控制问题值函数的某个子流形的限制，因此，提升复制技术被看作是区域最优控制问题值函数的一种去奇异化.反过来，我们将经典的灵敏度关系推广到区域最优控制问题，并证明了这个值函数的正则性是相同的（即，不是更退化）比一个更高维的经典最优控制问题，解除问题。
Asymptotic behavior of second-order dissipative evolution equations combining potential with non-potential effects
研究了结合势效应和非势效应的二阶耗散发展方程的轨线在时间变量趋于无穷大时的渐近收敛性.我们展示了一个尖锐的条件，涉及的阻尼参数和非潜在的运营商的cocoercive系数，保证收敛到平衡的轨迹。应用给出约束优化，不动点问题，动态方法纳什均衡，和渐近稳定的情况下，连续的平衡。
A Dynamic Programming Framework for Optimal Planning of Redundant Robots Along Prescribed Paths With Kineto-Dynamic Constraints
针对冗余机器人沿指定路径的最优规划两步法易产生次优解的问题，提出基于动态规划的统一框架，可适配任意约束与目标，在Panda机器人上验证了其能利用运动学冗余生成可在真实硬件执行的最优可行轨迹。
This paper considers the problem of linear time-invariant (LTI) system identification using input/output data. Recent work has provided non-asymptotic results on partially observed LTI system identification using a single trajectory but is only suitable for stable systems. We provide finite-time analysis for learning Markov parameters based on the ordinary least-squares (OLS) estimator using multiple trajectories, which covers both stable and unstable systems. For unstable systems, our results suggest that the Markov parameters are harder to estimate in the presence of process noise. Without p…
A fundamental challenge in learning to control an unknown dynamical system is to reduce model uncertainty by making measurements while maintaining safety. In this work, we formulate a mathematical definition of what it means to safely learn a dynamical system by sequentially deciding where to initialize the next trajectory. In our framework, the state of the system is required to stay within a given safety region under the (possibly repeated) action of all dynamical systems that are consistent with the information gathered so far. For our first two results, we consider the setting of safely le…
For any quantity of interest in a system governed by ordinary differential equations, it is natural to seek the largest (or smallest) long-time average among solution trajectories, as well as the extremal trajectories themselves. Upper bounds on time averages can be proved a priori using auxiliary functions, the optimal choice of which is a convex optimization problem. We prove that the problems of finding maximal trajectories and minimal auxiliary functions are strongly dual. Thus, auxiliary functions provide arbitrarily sharp upper bounds on time averages. Moreover, any nearly minimal auxili…
The paper develops the Adaptive Dynamic Programming Toolbox (ADPT), which solves optimal control problems for continuous-time nonlinear systems. Based on the adaptive dynamic programming technique, the ADPT computes optimal feedback controls from the system dynamics in the model-based working mode, or from measurements of trajectories of the system in the model-free working mode without the requirement of knowledge of the system model. Multiple options are provided such that the ADPT can accommodate various customized circumstances. Compared to other popular software toolboxes for optimal cont…
Evolutionary stability implies asymptotic stability under multiplicative weights
我们表明，进化稳态在一般（非线性）人口游戏（可以被看作是连续向量场约束的多面体）是渐近稳定的乘法权重动态下（在适当选择的参数称为学习率或步长，我们证明是至关重要的实现收敛，否则甚至混沌行为是可能的表现）。我们的结果意味着，基于乘法权重的进化理论与基于进化稳定性概念的进化理论是相容的（原则上，更一般）。然而，我们的结果进一步建立了乘法权重作为一个非线性规划原语（与标准的非线性规划方法相同）由于各种非线性优化问题，例如在非原子拥塞博弈中找到Nash/Wardrop均衡，众所周知，其配备有凸势函数，以及找到二次规划问题的严格局部最大值，是非线性种群博弈中计算进化稳态问题的特例。
Joint Chance Constrained Safe-Optimal Control
针对轨迹安全概率约束的随机系统有限时域最优控制问题，提出仅最小化安全轨迹期望成本的方法，通过增广状态空间的约束马尔可夫决策过程求解，并推导网格近似误差下的安全边界。
Dynamic programming for infinite horizon boundary control problems of PDE's with age structure
针对一类具有线性状态方程和凸代价的无限时域边界控制问题，提出了一种动态规划方法。证明了该问题的值函数是相应的定态Hamilton-Jacobi-Bellman方程的唯一正则解，并以此证明了反馈控制的存在唯一性.研究这类问题的想法来自于经济应用，特别是来自于具有陈年资本的最优投资模型。Faggian已经在有限视界的情况下研究了这类问题。无限视界的情况更难处理，从经济应用的角度来看，它更有趣，其中主要的问题是长期的最优轨迹和控制的行为。无限视界的研究在这里是通过相应的有限视界问题的非平凡的限制程序。
On characterizing optimal learning trajectories in a class of learning problems
本文利用最大值原理与动态规划的关系，将一类高维非线性函数点估计学习问题中的最优学习轨迹刻画为弱受控梯度系统的最优控制问题，并提出基于逐次Galerkin逼近的构造算法。
Pontryagin-Bellman Differential Dynamic Programming for Low-Thrust Trajectory Optimization with Path Constraints
提出一种结合庞特里亚金极小值原理与微分动态规划的新算法PDDP，通过零空间信赖域方法优化协态，并采用增广拉格朗日方法和二次罚函数处理约束，显著改进了间接法对状态约束问题的鲁棒性和效率。
The trend toward onboard autonomy and spacecraft minimization present significant potential for advances in efficient Line of Sight management by making optimal use of the limited torque resources available. At SENER Aeroespacial, we are implementing AOCS algorithms capable of providing agility in different observation scenarios in which the exploitation of the resources is fundamental for the mission success. In this contribution, we present an in-house optimization toolbox for onboard guidance, the SENER Optimization Toolbox, and we propose its use for online attitude guidance of an agile sp…
Accelerated Gradient Methods for Nonconvex Optimization: Escape Trajectories From Strict Saddle Points and Convergence to Local Minima
本文研究了加速梯度方法在非凸函数上的行为，提出了一类Nesterov型方法，分析了从鞍点逃逸和收敛到局部极小值的机制，提供了渐近和非渐近分析的收敛速率和发散度度量。
The goal of this paper is to investigate new and simple convergence analysis of dynamic programming for linear quadratic regulator problem of discrete-time linear time-invariant systems. In particular, bounds on errors are given in terms of both matrix inequalities and matrix norm. Under a mild assumption on the initial parameter, we prove that the Q-value iteration exponentially converges to the optimal solution. Moreover, a global asymptotic convergence is also presented. These results are then extended to the policy iteration. We prove that in contrast to the Q-value iteration, the policy i…
Sample Average Approximation for Stochastic Programming with Equality Constraints
针对带等式约束的非凸随机规划中SAA方法无渐近最优性保证的问题，通过松弛约束提出改进SAA方法并证明其渐近最优性，应用于随机最优控制，验证了可靠性与降不确定性效果。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。