In recent years, considerable work has been done to tackle the issue of designing control laws based on observations to allow unknown dynamical systems to perform pre-specified tasks. At least as important for autonomy, however, is the issue of learning which tasks can be performed in the first place. This is particularly critical in situations where multiple (possibly conflicting) tasks and requirements are demanded from the agent, resulting in infeasible specifications. Such situations arise due to over-specification or dynamic operating conditions and are only aggravated when the dynamical system model is learned through simulations. Often, these issues are tackled using regularization and penalties tuned based on application-specific expert knowledge. Nevertheless, this solution becomes impractical for large-scale systems, unknown operating conditions, and/or in online settings where expert input would be needed during the system operation. Instead, this work enables agents to autonomously pose, tune, and solve optimal control problems by compromising between performance and specification costs. Leveraging duality theory, it puts forward a counterfactual optimization algorithm that directly determines the specification trade-off while solving the optimal control problem.
Counterfactual explanations is one of the post-hoc methods used to provide explainability to machine learning models that have been attracting attention in recent years. Most examples in the literature, address the problem of generating post-hoc explanations for black-box machine learning models after the rejection of a loan application. In contrast, in this work, we investigate mathematical programming formulations for scorecard models, a type of interpretable model predominant within the banking industry for lending. The proposed mixed-integer programming formulations combine objective funct…
Pontifical Catholic University of Rio de Janeiro (PUC-Rio)（里约热内卢天主教大学（PUC-Rio））；Polytechnique Montreal（蒙特利尔理工学院）
Counterfactual explanations are usually generated through heuristics that are sensitive to the search's initial conditions. The absence of guarantees of performance and robustness hinders trustworthiness. In this paper, we take a disciplined approach towards counterfactual explanations for tree ensembles. We advocate for a model-based search aiming at "optimal" explanations and propose efficient mixed-integer programming approaches. We show that isolation forests can be modeled within our framework to focus the search on plausible explanations with a low outlier score. We provide comprehensive…
Here, necessary optimal condition for Optimistic Bilevel programming problem is obtained in Asplund spaces. Also we have got necessary optimal conditions in finite dimensional spaces, by assuming differentiability on the given functions.
It has been recently established that a deterministic infinite horizon discounted optimal control problem in discrete time is closely related to a certain infinite dimensional linear programming problem and its dual. In the present paper, we use these results to establish necessary and sufficient optimality conditions for this optimal control problem and apply them to construct a near optimal control.
We introduce and study the infinite dimensional linear programming problem which along with its dual allows one to characterize the optimal value of the deterministic long-run average optimal control problem in the general case when the latter may depend on the initial conditions of the system.
This paper is devoted to a study of infinite horizon optimal control problems with time discounting and time averaging criteria in discrete time. We establish that these problems are related to certain infinite-dimensional linear programming (IDLP) problems. We also establish asymptotic relationships between the optimal values of problems with time discounting and long-run average criteria.
On Certain Hypotheses in Optimal Control Theory and the Relationship of the Maximum Principle with the Dynamic Programming Method Proposed by L. I. Rozonoer
本文将研究L. I. Rozonoer（Automation and Remote Control，2003，vol.64，no.8，pp.1237- 1240）中的最优控制理论，以推导出在所有初始条件下最优控制存在的条件，以及Pontryagin最大值原理与动态规划方法之间的关系。
We consider the coupled problems of optimal thermostat programming and optimal pricing of electricity. Our framework consists of a single user and a single provider (a regulated utility). The provider sets prices for the user, who pays for both total energy consumed (\$/kWh, including peak and off-peak rates) and the peak rate of consumption in a month (a demand charge) (\$/kW). The cost of electricity for the provider is based on a combination of capacity costs (\$/kW) and fuel costs (\$/kWh). In the optimal thermostat programming problem, the user minimizes the amount paid for electricity wh…
In this paper, we study a stochastic optimal control problem under degenerate G-expectation. By using implied partition method, we show that the approximation result for admissible controls still hold. Based on this result, we prove that the value function is deterministic, and obtain the dynamic programming principle. Furthermore, we prove that the value function is the unique viscosity solution to the related HJB equation under degenerate case.
We consider infinite horizon optimal control problems with time averaging and time discounting criteria and give estimates for the Cesaro and Abel limits of their optimal values in the case when they depend on the initial conditions. We establish that these limits are bounded from above by the optimal value of a certain infinite dimensional (ID) linear programming (LP) problem and that they are bounded from below by the optimal value of the corresponding dual problem. (These estimates imply, in particular, that the Cesaro and Abel limits exist and are equal to each other if there is no duality…
This paper investigates an efficient algorithm for trajectory planning problem of autonomous unmanned aerial vehicles which fly over three-dimensional terrains. The proposed algorithm combines convex optimization with disjunctive programming and receding horizon concept, which has many advantages, such as a high computational speed. Disjunctive programming is applied in order to relax the non-convex constraints of the problem. Moreover, the B-spline curves are employed to represent the trajectories which should be generated in the optimization process.
In this paper, we study a stochastic recursive optimal control problem in which the cost functional is described by the solution of a backward stochastic differential equation driven by G-Brownian motion. Under standard assumptions, we establish the dynamic programming principle and the related fully nonlinear HJB equation in the framework of G-expectation. Finally, we show that the value function is the viscosity solution of the obtained HJB equation.
This paper investigates an infinite-horizon linear quadratic stochastic (LQS) optimal control problem for a class of continuous-time stochastic systems. By employing the technique of adaptive dynamic programming (ADP), we propose a novel model-free policy iteration (PI) algorithm. Without needing all information of the system coefficient matrices, the proposed PI algorithm iterates by using the data of the input and system state collected on a fixed time interval. Finally, a numerical example is presented to demonstrate the feasibility of the obtained algorithm.
We develop the linear programming approach to mean-field games in a general setting. This relaxed control approach allows to prove existence results under weak assumptions, and lends itself well to numerical implementation. We consider mean-field game problems where the representative agent chooses both the optimal control and the optimal time to exit the game, where the instantaneous reward function and the coefficients of the state process may depend on the distribution of the other agents. Furthermore, we establish the equivalence between mean-field games equilibria obtained by the linear p…
This paper studies a discrete-time optimal switching problem on a finite horizon. The underlying model has a running reward, terminal reward and signed (positive and negative) switching costs. Using the martingale approach to optimal stopping problems, we extend a well known explicit dynamic programming method for computing the value function and the optimal strategy to the case of signed switching costs.
We consider an optimal stopping problem where a constraint is placed on the distribution of the stopping time. Reformulating the problem in terms of so-called measure-valued martingales allows us to transform the marginal constraint into an initial condition and view the problem as a stochastic control problem; we establish the corresponding dynamic programming principle.
Counterfactual optimization for fault prevention in complex wind energy systems
Department of Statistics；Operations Research, University of Seville, Spain；Vattenfall BA Wind, the Netherlands；Department of Statistics, Operations Research；Applied Mathematics, University of Malaga, Spain
In this paper, we study backward doubly stochastic recursive optimal control problem where the cost function is described by the solution of a backward doubly stochastic differential equation. We give the dynamical programming principle for this kind of optimal control problem and show that the value function is the unique Sobolev weak solution for the corresponding stochastic Hamilton-Jacobi-Bellman equation.
We establish the existence of minimizers in a rather general setting of dynamic stochastic optimization without assuming either convexity or coercivity of the objective function. We apply this to prove the existence of optimal portfolios for non-concave utility maximization problems in financial market models with frictions (such as illiquidity), a first result of its kind. The proofs are based on the dynamic programming principle whose validity is established under quite general assumptions.
The optimal binning is the optimal discretization of a variable into bins given a discrete or continuous numeric target. We present a rigorous and extensible mathematical programming formulation for solving the optimal binning problem for a binary, continuous and multi-class target type, incorporating constraints not previously addressed. For all three target types, we introduce a convex mixed-integer programming formulation. Several algorithmic enhancements, such as automatic determination of the most suitable monotonic trend via a Machine-Learning-based classifier and implementation aspects …
We study the McKean-Vlasov optimal control problem with common noise in various formulations, namely the strong and weak formulation, as well as the Markovian and non-Markovian formulations, and allowing for the law of the control process to appear in the state dynamics. By interpreting the controls as probability measures on an appropriate canonical space with two filtrations, we then develop the classical measurable selection, conditioning and concatenation arguments in this new context, and establish the dynamic programming principle under general conditions.
This work introduces a sequential convex programming framework for non-linear, finite-dimensional stochastic optimal control, where uncertainties are modeled by a multidimensional Wiener process. We prove that any accumulation point of the sequence of iterates generated by sequential convex programming is a candidate locally-optimal solution for the original problem in the sense of the stochastic Pontryagin Maximum Principle. Moreover, we provide sufficient conditions for the existence of at least one such accumulation point. We then leverage these properties to design a practical numerical me…
In this paper, we propose a novel computational method for solving non-linear optimal control problems. The method is based on the use of Fourier--Hermite series for approximating the action-value function arising in dynamic programming instead of the conventional Taylor series expansion used in differential dynamic programming (DDP). The coefficients of the Fourier--Hermite series can be numerically computed by using sigma-point methods, which leads to a novel class of sigma-point based dynamic programming methods. We also prove the quadratic convergence of the method and experimentally test …
