Distributionally robust optimization involves various probability measures in its problem formulation. They can be bundled to constitute a risk functional. For this equivalence, risk functionals constitute a fundamental building block in distributionally robust stochastic programming. Multistage programming requires conditional versions of risk functionals to re-assess future risk after partial realizations and after preceding decisions.
This paper discusses a construction of the conditional counterpart of a risk functional by passing its genuine characteristics to its conditional counterparts. The conditional risk functionals turn out to be different from the nested analogues of the original (law invariant) risk measure. It is demonstrated that the initial measure and its nested decomposition can be used in a distributionally robust multistage setting.
We study multistage distributionally robust optimization (DRO) to hedge against ambiguity in quantifying the underlying uncertainty of a problem. Recognizing that not all the realizations and scenario paths might have an "effect" on the optimal value, we investigate the question of how to define and identify critical scenarios for nested multistage DRO problems. Our analysis extends the work of Rahimian, Bayraksan, and Homem-de-Mello [Math. Program. 173(1--2): 393--430, 2019], which was in the context of a static/two-stage setting, to the multistage setting. To this end, we define the notions …
In this paper, we focus on a data-driven risk-averse multistage stochastic programming (RMSP) model considering distributional robustness. We optimize the RMSP over the worst-case distribution within an ambiguity set of probability distributions constructed directly from historical data samples. The proposed RMSP is intractable due to the multistage nested minimax structure in its objective function, so we reformulate it into a deterministic equivalent that contains a series of convex combination of expectation and conditional value at risk (CVaR), which can be solved by a customized stochasti…
We study multistage distributionally robust mixed-integer programs under endogenous uncertainty, where the probability distribution of stage-wise uncertainty depends on the decisions made in previous stages. We first consider two ambiguity sets defined by decision-dependent bounds on the first and second moments of uncertain parameters and by mean and covariance matrix that exactly match decision-dependent empirical ones, respectively. For both sets, we show that the subproblem in each stage can be recast as a mixed-integer linear program (MILP). Moreover, we extend the general moment-based am…
This paper investigates a Multistage Distributionally Robust Optimization (MDRO) approach to water allocation under climate uncertainty. The MDRO is formed by creating sets of conditional distributions (called conditional ambiguity sets) on a finite scenario tree. The distributions in the conditional ambiguity sets remain close to a nominal conditional distribution according a $ϕ$-divergence (e.g., Kullback-Liebler divergence, Hellinger distance, Burg entropy, etc.). The paper discusses a decomposition algorithm to solve the resulting MDRO and applies the modeling and solution techniques to al…
Recently, there has been a growing interest in developing inventory control policies which are robust to model misspecification. One approach is to posit that nature selects a worst-case distribution for any stochastic primitives from some pre-specified family. Several communities have observed that a subtle phenomena known as time inconsistency can arise in this framework. In particular, it becomes possible that a policy which is optimal at time zero may not be optimal for the associated optimization problem in which the decision-maker recomputes her policy at each point in time, which has im…
Multistage stochastic optimization problems are oftentimes formulated informally in a pathwise way. These are correct in a discrete setting and suitable when addressing computational challenges, for example. But the pathwise problem statement does not allow an analysis with mathematical rigor and is therefore not appropriate. This paper addresses the foundations. We provide a novel formulation of multistage stochastic optimization problems by involving adequate stochastic processes as control. The fundamental contribution is a proof that there exist measurable versions of intermediate value fu…
On the Foundation of Distributionally Robust Reinforcement Learning
Department of Industrial and Systems Engineering, University of Southern California（工业与系统工程系，南加州大学）；Industrial Engineering and Decision Analytics, Hong Kong University of Science and Technology（工业工程与决策分析，香港科技大学）；Management Science and Engineering, Stanford University（管理科学与工程，斯坦福大学）；Stern School of Business, New York University（斯特恩商学院，纽约大学）
This paper investigates the regret associated with the Distributionally Robust Control (DRC) strategies used to address multistage optimization problems where the involved probability distributions are not known exactly, but rather are assumed to belong to specified ambiguity families. We quantify the price (distributional regret) that one ends up paying for not knowing the exact probability distribution of the stochastic system uncertainty while aiming to control it using the DRC strategies. The conservatism of the DRC strategies for being robust to worst-case uncertainty distribution in the …
This paper reviews recent advances in the field of optimization under uncertainty via a modern data lens, highlights key research challenges and promise of data-driven optimization that organically integrates machine learning and mathematical programming for decision-making under uncertainty, and identifies potential research opportunities. A brief review of classical mathematical programming techniques for hedging against uncertainty is first presented, along with their wide spectrum of applications in Process Systems Engineering. A comprehensive review and classification of the relevant publ…
