In the optimization under uncertainty, decision-makers first select a wait-and-see policy before any realization of uncertainty and then place a here-and-now decision after the uncertainty has been observed. Two-stage stochastic programming is a popular modeling paradigm for the optimization under uncertainty that the decision-makers first specifies a probability distribution, and then seek the best decisions to jointly optimize the deterministic wait-and-see and expected here-and-now costs. In practice, such a probability distribution may not be fully available but is probably observable through an empirical dataset. Therefore, this paper studies distributionally robust two-stage stochastic program (DRTSP) which jointly optimizes the deterministic wait-and-see and worst-case expected here-and-now costs, and the probability distribution comes from a family of distributions which are centered at the empirical distribution using $\infty-$Wasserstein metric. There have been successful developments on deriving tractable approximations of the worst-case expected here-and-now cost in DRTSP. Unfortunately, limited results on exact tractable reformulations of DRTSP. This paper fills this gap by providing sufficient conditions under which the worst-case expected here-and-now cost in DRTSP can be efficiently computed via a tractable convex program. By exploring the properties of binary variables, the developed reformulation techniques are extended to DRTSP with binary random parameters. The main tractable reformulations in this paper are projected into the original decision space and thus can be interpreted as conventional two-stage stochastic programs under discrete support with extra penalty terms enforcing the robustness. These tractable results are further demonstrated to be sharp through complexity analysis.
Optimal transport has recently proved to be a useful tool in various machine learning applications needing comparisons of probability measures. Among these, applications of distributionally robust optimization naturally involve Wasserstein distances in their models of uncertainty, capturing data shifts or worst-case scenarios. Inspired by the success of the regularization of Wasserstein distances in optimal transport, we study in this paper the regularization of Wasserstein distributionally robust optimization. First, we derive a general strong duality result of regularized Wasserstein distrib…
We study distributionally robust optimization (DRO) problems where the ambiguity set is defined using the Wasserstein metric. We show that this class of DRO problems can be reformulated as semi-infinite programs. We give an exchange method to solve the reformulated problem for the general nonlinear model, and a central cutting-surface method for the convex case, assuming that we have a separation oracle. We used a distributionally robust generalization of the logistic regression model to test our algorithm. Numerical experiments on the distributionally robust logistic regression models show th…
Data-driven distributionally robust optimization is a recently emerging paradigm aimed at finding a solution that is driven by sample data but is protected against sampling errors. An increasingly popular approach, known as Wasserstein distributionally robust optimization (DRO), achieves this by applying the Wasserstein metric to construct a ball centred at the empirical distribution and finding a solution that performs well against the most adversarial distribution from the ball. In this paper, we present a general framework for studying different choices of a Wasserstein metric and point out…
We study decision dependent distributionally robust optimization models, where the ambiguity sets of probability distributions can depend on the decision variables. These models arise in situations with endogenous uncertainty. The developed framework includes two-stage decision dependent distributionally robust stochastic programming as a special case. Decision dependent generalizations of five types of ambiguity sets are considered. These sets are based on bounds on moments, Wasserstein metric, $ϕ$-divergence and Kolmogorov-Smirnov test. For the finite support case, we use linear, conic or La…
Adaptive robust optimization problems are usually solved approximately by restricting the adaptive decisions to simple parametric decision rules. However, the corresponding approximation error can be substantial. In this paper we show that two-stage robust and distributionally robust linear programs can often be reformulated exactly as conic programs that scale polynomially with the problem dimensions. Specifically, when the ambiguity set constitutes a 2-Wasserstein ball centered at a discrete distribution, then the distributionally robust linear program is equivalent to a copositive program (…
We consider a distributionally robust second-order stochastic dominance constrained optimization problem. We require the dominance constraints hold with respect to all probability distributions in a Wasserstein ball centered at the empirical distribution. We adopt the sample approximation approach to develop a linear programming formulation that provides a lower bound. We propose a novel split-and-dual decomposition framework which provides an upper bound. We establish quantitative convergency for both lower and upper approximations given some constraint qualification conditions. To efficientl…
This paper studies a distributionally robust chance constrained program (DRCCP) with Wasserstein ambiguity set, where the uncertain constraints should be satisfied with a probability at least a given threshold for all the probability distributions of the uncertain parameters within a chosen Wasserstein distance from an empirical distribution. In this work, we investigate equivalent reformulations and approximations of such problems. We first show that a DRCCP can be reformulated as a conditional value-at-risk constrained optimization problem, and thus admits tight inner and outer approximation…
In this paper, we develop an exact reformulation and a deterministic approximation for distributionally robust joint chance-constrained programmings (DRCCPs) with a general class of convex uncertain constraints under data-driven Wasserstein ambiguity sets. It is known that robust chance constraints can be conservatively approximated by worst-case conditional value-at-risk (CVaR) constraints. It is shown that the proposed worst-case CVaR approximation model can be reformulated as an optimization problem involving biconvex constraints for joint DRCCP. We then derive a convex relaxation of this a…
Aggregation of heating, ventilation, and air conditioning (HVAC) loads can provide reserves to absorb volatile renewable energy, especially solar photo-voltaic (PV) generation. In this paper, we decide HVAC control schedules under uncertain PV generation, using a distributionally robust chance-constrained (DRCC) building load control model under two typical ambiguity sets: the moment-based and Wasserstein ambiguity sets. We derive mixed-integer linear programming (MILP) reformulations for DRCC problems under both sets. Especially, for the Wasserstein ambiguity set, we utilize the right-hand si…
This paper proposes a data-driven distributionally robust shortest path (DRSP) model where the distribution of the travel time in the transportation network can only be partially observed through a finite number of samples. Specifically, we aim to find an optimal path to minimize the worst-case $α$-reliable mean-excess travel time (METT) over a Wasserstein ball, which is centered at the empirical distribution of the sample dataset and the ball radius quantifies the level of its confidence. In sharp contrast to the existing DRSP models, our model is equivalently reformulated as a tractable mixe…
Chance constrained optimal power flow (OPF) has been recognized as a promising framework to manage the risk from variable renewable energy (VRE). In presence of VRE uncertainties, this paper discusses a distributionally robust chance constrained approximate AC-OPF. The power flow model employed in the proposed OPF formulation combines an exact AC power flow model at the nominal operation point and an approximate linear power flow model to reflect the system response under uncertainties. The ambiguity set employed in the distributionally robust formulation is the Wasserstein ball centered at th…
The growing amount of fluctuating renewable infeeds and market liberalization increases uncertainty in power system operation. To capture the influence of fluctuations in operational planning, we model the forecast errors of the uncertain in-feeds as random variables and formulate a security constrained optimal power flow using chance constraints. The chance constraints limit the probability of violations of technical constraints, such as generation and transmission limits, but require a tractable reformulation. In this paper, we discuss different analytical reformulations of the chance constr…
We consider exact deterministic mixed-integer programming (MIP) reformulations of distributionally robust chance-constrained programs (DR-CCP) with random right-hand sides over Wasserstein ambiguity sets. The existing MIP formulations are known to have weak continuous relaxation bounds, and, consequently, for hard instances with small radius, or with large problem sizes, the branch-and-bound based solution processes suffer from large optimality gaps even after hours of computation time. This significantly hinders the practical application of the DR-CCP paradigm. Motivated by these challenges, …
Wasserstein distributionally robust optimization estimators are obtained as solutions of min-max problems in which the statistician selects a parameter minimizing the worst-case loss among all probability models within a certain distance (in a Wasserstein sense) from the underlying empirical measure. While motivated by the need to identify optimal model parameters or decision choices that are robust to model misspecification, these distributionally robust estimators recover a wide range of regularized estimators, including square-root lasso and support vector machines, among others, as particu…
We consider stochastic programs where the distribution of the uncertain parameters is only observable through a finite training dataset. Using the Wasserstein metric, we construct a ball in the space of (multivariate and non-discrete) probability distributions centered at the uniform distribution on the training samples, and we seek decisions that perform best in view of the worst-case distribution within this Wasserstein ball. The state-of-the-art methods for solving the resulting distributionally robust optimization problems rely on global optimization techniques, which quickly become comput…
A data-driven MPC scheme is proposed to safely control constrained stochastic linear systems using distributionally robust optimization. Distributionally robust constraints based on the Wasserstein metric are imposed to bound the state constraint violations in the presence of process disturbance. A feedback control law is solved to guarantee that the predicted states comply with constraints. The stochastic constraints are satisfied with regard to the worst-case distribution within the Wasserstein ball centered at their discrete empirical probability distribution. The resulting distributionally…
In many applications in statistics and machine learning, the availability of data samples from multiple possibly heterogeneous sources has become increasingly prevalent. On the other hand, in distributionally robust optimization, we seek data-driven decisions which perform well under the most adverse distribution from a nominal distribution constructed from data samples within a certain discrepancy of probability distributions. However, it remains unclear how to achieve such distributional robustness in model learning and estimation when data samples from multiple sources are available. In thi…
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；TU Delft（代尔夫特理工大学）
We study a distributionally robust mean square error estimation problem over a nonconvex Wasserstein ambiguity set containing only normal distributions. We show that the optimal estimator and the least favorable distribution form a Nash equilibrium. Despite the non-convex nature of the ambiguity set, we prove that the estimation problem is equivalent to a tractable convex program. We further devise a Frank-Wolfe algorithm for this convex program whose direction-searching subproblem can be solved in a quasi-closed form. Using these ingredients, we introduce a distributionally robust Kalman filt…
We study stochastic optimization problems with chance and risk constraints, where in the latter, risk is quantified in terms of the conditional value-at-risk (CVaR). We consider the distributionally robust versions of these problems, where the constraints are required to hold for a family of distributions constructed from the observed realizations of the uncertainty via the Wasserstein distance. Our main results establish that if the samples are drawn independently from an underlying distribution and the problems satisfy suitable technical assumptions, then the optimal value and optimizers of …
Tsinghua University（清华大学）；University of Toronto（多伦多大学）
In prescriptive analytics, the decision-maker observes historical samples of $(X, Y)$, where $Y$ is the uncertain problem parameter and $X$ is the concurrent covariate, without knowing the joint distribution. Given an additional covariate observation $x$, the goal is to choose a decision $z$ conditional on this observation to minimize the cost $\mathbb{E}[c(z,Y)|X=x]$. This paper proposes a new distributionally robust approach under Wasserstein ambiguity sets, in which the nominal distribution of $Y|X=x$ is constructed based on the Nadaraya-Watson kernel estimator concerning the historical dat…
Markov decision processes (MDPs) are known to be sensitive to parameter specification. Distributionally robust MDPs alleviate this issue by allowing for \emph{ambiguity sets} which give a set of possible distributions over parameter sets. The goal is to find an optimal policy with respect to the worst-case parameter distribution. We propose a framework for solving Distributionally robust MDPs via first-order methods, and instantiate it for several types of Wasserstein ambiguity sets. By developing efficient proximal updates, our algorithms achieve a convergence rate of $O\left(NA^{2.5}S^{3.5}\…
Distributionally robust stochastic optimization (DRSO) is an approach to optimization under uncertainty in which, instead of assuming that there is a known true underlying probability distribution, one hedges against a chosen set of distributions. In this paper we first point out that the set of distributions should be chosen to be appropriate for the application at hand, and that some of the choices that have been popular until recently are, for many applications, not good choices. We next consider sets of distributions that are within a chosen Wasserstein distance from a nominal distribution…
In this paper we discuss distributional robustness in the context of stochastic model predictive control (SMPC) for linear time-invariant systems. We derive a simple approximation of the MPC problem under an additive zero-mean i.i.d. noise with quadratic cost. Due to the lack of distributional information, chance constraints are enforced as distributionally robust (DR) chance constraints, which we opt to unify with the concept of probabilistic reachable sets (PRS). For Wasserstein ambiguity sets, we propose a simple convex optimization problem to compute the DR-PRS based on finitely many distu…
We provide an exact deterministic reformulation for data-driven chance constrained programs over Wasserstein balls. For individual chance constraints as well as joint chance constraints with right-hand side uncertainty, our reformulation amounts to a mixed-integer conic program. In the special case of a Wasserstein ball with the $1$-norm or the $\infty$-norm, the cone is the nonnegative orthant, and the chance constrained program can be reformulated as a mixed-integer linear program. Our reformulation compares favourably to several state-of-the-art data-driven optimization schemes in our numer…
Distributionally robust control (DRC) aims to effectively manage distributional ambiguity in stochastic systems. While most existing works address inaccurate distributional information in fully observable settings, we consider a partially observable DRC problem for discrete-time linear systems using the Wasserstein metric. For a tractable solution, we propose a novel approximation method exploiting the Gelbrich bound of Wasserstein distance. Using techniques from modern distributionally robust optimization, we derive a closed-form expression for the optimal control policy and a tractable semid…
Distributionally robust chance-constrained programs (DR-CCP) over Wasserstein ambiguity sets exhibit attractive out-of-sample performance and admit big-$M$-based mixed-integer programming (MIP) reformulations with conic constraints. However, the resulting formulations often suffer from scalability issues as sample size increases. To address this shortcoming, we derive stronger formulations that scale well with respect to the sample size. Our focus is on ambiguity sets under the so-called left-hand side (LHS) uncertainty, where the uncertain parameters affect the coefficients of the decision va…
We study elective surgery planning in flexible operating rooms (ORs) where emergency patients are accommodated in the existing elective surgery schedule. Specifically, elective surgeries can be scheduled weeks or months in advance. In contrast, an emergency surgery arrives randomly and must be performed on the day of arrival. Probability distributions of the actual durations of elective and emergency surgeries are unknown, and only a possibly small set of historical realizations may be available. To address distributional uncertainty, we first construct an ambiguity set that encompasses all po…
Intensively studied in theory as a promising data-driven tool for decision-making under ambiguity, two-stage distributionally robust optimization (DRO) problems over Wasserstein balls are not necessarily easy to solve in practice. This is partly due to large sample size. In this article, we study a generic two-stage distributionally robust linear program (2-DRLP) over a 1-Wasserstein ball using an affine policy. The 2-DRLP has right-hand-side uncertainty with a rectangular support. Our main contribution is to show that the 2-DRLP problem has a tractable reformulation with a scale independent o…
In this paper, we identify partial correlation information structures that allow for simpler reformulations in evaluating the maximum expected value of mixed integer linear programs with random objective coefficients. To this end, assuming only the knowledge of the mean and the covariance matrix entries restricted to block-diagonal patterns, we develop a reduced semidefinite programming formulation, the complexity of solving which is related to characterizing a suitable projection of the convex hull of the set $\{(\bold{x}, \bold{x}\bold{x}'): \bold{x} \in \mathcal{X}\}$ where $\mathcal{X}$ is…
University of Texas at Austin（德克萨斯大学奥斯汀分校）；New York University（纽约大学）；Georgia Institute of Technology（佐治亚理工学院）
Wasserstein distributionally robust optimization (DRO) has recently achieved empirical success for various applications in operations research and machine learning, owing partly to its regularization effect. Although connection between Wasserstein DRO and regularization has been established in several settings, existing results often require restrictive assumptions, such as smoothness or convexity, that are not satisfied for many problems. In this paper, we develop a general theory on the variation regularization effect of the Wasserstein DRO - a new form of regularization that generalizes tot…
