We consider an optimization problem where the decision variable is a string of bounded length. For some time there has been an interest in bounding the performance of the greedy strategy for this problem. Here, we provide weakened sufficient conditions for the greedy strategy to be bounded by a factor of $(1-(1-1/K)^K)$, where $K$ is the optimization horizon length. Specifically, we introduce the notions of $K$-submodularity and $K$-GO-concavity, which together are sufficient for this bound to hold. By introducing a notion of \emph{curvature} $η\in(0,1]$, we prove an even tighter bound with the factor $(1/η)(1-e^{-η})$. Finally, we illustrate the strength of our results by considering two example applications. We show that our results provide weaker conditions on parameter values in these applications than in previous results.
For years, there has been interest in approximation methods for solving dynamic programming problems, because of the inherent complexity in computing optimal solutions characterized by Bellman's principle of optimality. A wide range of approximate dynamic programming (ADP) methods now exists. It is of great interest to guarantee that the performance of an ADP scheme be at least some known fraction, say $β$, of optimal. This paper introduces a general approach to bounding the performance of ADP methods, in this sense, in the stochastic setting. The approach is based on new results for bounding …
In this work, we consider the problem of steering the first two moments of the uncertain state of a discrete time nonlinear stochastic system to prescribed goal quantities at a given final time. In principle, the latter problem can be formulated as a density tracking problem, which seeks for a feedback policy that will keep the probability density function of the state of the system close, in terms of an appropriate metric, to the goal density. The solution to the latter infinite-dimensional problem can be, however, a complex and computationally expensive task. Instead, we propose a more tract…
Consider the problem of choosing a set of actions to optimize an objective function that is a real-valued polymatroid function subject to matroid constraints. The greedy strategy provides an approximate solution to the optimization problem, and it is known to satisfy some performance bounds in terms of the total curvature. The total curvature depends on the value of objective function on sets outside the constraint matroid. If we are given a function defined only on the matroid, the problem still makes sense, but the existing bounds involving the total curvature do not apply. This is puzzling:…
In this paper, the problem of finding state bounds is considered, for the first time, for a class of positive time-delay coupled differential-difference equations (CDDEs) with bounded disturbances. First, we present a novel method, which is based on nonnegative matrices and optimization techniques, for computing a like-exponential componentwise upper bound of the state vector of the CDDEs without disturbances. The main idea is to establish bounds of the state vector on finite-time intervals and then, by using the solution comparison method and the linearity of the system, extend to infinite ti…
This paper studies function approximation for finite horizon discrete time Markov decision processes under certain convexity assumptions. Uniform convergence of these approximations on compact sets is proved under several sampling schemes for the driving random variables. Under some conditions, these approximations form a monotone sequence of lower or upper bounding functions. Numerical experiments involving piecewise linear functions demonstrate that very tight bounding functions for the fair price of a Bermudan put option can be obtained with excellent speed (fractions of a cpu second). Resu…
KAUST（阿卜杜拉国王科技大学）；Purdue University（普渡大学）；Iowa State University（爱荷华州立大学）
We investigate the problem of unconstrained combinatorial multi-armed bandits with full-bandit feedback and stochastic rewards for submodular maximization. Previous works investigate the same problem assuming a submodular and monotone reward function. In this work, we study a more general problem, i.e., when the reward function is not necessarily monotone, and the submodularity is assumed only in expectation. We propose Randomized Greedy Learning (RGL) algorithm and theoretically prove that it achieves a $\frac{1}{2}$-regret upper bound of $\tilde{\mathcal{O}}(n T^{\frac{2}{3}})$ for horizon $…
The problem of real-time control and optimization of components' routing in discrete manufacturing plants, where distinct items must undergo a sequence of jobs, is considered. This problem features a large number of discrete control inputs and the presence of temporal-logic constraints. A new approach is proposed, adopting a shift of perspective with respect to previous contributions, from a Eulerian system model that tracks the state of plant nodes, to a Lagrangian model that tracks the state of each part being processed. The approach features a hierarchical structure. At a higher level, a pr…
*The following abbreviates the abstract. Please refer to the thesis for the full abstract.* After a disaster, locating and extracting victims quickly is critical because mortality rises rapidly after the first two days. To assist search and rescue teams and improve response times, teams of camera-equipped aerial robots can engage in tasks such as mapping buildings and locating victims. These sensing tasks encapsulate difficult (NP-Hard) problems. One way to simplify planning for these tasks is to focus on maximizing sensing performance over a short time horizon. Specifically, consider the prob…
This paper considers the efficient exact computation of the counterpart of the Gittins index for a finite-horizon discrete-state bandit, which measures for each initial state the average productivity, given by the maximum ratio of expected total discounted reward earned to expected total discounted time expended that can be achieved through a number of successive plays stopping by the given horizon. Besides characterizing optimal policies for the finite-horizon one-armed bandit problem, such an index provides a suboptimal heuristic index rule for the intractable finite-horizon multiarmed bandi…
In this paper we prove the efficacy of a simple greedy algorithm for a finite horizon online resource allocation/matching problem, when the corresponding static planning linear program (SPP) exhibits a non-degeneracy condition called the general position gap (GPG). The key intuition that we formalize is that the solution of the reward maximizing SPP is the same as a feasibility Linear Program restricted to the optimal basic activities, and under GPG this solution can be tracked with bounded regret by a greedy algorithm, i.e., without the commonly used technique of periodically resolving the SP…
We address the multi-agent persistent monitoring problem defined on a set of nodes (targets) interconnected over a network topology. A measure of mean overall node state uncertainty evaluated over a finite period is to be minimized by controlling the motion of a cooperating team of agents. To address this problem, we propose an event-driven receding horizon control approach that is computationally efficient, distributed and on-line. The proposed controller differs from the existing on-line gradient-based parametric controllers and off-line greedy cycle search methods that often lead to either …
Actuator placement is a fundamental problem in control design for large-scale networks. In this paper, we study the problem of finding a set of actuator positions by minimizing a given metric, while satisfying a structural controllability requirement and a constraint on the number of actuators. We first extend the classical forward greedy algorithm for applications to graphs that are not necessarily strongly connected. We then improve this greedy algorithm by extending its horizon. This is done by evaluating the actuator position set expansions at the further steps of the classical greedy algo…
We study a risk-aware robot planning problem where a dispatcher must construct a package delivery plan that maximizes the expected reward for a robot delivering packages across multiple epochs. Each package has an associated reward for delivery and a risk of failure. If the robot fails while delivering a package, no future packages can be delivered and the cost of replacing the robot is incurred. The package delivery plan takes place over the course of either a finite or an infinite number of epochs, denoted as the finite horizon problem and infinite horizon problem, respectively. The dispatch…
We study the problem of bounding path-dependent expectations (within any finite time horizon $d$) over the class of discrete-time martingales whose marginal distributions lie within a prescribed tolerance of a given collection of benchmark marginal distributions. This problem is a relaxation of the martingale optimal transport (MOT) problem and is motivated by applications to super-hedging in financial markets. We show that the empirical version of our relaxed MOT problem can be approximated within $O\left( n^{-1/2}\right)$ error where $n$ is the number of samples of each of the individual mar…
