We propose and analyze batch greedy heuristics for cardinality constrained maximization of non-submodular non-decreasing set functions. We consider the standard greedy paradigm, along with its distributed greedy and stochastic greedy variants. Our theoretical guarantees are characterized by the combination of submodularity and supermodularity ratios. We argue how these parameters define tight modular bounds based on incremental gains, and provide a novel reinterpretation of the classical greedy algorithm using the minorize-maximize (MM) principle. Based on that analogy, we propose a new class of methods exploiting any plausible modular bound. In the context of optimal experimental design for linear Bayesian inverse problems, we bound the submodularity and supermodularity ratios when the underlying objective is based on mutual information. We also develop novel modular bounds for the mutual information in this setting, and describe certain connections to polyhedral combinatorics. We discuss how algorithms using these modular bounds relate to established statistical notions such as leverage scores and to more recent efforts such as volume sampling. We demonstrate our theoretical findings on synthetic problems and on a real-world climate monitoring example.
We investigate the performance of the standard Greedy algorithm for cardinality constrained maximization of non-submodular nondecreasing set functions. While there are strong theoretical guarantees on the performance of Greedy for maximizing submodular functions, there are few guarantees for non-submodular ones. However, Greedy enjoys strong empirical performance for many important non-submodular functions, e.g., the Bayesian A-optimality objective in experimental design. We prove theoretical guarantees supporting the empirical performance. Our guarantees are characterized by a combination of …
The greedy strategy is an approximation algorithm to solve optimization problems arising in decision making with multiple actions. How good is the greedy strategy compared to the optimal solution? In this survey, we mainly consider two classes of optimization problems where the objective function is submodular. The first is set submodular optimization, which is to choose a set of actions to optimize a set submodular objective function, and the second is string submodular optimization, which is to choose an ordered set of actions to optimize a string submodular function. Our emphasis here is on…
KAUST（阿卜杜拉国王科技大学）；Purdue University（普渡大学）；Iowa State University（爱荷华州立大学）
We investigate the problem of unconstrained combinatorial multi-armed bandits with full-bandit feedback and stochastic rewards for submodular maximization. Previous works investigate the same problem assuming a submodular and monotone reward function. In this work, we study a more general problem, i.e., when the reward function is not necessarily monotone, and the submodularity is assumed only in expectation. We propose Randomized Greedy Learning (RGL) algorithm and theoretically prove that it achieves a $\frac{1}{2}$-regret upper bound of $\tilde{\mathcal{O}}(n T^{\frac{2}{3}})$ for horizon $…
We consider classes of objective functions of cardinality constrained maximization problems for which the greedy algorithm guarantees a constant approximation. We propose the new class of $γ$-$α$-augmentable functions and prove that it encompasses several important subclasses, such as functions of bounded submodularity ratio, $α$-augmentable functions, and weighted rank functions of an independence system of bounded rank quotient - as well as additional objective functions for which the greedy algorithm yields an approximation. For this general class of functions, we show a tight bound of $\fr…
We present new tight performance guarantees for the greedy maximization of nondecreasing submodular set functions. Our main result first provides a performance guarantee in terms of the overlap of the optimal and greedy solutions. As a consequence we improve performance guarantees of Nemhauser, Wolsey and Fisher (1978) and Conforti and Cornuéjols (1984) for maximization over subsets, which are at least half the size of the problem domain. As a further application, we obtain a new tight performance guarantee in terms of the cardinality of the problem domain.
We consider a class of discrete optimization problems that aim to maximize a submodular objective function subject to a distributed partition matroid constraint. More precisely, we consider a networked scenario in which multiple agents choose actions from local strategy sets with the goal of maximizing a submodular objective function defined over the set of all possible actions. Given this distributed setting, we develop Constraint-Distributed Continuous Greedy (CDCG), a message passing algorithm that converges to the tight $(1-1/e)$ approximation factor of the optimum global solution using on…
University of Cambridge（剑桥大学）；Georgia Institute of Technology（佐治亚理工学院）
We study the worst-case adaptive optimization problem with budget constraint that is useful for modeling various practical applications in artificial intelligence and machine learning. We investigate the near-optimality of greedy algorithms for this problem with both modular and non-modular cost functions. In both cases, we prove that two simple greedy algorithms are not near-optimal but the best between them is near-optimal if the utility function satisfies pointwise submodularity and pointwise cost-sensitive submodularity respectively. This implies a combined algorithm that is near-optimal w…
This paper studies selecting a subset of the system's output to minimize the state estimation mean square error (MSE). This results in the maximization problem of a set function defined on possible sensor selections subject to a cardinality constraint. We consider to solve it approximately by a greedy search. Since the MSE function is not submodular nor supermodular, the well-known performance guarantees for the greedy solutions do not hold in the present case. Thus, we use the quantities---the submodularity ratio and the curvature---to evaluate the degrees of submodularity and supermodularity…
Submodular function minimization is well studied, and existing algorithms solve it exactly or up to arbitrary accuracy. However, in many applications, such as structured sparse learning or batch Bayesian optimization, the objective function is not exactly submodular, but close. In this case, no theoretical guarantees exist. Indeed, submodular minimization algorithms rely on intricate connections between submodularity and convexity. We show how these relations can be extended to obtain approximation guarantees for minimizing non-submodular functions, characterized by how close the function is t…
The $k$-batch greedy strategy is an approximate algorithm to solve optimization problems where the optimal solution is hard to obtain. Starting with the empty set, the $k$-batch greedy strategy adds a batch of $k$ elements to the current solution set with the largest gain in the objective function while satisfying the constraints. In this paper, we bound the performance of the $k$-batch greedy strategy with respect to the optimal strategy by defining the total curvature $α_k$. We show that when the objective function is nondecreasing and submodular, the $k$-batch greedy strategy satisfies a ha…
The batched greedy strategy is an approximation algorithm to maximize a set function subject to a matroid constraint. Starting with the empty set, the batched greedy strategy iteratively adds to the current solution set a batch of elements that results in the largest gain in the objective function while satisfying the matroid constraints. In this paper, we develop bounds on the performance of the batched greedy strategy relative to the optimal strategy in terms of a parameter called the total batched curvature. We show that when the objective function is a polymatroid set function, the batched…
A $k$-submodular function is an extension of a submodular function in that its input is given by $k$ disjoint subsets instead of a single subset. For unconstrained nonnegative $k$-submodular maximization, Ward and Živný proposed a constant-factor approximation algorithm, which was improved by the recent work of Iwata, Tanigawa and Yoshida presenting a $1/2$-approximation algorithm. Iwata et al. also provided a $k/(2k-1)$-approximation algorithm for monotone $k$-submodular maximization and proved that its approximation ratio is asymptotically tight. More recently, Ohsaka and Yoshida proposed co…
University of Padova（帕多瓦大学）；Czech Technical University in Prague（布拉格捷克理工大学）
Optimization problems with set submodular objective functions have many real-world applications. In discrete scenarios, where the same item can be selected more than once, the domain is generalized from a 2-element set to a bounded integer lattice. In this work, we consider the problem of maximizing a monotone submodular function on the bounded integer lattice subject to a cardinality constraint. In particular, we focus on maximizing DR-submodular functions, i.e., functions defined on the integer lattice that exhibit the diminishing returns property. Given any epsilon > 0, we present a randomi…
In this note we study multiple-ratio fractional 0--1 programs, a broad class of NP-hard combinatorial optimization problems. In particular, under some relatively mild assumptions we provide a complete characterization of the conditions, which ensure that a single-ratio function is submodular. Then we illustrate our theoretical results with the assortment optimization and facility location problems, and discuss practical situations that guarantee submodularity in the considered application settings. In such cases, near-optimal solutions for multiple-ratio fractional 0--1 programs can be found v…
Yale Institute for Network Science（耶鲁网络科学研究所）；Yale University（耶鲁大学）；University of Pennsylvania（宾夕法尼亚大学）
In this paper, we consider the problem of black box continuous submodular maximization where we only have access to the function values and no information about the derivatives is provided. For a monotone and continuous DR-submodular function, and subject to a bounded convex body constraint, we propose Black-box Continuous Greedy, a derivative-free algorithm that provably achieves the tight $[(1-1/e)OPT-ε]$ approximation guarantee with $O(d/ε^3)$ function evaluations. We then extend our result to the stochastic setting where function values are subject to stochastic zero-mean noise. It is thro…
Algorithmically Fair Maximization of Multiple Submodular Objective Functions and Implications to Constrained Fair Division
Athens University of Economics and Business（雅典经济与商业大学）；Archimedes, Athena Research Center（阿尔基米德，阿瑞纳研究中心）；University of Liverpool（利物浦大学）；Sapienza University of Rome（罗马第一大学）；University of Amsterdam（阿姆斯特丹大学）
In this paper, we study stochastic submodular maximization problems with general matroid constraints, that naturally arise in online learning, team formation, facility location, influence maximization, active learning and sensing objective functions. In other words, we focus on maximizing submodular functions that are defined as expectations over a class of submodular functions with an unknown distribution. We show that for monotone functions of this form, the stochastic continuous greedy algorithm attains an approximation ratio (in expectation) arbitrarily close to $(1-1/e) \approx 63\%$ usin…
We consider the optimal coverage problem where a multi-agent network is deployed in an environment with obstacles to maximize a joint event detection probability. The objective function of this problem is non-convex and no global optimum is guaranteed by gradient-based algorithms developed to date. We first show that the objective function is monotone submodular, a class of functions for which a simple greedy algorithm is known to be within 0.63 of the optimal solution. We then derive two tighter lower bounds by exploiting the curvature information (total curvature and elemental curvature) of …
Submodularity is a key property in discrete optimization. Submodularity has been widely used for analyzing the greedy algorithm to give performance bounds and providing insight into the construction of valid inequalities for mixed-integer programs. In recent years, researchers started to study approximate submodularity, with a primary focus on providing performance bounds for iterative approaches. In this paper, we study approximate submodularity from a different perspective in order to broaden its use cases in discrete optimization. We define metrics that quantify approximate submodularity, w…
We study the problem of maximizing a submodular function, subject to a cardinality constraint, with a set of agents communicating over a connected graph. We propose a distributed greedy algorithm that allows all the agents to converge to a near-optimal solution to the global maximization problem using only local information and communication with neighbors in the graph. The near-optimal solution approaches the (1-1/e) approximation of the optimal solution to the global maximization problem with an additive factor that depends on the number of communication steps in the algorithm. We then analy…
Resilient submodular maximization refers to the combinatorial problems studied by Nemhauser and Fisher and asks how to maximize an objective given a number of adversarial removals. For example, one application of this problem is multi-robot sensor planning with adversarial attacks. However, more general applications of submodular maximization are also relevant. Tzoumas et al. obtain near-optimal solutions to this problem by taking advantage of a property called curvature to produce a mechanism which makes certain bait elements interchangeable with other elements of the solution that are produc…
This letter studies the problem of minimizing increasing set functions, or equivalently, maximizing decreasing set functions, over the base of a matroid. This setting has received great interest, since it generalizes several applied problems including actuator and sensor placement problems in control theory, multi-robot task allocation problems, video summarization, and many others. We study two greedy heuristics, namely, the forward and the reverse greedy algorithms. We provide two novel performance guarantees for the approximate solutions obtained by these heuristics depending on both the su…
The selection problem of an optimal set of sensors estimating the snapshot of high-dimensional data is considered. The objective functions based on various criteria of optimal design are adopted to the greedy method: D-optimality, A-optimality, and E-optimality, which maximizes the determinant, minimize the trace of inverse, and maximize the minimum eigenvalue of the Fisher information matrix, respectively. First, the Fisher information matrix is derived depending on the numbers of latent state variables and sensors. Then, the unified formulation of the objective function based on A-optimality…
