This material is the appendix part of my book collaborated with Professor Jianhui Wang at Southern Methodist University. Wei Wei, Jianhui Wang. Modeling and Optimization of Interdependent Energy Infrastructures. Springer Nature Switzerland, 2020
This material provides thorough tutorials on some optimization techniques frequently used in various engineering disciplines, including convex optimization, linearization techniques and mixed-integer linear programming, robust optimization, and equilibrium/game problems. It discusses how to reformulate a difficult (non-convex, multi-agent, min-max) problem to a solver-compatible form (semidefinite program, mixed-integer linear program) via convexification, linearization, and decomposition, so the original problem can be reliably solved by commercial/open-source software. Fundamental algorithms are not the main focus. This material is a good reference for self-learners who have basic knowledge in linear algebra and linear programming. It is one of the main references for an optimization course taught at Tsinghua University.
We consider several problems in the field of distributed optimization and hypothesis testing. We show how to obtain convergence times for these problems that scale linearly with the total number of nodes in the network by using a recent linear-time algorithm for the average consensus problem.
This report is an introduction to transcription methods for trajectory optimization techniques. The first few sections describe the two classes of transcription methods (shooting \& simultaneous) that are used to convert the trajectory optimization problem into a general constrained optimization form. The middle of the report discusses a few extensions to the basic methods, including how to deal with hybrid systems (such as walking robots). The final section goes over a variety of implementation details.
Optimization problems, generalized equations, and the multitude of other variational problems invariably lead to the analysis of sets and set-valued mappings as well as their approximations. We review the central concept of set-convergence and explain its role in defining a notion of proximity between sets, especially for epigraphs of functions and graphs of set-valued mappings. The development leads to an approximation theory for optimization problems and generalized equations with profound consequences for the construction of algorithms. We also introduce the role of set-convergence in varia…
INRIA（法国国家信息与自动化研究所）；Laboratoire d’Informatique de l’Ecole Normale Supérieure（巴黎高等师范学院计算机实验室）
Set-functions appear in many areas of computer science and applied mathematics, such as machine learning, computer vision, operations research or electrical networks. Among these set-functions, submodular functions play an important role, similar to convex functions on vector spaces. In this tutorial, the theory of submodular functions is presented, in a self-contained way, with all results shown from first principles. A good knowledge of convex analysis is assumed.
Bayesian optimization is an approach to optimizing objective functions that take a long time (minutes or hours) to evaluate. It is best-suited for optimization over continuous domains of less than 20 dimensions, and tolerates stochastic noise in function evaluations. It builds a surrogate for the objective and quantifies the uncertainty in that surrogate using a Bayesian machine learning technique, Gaussian process regression, and then uses an acquisition function defined from this surrogate to decide where to sample. In this tutorial, we describe how Bayesian optimization works, including Gau…
The calibration of an advanced constitutive law for soil is a challenging task. This work describes GA-cal, a Fortran software for automatically calibrating constitutive laws using Genetic Algorithms (GA) optimization. The proposed approach sets the calibration problem as a regression, and the GA optimization is used to adjust the model parameters so that a numerical model matches experimental data. This document provides a user guide and a simple tutorial. We showcase GA-cal on the calibration of the Sand Hypoplastic law proposed by von Wolffersdorff, with the oedometer and triaxial drained t…
Bayesian optimization (BO) is a framework for global optimization of expensive-to-evaluate objective functions. Classical BO methods assume that the objective function is a black box. However, internal information about objective function computation is often available. For example, when optimizing a manufacturing line's throughput with simulation, we observe the number of parts waiting at each workstation, in addition to the overall throughput. Recent BO methods leverage such internal information to dramatically improve performance. We call these "grey-box" BO methods because they treat objec…
Technische Universität Berlin（柏林工业大学）；Zuse Institute Berlin（柏林祖斯研究所）；Univ. Grenoble Alpes, Inria, CNRS, LIG（格勒诺布尔阿尔卑斯大学、法国国家信息与自动化研究所、法国国家科学研究中心、格勒诺布尔实验室）
University of Waterloo（滑铁卢大学）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Department of Statistics and Actuarial Science（统计与精算科学系）；David R. Cheriton School of Computer Science（大卫·R·谢里顿计算机科学学院）；Centre for Pattern Analysis and Machine Intelligence（模式分析与机器智能中心）；Machine Learning Laboratory（机器学习实验室）；Data Analytics Laboratory（数据分析实验室）
This is a tutorial and survey paper on Karush-Kuhn-Tucker (KKT) conditions, first-order and second-order numerical optimization, and distributed optimization. After a brief review of history of optimization, we start with some preliminaries on properties of sets, norms, functions, and concepts of optimization. Then, we introduce the optimization problem, standard optimization problems (including linear programming, quadratic programming, and semidefinite programming), and convex problems. We also introduce some techniques such as eliminating inequality, equality, and set constraints, adding sl…
Many dynamical systems of interest are nonlinear, with examples in turbulence, epidemiology, neuroscience, and finance, making them difficult to control using linear approaches. Model predictive control (MPC) is a powerful model-based optimization technique that enables the control of such nonlinear systems with constraints. However, modern systems often lack computationally tractable models, motivating the use of system identification techniques to learn accurate and efficient models for real-time control. In this tutorial article, we review emerging data-driven methods for model discovery an…
Lecture notes for the tutorial at the workshop HPOPT 2008 - 10th International Workshop on High Performance Optimization Techniques (Algebraic Structure in Semidefinite Programming), June 11th to 13th, 2008, Tilburg University, The Netherlands.
A variety of optimal control, estimation, system identification and design problems can be formulated as functional optimization problems with differential equality and inequality constraints. Since these problems are infinite-dimensional and often do not have a known analytical solution, one has to resort to numerical methods to compute an approximate solution. This paper uses a unifying notation to outline some of the techniques used in the transcription step of simultaneous direct methods (which discretize-then-optimize) for solving continuous-time dynamic optimization problems. We focus on…
The Quadratic Unconstrained Binary Optimization (QUBO) model has gained prominence in recent years with the discovery that it unifies a rich variety of combinatorial optimization problems. By its association with the Ising problem in physics, the QUBO model has emerged as an underpinning of the quantum computing area known as quantum annealing and has become a subject of study in neuromorphic computing. Through these connections, QUBO models lie at the heart of experimentation carried out with quantum computers developed by D-Wave Systems and neuromorphic computers developed by IBM. Computatio…
The computational method of parametric probability analysis is introduced. It is demonstrated how to embed logical formulas from the propositional calculus into parametric probability networks, thereby enabling sound reasoning about the probabilities of logical propositions. An alternative direct probability encoding scheme is presented, which allows statements of implication and quantification to be modeled directly as constraints on conditional probabilities. Several example problems are solved, from Johnson-Laird's aces to Smullyan's zombies. Many apparently challenging problems in logic tu…
Université Paris Seine（巴黎塞纳大学）；Université Cergy-Pontoise（塞吉-蓬图瓦兹大学）；ENSEA（法国国立高等电子及应用学院）；CNRS（法国国家科学研究中心）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inria（法国国家信息与自动化研究所）；LIG（格勒诺布尔信息学实验室）；ESIEE Paris（巴黎高等电子与电工技术工程师学院）；Université Paris-Est（巴黎东部大学）；University of Pisa（比萨大学）；CentraleSupélec（法国中央理工-高等电力学院）；Université Paris-Saclay（巴黎萨克雷大学）
Spurred by the enthusiasm surrounding the "Big Data" paradigm, the mathematical and algorithmic tools of online optimization have found widespread use in problems where the trade-off between data exploration and exploitation plays a predominant role. This trade-off is of particular importance to several branches and applications of signal processing, such as data mining, statistical inference, multimedia indexing and wireless communications (to name but a few). With this in mind, the aim of this tutorial paper is to provide a gentle introduction to online optimization and learning algorithms t…
