随机最优控制旨在驱动含噪声系统的行为，广泛应用于科学、工程和人工智能领域。本文提出了Stochastic Optimal Control Matching (SOCM)，一种用于随机最优控制的新型迭代扩散优化(IDO)技术，其源于与扩散模型的条件得分匹配损失相同的理念。即，通过尝试拟合一个匹配向量场，将控制学习转化为最小二乘问题。该训练损失与交叉熵损失密切相关，并针对控制函数和出现在匹配向量场中的一族重参数化矩阵进行优化。针对重参数化矩阵的优化旨在最小化匹配向量场的方差。实验表明，在四个控制问题中的三个问题上，我们的算法比所有现有的随机最优控制IDO技术实现了更低的误差，在某些情况下低了一个数量级。SOCM的核心思想是路径重参数化技巧，这是一种可能具有独立研究价值的新技术。代码位于https://github.com/facebookresearch/SOC-matching
英文摘要：
Stochastic optimal control, which has the goal of driving the behavior of noisy systems, is broadly applicable in science, engineering and artificial intelligence. Our work introduces Stochastic Optimal Control Matching (SOCM), a novel Iterative Diffusion Optimization (IDO) technique for stochastic optimal control that stems from the same philosophy as the conditional score matching loss for diffusion models. That is, the control is learned via a least squares problem by trying to fit a matching vector field. The training loss, which is closely connected to the cross-entropy loss, is optimized with respect to both the control function and a family of reparameterization matrices which appear in the matching vector field. The optimization with respect to the reparameterization matrices aims at minimizing the variance of the matching vector field. Experimentally, our algorithm achieves lower error than all the existing IDO techniques for stochastic optimal control for three out of four control problems, in some cases by an order of magnitude. The key idea underlying SOCM is the path-wise reparameterization trick, a novel technique that may be of independent interest. Code at https://github.com/facebookresearch/SOC-matching
We introduce a novel paradigm for learning non-parametric drift and diffusion functions for stochastic differential equation (SDE). The proposed model learns to simulate path distributions that match observations with non-uniform time increments and arbitrary sparseness, which is in contrast with gradient matching that does not optimize simulated responses. We formulate sensitivity equations for learning and demonstrate that our general stochastic distribution optimisation leads to robust and efficient learning of SDE systems.
On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity
Université Jean Monnet Saint-Étienne, CNRS, Institut d’Optique Graduate School, Inria, Laboratoire Hubert Curien UMR 5516；ENS de Lyon, CNRS, Université Claude Bernard Lyon 1, Inria, LIP UMR 5668；Inria, ENS de Lyon, CNRS, Université Claude Bernard Lyon 1, LIP UMR 5668；Institut Universitaire de France
Simulation-based techniques such as variants of stochastic Runge-Kutta are the de facto approach for inference with stochastic differential equations (SDEs) in machine learning. These methods are general-purpose and used with parametric and non-parametric models, and neural SDEs. Stochastic Runge-Kutta relies on the use of sampling schemes that can be inefficient in high dimensions. We address this issue by revisiting the classical SDE literature and derive direct approximations to the (typically intractable) Fokker-Planck-Kolmogorov equation by matching moments. We show how this workflow is f…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）；Artificial Intelligence Research Institute（人工智能研究院）
Schrödinger Bridges (SB) have recently gained the attention of the ML community as a promising extension of classic diffusion models which is also interconnected to the Entropic Optimal Transport (EOT). Recent solvers for SB exploit the pervasive bridge matching procedures. Such procedures aim to recover a stochastic process transporting the mass between distributions given only a transport plan between them. In particular, given the EOT plan, these procedures can be adapted to solve SB. This fact is heavily exploited by recent works giving rise to matching-based SB solvers. The cornerstone he…
In this paper we study the problem of model reduction by moment matching for stochastic systems. We characterize the mathematical object which generalizes the notion of moment to stochastic differential equations and we find a class of models which achieve moment matching. However, differently from the deterministic case, these reduced-order models cannot be considered "simpler" because of the high computational cost paid to determine the moment. To overcome this difficulty, we relax the moment matching problem in two different ways and we present two classes of reduced-order models which, app…
We construct structured H-Infinity optimal model matching problems with rational coefficients, in which the optimal solution is not rational, in the sense that the cost does not achieve its maximal lower bound on the set of rational matching models, but the same maximal lower bound can be reached by using a continuous non-rational matching model.
Morse matchings capture the essential structural information of discrete Morse functions. We show that computing optimal Morse matchings is NP-hard and give an integer programming formulation for the problem. Then we present polyhedral results for the corresponding polytope and report on computational results.
We develop an encompassing framework for matching, covariate balancing, and doubly-robust methods for causal inference from observational data called generalized optimal matching (GOM). The framework is given by generalizing a new functional-analytical formulation of optimal matching, giving rise to the class of GOM methods, for which we provide a single unified theory to analyze tractability, consistency, and efficiency. Many commonly used existing methods are included in GOM and, using their GOM interpretation, can be extended to optimally and automatically trade off balance for variance and…
We provide $\widetilde{O}(ε^{-1})$-pass semi-streaming algorithms for computing $(1-ε)$-approximate maximum cardinality matchings in bipartite graphs. Our most efficient methods are deterministic and use optimal, $O(n)$, space, improving upon the space complexity of the previous state-of-the-art $\widetilde{O}(ε^{-1})$-pass algorithm of Ahn and Guha. To obtain our results we provide semi-streaming adaptations of more general continuous optimization tools. Further, we leverage these techniques to obtain improvements for streaming variants of approximate linear programming, optimal transport, ex…
We consider a discrete-time bipartite matching model with random arrivals of units of supply and demand that can wait in queues located at the nodes in the network. A control policy determines which are matched at each time. The focus is on the infinite-horizon average-cost optimal control problem. A relaxation of the stochastic control problem is proposed, which is found to be a special case of an inventory model, as treated in the classical theory of Clark and Scarf. The optimal policy for the relaxation admits a closed-form expression. Based on the policy for this relaxation, a new matching…
We address the following problem: given two smooth densities on a manifold, find an optimal diffeomorphism that transforms one density into the other. Our framework builds on connections between the Fisher-Rao information metric on the space of probability densities and right-invariant metrics on the infinite-dimensional manifold of diffeomorphisms. This optimal information transport, and modifications thereof, allows us to construct numerical algorithms for density matching. The algorithms are inherently more efficient than those based on optimal mass transport or diffeomorphic registration. …
A two-sided matching system is considered, where servers are assumed to arrive at a fixed rate, while the arrival rate of customers is modulated via a price-control mechanism. We analyse a loss model, wherein customers who are not served immediately upon arrival get blocked, as well as a queueing model, wherein customers wait in a queue until they receive service. The objective is to maximize the platform profit generated from matching servers and customers, subject to quality of service constraints, such as the expected wait time of servers in the loss system model, and the stability of the c…
Optimal transport has been used extensively in resource matching to promote the efficiency of resources usages by matching sources to targets. However, it requires a significant amount of computations and storage spaces for large-scale problems. In this paper, we take a consensus-based approach to decentralize discrete optimal transport problems and develop fully distributed algorithms with alternating direction method of multipliers. We show that our algorithms guarantee certain levels of efficiency and privacy besides the distributed nature. We further derive primal and dual algorithms by ex…
