Christian Bayer, Denis Belomestny, Paul Hager, Paolo Pigato, John Schoenmakers
In this paper we study randomized optimal stopping problems and consider corresponding forward and backward Monte Carlo based optimisation algorithms. In particular we prove the convergence of the proposed algorithms and derive the corresponding convergence rates.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
The purpose of this paper is two-fold: We extend the well-known relation between optimal stopping and randomized stopping of a given stochastic process to a situation where the available information flow is a filtration with no a priori assumed relation to the filtration of the process. We call these problems optimal stopping and randomized stopping with general information. Following an idea of Krylov [K] we introduce a special singular stochastic control problem with general information and show that this is also equivalent to the partial information optimal stopping and randomized stopping …
We study two-player zero-sum stopping games in continuous time and infinite horizon. We prove that the value in randomized stopping times exists as soon as the payoff processes are right-continuous. In particular, as opposed to existing literature, we do not assume any conditions on the relations between the payoff processes. We also show that both players have simple epsilon- optimal randomized stopping times; namely, randomized stopping times which are small perturbations of non-randomized stopping times.
A general result on the method of randomized stopping is proved. It is applied to optimal stopping of controlled diffusion processes with unbounded coefficients to reduce it to an optimal control problem without stopping. This is motivated by recent results of Krylov on numerical solutions to the Bellman equation.
In this article we study and classify optimal martingales in the dual formulation of optimal stopping problems. In this respect we distinguish between weakly optimal and surely optimal martingales. It is shown that the family of weakly optimal and surely optimal martingales may be quite large. On the other hand it is shown that the Doob-martingale, that is, the martingale part of the Snell envelope, is in a certain sense the most robust surely optimal martingale under random perturbations. This new insight leads to a novel randomized dual martingale minimization algorithm that doesn't require …
Existence of Strong Randomized Equilibria in Mean-Field Games of Optimal Stopping with Common Noise
本文借助 Bank-El Karoui 表示问题研究带共同噪声的最优停止平均场博弈，证明强随机均衡存在性，并分析严格停止均衡集合的比较静态性质。
Randomized Optimal Stopping Problem in Continuous time and Reinforcement Learning Algorithm
本文提出探索性框架下的最优停止问题，通过熵正则化转化为最优控制，推导HJB方程并证明可解性，给出策略迭代收敛速率，设计强化学习算法并验证数值结果。
Optimal stopping is the problem of determining when to stop a stochastic system in order to maximize reward, which is of practical importance in domains such as finance, operations management and healthcare. Existing methods for high-dimensional optimal stopping that are popular in practice produce deterministic linear policies -- policies that deterministically stop based on the sign of a weighted sum of basis functions -- but are not guaranteed to find the optimal policy within this policy class given a fixed basis function architecture. In this paper, we propose a new methodology for optima…
General Markovian randomized equilibrium existence and construction in zero-sum Dynkin games for diffusions
本文研究一维扩散下的零和Dynkin停时博弈，提出基于随机化概念的新方法，在无收益序条件下构造出任意ε>0的马尔可夫随机全局ε-Nash均衡，还给出ε=0时纯策略与随机化Nash均衡的存在条件，可显式求解均衡停时与对应值。
Finite-player Optimal Stopping Games: Randomization, $α$-potentiality, and Learning
针对有限玩家最优停止博弈复杂度随玩家数快速增长的问题，引入独立随机化形式，构造α-位势函数，提出Potential-CT-DDPG学习算法，数值实验验证了方法的有效性。
Early stopping of stochastic variance reduced gradient for linear inverse problems by the discrepancy principle
本文为随机方差缩减梯度法配备偏差原理，在希尔伯特空间线性反问题中确立其正则化性质并推导收敛速率，为首个先验后停止规则下的随机迭代方法收敛速率结果，数值实验验证了理论。
In this paper we study simulation based optimization algorithms for solving discrete time optimal stopping problems. This type of algorithms became popular among practioneers working in the area of quantitative finance. Using large deviation theory for the increments of empirical processes, we derive optimal convergence rates and show that they can not be improved in general. The rates derived provide a guide to the choice of the number of simulated paths needed in optimization step, which is crucial for the good performance of any simulation based optimization algorithm. Finally, we present a…
Randomized block coordinate descent method for linear ill-posed problems
针对大规模线性不适定问题全梯度迭代法耗时耗存的问题，本文研究了含噪数据下带先验/后验停止准则的随机块坐标下降法的收敛性，给出弱收敛、强收敛结果，引入凸正则化增强解特征检测，并通过成像领域数值实验验证。
Local $W^{2,1}_{p}$ convergence of penalization for time-inconsistent stopping problems
本文针对时间不一致停止问题提出惩罚方法，建立惩罚PDE解局部 $W^{2,1}_{p}$ 收敛性，并在凹性假设下证明控制问题收敛，数值示例验证理论。
Exploratory Optimal Stopping: A Singular Control Formulation
本文提出了一种基于强化学习的探索性最优停止方法，通过正则化和动态规划原理解决退化奇点随机控制问题，并设计了可扩展的 actor-critic 类型无模型算法。
The randomized version of the Kaczmarz method for the solution of linear systems is known to converge linearly in expectation. In this work we extend this result and show that the recently proposed Randomized Sparse Kaczmarz method for recovery of sparse solutions, as well as many variants, also converges linearly in expectation. The result is achieved in the framework of split feasibility problems and their solution by randomized Bregman projections with respect to strongly convex functions. To obtain the expected convergence rates we prove extensions of error bounds for projections. The conv…
Since the late 1950's when quasi-Newton methods first appeared, they have become one of the most widely used and efficient algorithmic paradigms for unconstrained optimization. Despite their immense practical success, there is little theory that shows why these methods are so efficient. We provide a semi-local rate of convergence for the randomized BFGS method which can be significantly better than that of gradient descent, finally giving theoretical evidence supporting the superior empirical performance of the method.
Randomized Methods for Linear Constraints: Convergence Rates and Conditioning
研究线性方程组坐标下降和线性不等式系统迭代投影的随机变体，通过适当概率分布给出期望线性收敛速率与自然线性代数条件数的关系，并推广到度量正则性假设下的凸系统。
We consider the game-theoretic approach to time-inconsistent stopping of a one-dimensional diffusion where the time-inconsistency is due to the presence of a non-exponential (weighted) discount function. In particular, we study (weak) equilibria for this problem in a novel class of mixed (i.e., randomized) stopping times based on a local time construction of the stopping intensity. For a general formulation of the problem we provide a verification theorem giving sufficient conditions for mixed (and pure) equilibria in terms of a set of variational inequalities, including a smooth fit condition…
Preasymptotic Convergence of Randomized Kaczmarz Method
本文研究了随机Kaczmarz方法的预渐进行为，发现低频误差衰减快于高频误差，并提出方差减少策略以稳定渐近收敛。
Reinforcement Learning in Real Option Models
本文提出了一种熵正则化的强化学习方法，用于解决现实期权模型中的最优停止问题，通过平衡探索与利用，提高了在不确定性下的停止决策性能。
Breaking a Logarithmic Barrier in the Stopping Time Convergence Rate of Stochastic First-order Methods
针对算法自适应终止的实际，提出基于停时的随机优化收敛分析新方法，通过开发控制近超鞅大偏差性质的引理，打破了现有结果的对数障碍。
Stopping criteria for Stochastic Gradient Descent (SGD) methods play important roles from enabling adaptive step size schemes to providing rigor for downstream analyses such as asymptotic inference. Unfortunately, current stopping criteria for SGD methods are often heuristics that rely on asymptotic normality results or convergence to stationary distributions, which may fail to exist for nonconvex functions and, thereby, limit the applicability of such stopping criteria. To address this issue, in this work, we rigorously develop two stopping criteria for SGD that can be applied to a broad clas…
On the time consistent solution to optimal stopping problems with expectation constraint
针对带停时期望约束的一维扩散最优停时问题，本文以随机化且纯状态依赖的停时定义弱均衡，给出验证定理、必要条件及结构性质，并用示例说明猜测—验证方法。
The Stochastic Primal-Dual Hybrid Gradient (SPDHG) was proposed by Chambolle et al. (2018) and is an efficient algorithm to solve some nonsmooth large-scale optimization problems. In this paper we prove its almost sure convergence for convex but not necessarily strongly convex functionals. We also look into its application to parallel Magnetic Resonance Imaging reconstruction in order to test performance of SPDHG. Our numerical results show that for a range of settings SPDHG converges significantly faster than its deterministic counterpart.
We investigate the stability of equilibrium-induced optimal values with respect to (w.r.t.) reward functions $f$ and transition kernels $Q$ for time-inconsistent stopping problems under nonexponential discounting in discrete time. First, with locally uniform convergence of $f$ and $Q$ equipped with total variation distance, we show that the optimal value is semi-continuous w.r.t. $(f,Q)$. We provide examples showing that continuity may fail in general, and the convergence for $Q$ in total variation cannot be replaced by weak convergence. Next we show that with the uniform convergence of $f$ an…
Entropy Regularization in Mean-Field Games of Optimal Stopping
本文通过熵正则化将最优停止平均场博弈转化为奇异随机控制平均场博弈，建立均衡存在性与稳定性，并提出可收敛的虚拟对局算法，为无模型学习求解OS-MFG奠定理论基础。
We develop the fictitious play algorithm in the context of the linear programming approach for mean field games of optimal stopping and mean field games with regular control and absorption. This algorithm allows to approximate the mean field game population dynamics without computing the value function by solving linear programming problems associated with the distributions of the players still in the game and their stopping times/controls. We show the convergence of the algorithm using the topology of convergence in measure in the space of subprobability measures, which is needed to deal with…
In this work, we analyze the global convergence property of coordinate gradient descent with random choice of coordinates and stepsizes for non-convex optimization problems. Under generic assumptions, we prove that the algorithm iterate will almost surely escape strict saddle points of the objective function. As a result, the algorithm is guaranteed to converge to local minima if all saddle points are strict. Our proof is based on viewing coordinate descent algorithm as a nonlinear random dynamical system and a quantitative finite block analysis of its linearization around saddle points.
The randomized sparse Kaczmarz method was recently proposed to recover sparse solutions of linear systems. In this work, we introduce a greedy variant of the randomized sparse Kaczmarz method by employing the sampling Kaczmarz-Motzkin method, and prove its linear convergence in expectation with respect to the Bregman distance in the noiseless and noisy cases. This greedy variant can be viewed as a unification of the sampling Kaczmarz-Motzkin method and the randomized sparse Kaczmarz method, and hence inherits the merits of these two methods. Numerically, we report a couple of experimental resu…
In this paper we present a convergence rate analysis of inexact variants of several randomized iterative methods. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic subspace ascent. A common feature of these methods is that in their update rule a certain sub-problem needs to be solved exactly. We relax this requirement by allowing for the sub-problem to be solved inexactly. In particular, we propose and analyze inexact randomized iterative methods for solving three closely related problems: a convex stochastic quadratic optim…
Time-inconsistent mean-field stopping problems: A regularized equilibrium approach
针对非指数折扣下平均场MDP集中停时的时间不一致问题，提出熵正则化辅助方法，通过绍德尔不动点定理证明正则化均衡存在，且正则参数趋于0时收敛为原问题松弛均衡，并建立其与大N智能体MDP的联系。
Inertial randomized Kaczmarz algorithms for solving coherent linear systems
本研究针对相干线性系统求解问题，将双子空间Kaczmarz方法视为交替惯性随机Kaczmarz算法并提出更优收敛速率估计，进一步提出多步惯性随机Kaczmarz算法加速收敛，实验验证其性能显著更优。
Given an initial (resp., terminal) probability measure $μ$ (resp., $ν$) on $\mathbb{R}^d$, we characterize those optimal stopping times $τ$ that maximize or minimize the functional $\mathbb{E} |B_0 - B_τ|^α$, $α> 0$, where $(B_t)_t$ is Brownian motion with initial law $B_0\sim μ$ and with final distribution --once stopped at $τ$-- equal to $B_τ\sim ν$. The existence of such stopping times is guaranteed by Skorohod-type embeddings of probability measures in "subharmoic order" into Brownian motion. This problem is equivalent to an optimal mass transport problem with certain constraints, namely t…
Mixed Markov-Perfect Equilibria in the Continuous-Time War of Attrition
本文针对状态变量服从齐次线性扩散的连续时间消耗战模型，利用紧绝对收缩核拓扑和 Eilenberg-Montgomery 不动点定理证明了随机停止时间下混合策略马尔可夫完美均衡的存在性，并给出无纯策略均衡的实例。
On the convergence of iterative regularization method assisted by the graph Laplacian with early stopping
提出一种数据辅助迭代正则化方法IRMGL+Ψ，结合图拉普拉斯正则化与经典迭代技术，并证明差异原则作为早停准则的可靠性，实验表明伴随算子初始化效果最优。
From finite population optimal stopping to mean field optimal stopping
本文研究有限总体最优停止问题向平均场极限的收敛性，利用粘性解刻画和单调格式方法证明值函数收敛，并通过有限总体策略的聚点刻画平均场最优停止策略，同时推广了混沌传播至停止McKean-Vlasov扩散。
On the convergence and sampling of randomized primal-dual algorithms and their application to parallel MRI reconstruction
本文证明了随机原始-对偶混合梯度（SPDHG）算法在凸泛函及任意随机采样下的几乎必然收敛性，并将其应用于并行MRI重建，发现采样策略能显著影响其收敛速度且存在最优采样。
On Unifying Randomized Methods For Inverse Problems
本文通过随机优化框架统一分析多种反问题随机方法，证明其渐近与非渐近收敛性，并发现新方法，数值实验验证了理论结果。
Convergence of the randomized Kaczmarz method for phase retrieval
本文研究了随机Kaczmarz方法在相位恢复问题中的收敛性，证明了在高概率下，随机测量系统能保证均方收敛，收敛速度与线性设置相当。
Linear constrained convex programming has many practical applications, including support vector machine and machine learning portfolio problems. We propose the randomized primal-dual coordinate (RPDC) method, a randomized coordinate extension of the first-order primal-dual method by Cohen and Zhu, 1984 and Zhao and Zhu, 2019, to solve linear constrained convex programming. We randomly choose a block of variables based on a uniform distribution, linearize, and apply a Bregman-like function (core function) to the selected block to obtain simple parallel primal-dual decomposition. We then establi…
In this paper, we study randomized and cyclic coordinate descent for convex unconstrained optimization problems. We improve the known convergence rates in some cases by using the numerical semidefinite programming performance estimation method. As a spin-off we provide a method to analyse the worst-case performance of the Gauss-Seidel iterative method for linear systems where the coefficient matrix is positive semidefinite with a positive diagonal.
Last-Iterate Convergence of Randomized Kaczmarz and SGD with Greedy Step Size
研究SGD在贪心步长下对光滑二次插值问题的最后轮收敛，证明t次迭代达到O(1/t^{3/4})收敛速度，改进了Attia等人提出的O(1/t^{1/2})保证。
The importance of an adequate inner loop starting point (as opposed to a sufficient inner loop stopping rule) is discussed in the context of a numerical optimization algorithm consisting of nested primal-dual proximal-gradient iterations. While the number of inner iterations is fixed in advance, convergence of the whole algorithm is still guaranteed by virtue of a warm-start strategy for the inner loop, showing that inner loop "starting rules" can be just as effective as "stopping rules" for guaranteeing convergence. The algorithm itself is applicable to the numerical solution of convex optimi…
Time Consistent Stopping For The Mean-Standard Deviation Problem --- The Discrete Time Case
本文研究了均值-标准差问题的时间一致停止策略，通过子博弈完美纳什均衡确定无遗憾的时间一致策略，并探讨了清算策略在该问题中的适用性及最优性。
Convergence Analysis of Randomized SGDA under NC-PL Condition for Stochastic Minimax Optimization Problems
本文提出新分析框架，在NC-PL条件下改进RSGDA算法收敛性结果，并引入加速策略，在合成与真实数据上验证效率。
The iterative ensemble Kalman filter (IEnKF) is widely used in inverse problems to estimate system parameters from limited observations. However, the IEnKF, when applied to nonlinear systems, can be plagued by poor convergence. Here we provide a comprehensive convergence analysis of the IEnKF and propose a new method to improve its convergence. A theoretical analysis of the standard IEnKF is presented and we demonstrate that the interaction between the nonlinearity of the forward model and the diminishing effect of the Kalman updates results in "early stopping" of the IEnKF, i.e. the Kalman op…
Stability and convergence analysis of AdaGrad for non-convex optimization via novel stopping time-based techniques
本文针对非凸优化中 AdaGrad 理论分析不足的问题，引入概率论中的新型停时技术，在温和条件下建立了其稳定性、几乎必然与均方渐近收敛性，并证明了以期望平均平方梯度衡量的近最优非渐近收敛速率，填补了现有理论空白。
In this paper, we propose an extension of the forward improvement iteration algorithm, originally introduced in Irle (2006) and recently reconsidered in Miclo and Villeneuve (2021). The main new ingredient is a flexible window parameter describing the look-ahead distance in the improvement step. We consider the framework of a Markovian optimal stopping problem in discrete time with random discounting and infinite time horizon. We prove convergence and show that the additional flexibility may significantly reduce the runtime.
In this paper we consider discrete and continuous time risk sensitive optimal stopping problem. Using suitable properties of the underlying Feller-Markov process we prove continuity of the optimal stopping value function and provide formula for the optimal stopping policy. Next, we show how to link continuous time framework with its discrete time analogue. By considering suitable approximations, we obtain uniform convergence of the corresponding value functions.
Stochastic Gradient Descent in Non-Convex Problems: Asymptotic Convergence with Relaxed Step-Size via Stopping Time Methods
针对非凸问题中SGD步长方案多样但现有收敛分析依赖强假设的问题，本文提出基于停止时间方法的分析框架，在更松弛的步长条件和弱假设下证明SGD迭代的几乎必然及$L_2$收敛性，提升理论结果的一般性与实用性。
Stability and Convergence of a Randomized Model Predictive Control Strategy
本文针对用随机批次法加速迭代求解的高效模型预测控制变体RBM-MPC，推导了无约束线性系统下的稳定性与收敛性估计，并通过数值算例验证了估计结果及该方法的计算优势。
We study a class of optimal stopping games (Dynkin games) of preemption type, with uncertainty about the existence of competitors. The set-up is well-suited to model, for example, real options in the context of investors who do not want to publicly reveal their interest in a certain business opportunity. We show that there exists a Nash equilibrium in randomized stopping times which is described explicitly in terms of the corresponding one-player game.
Randomized Kaczmarz Methods with Beyond-Krylov Convergence
针对传统随机Kaczmarz方法收敛慢于Krylov子空间方法的问题，提出带自适应动量等改进的Kaczmarz++算法，可利用离群奇异值实现超Krylov收敛，半正定系统变体CD++运算效率与CG、GMRES相当。
Convergence Analysis of Randomized Subspace Normalized SGD under Heavy-Tailed Noise
本文提出随机子空间归一化SGD方法，在重尾噪声下实现高概率收敛界，并优于全维度归一化SGD的oracle复杂度。
We study the problem of stopping a Brownian motion at a given distribution $ν$ while optimizing a reward function that depends on the (possibly randomized) stopping time and the Brownian motion. Our first result establishes that the set $\mathcal{T}(ν)$ of stopping times embedding $ν$ is weakly dense in the set $\mathcal{R}(ν)$ of randomized embeddings. In particular, the optimal Skorokhod embedding problem over $\mathcal{T}(ν)$ has the same value as the relaxed one over $\mathcal{R}(ν)$ when the reward function is semicontinuous, which parallels a fundamental result about Monge maps and Kanto…
Convergence of a Randomized Newton Method in Nonconvex Optimization
本文分析了一种结合牛顿算法与状态相关高斯噪声的随机牛顿方法，用于非凸优化中定位唯一全局极小值点，并证明了有界域下的全局几乎必然收敛性。
Theoretical analysis of the randomized subspace regularized Newton method for non-convex optimization
本文提出随机子空间正则化牛顿法用于非凸优化，证明其全局收敛且速率与完整正则化牛顿法相同，并给出局部线性收敛速率及Hessian秩亏时的超线性收敛结果。
The problem of minimizing a separable convex function under linearly coupled constraints arises from various application domains such as economic systems, distributed control, and network flow. The main challenge for solving this problem is that the size of data is very large, which makes usual gradient-based methods infeasible. Recently, Necoara, Nesterov, and Glineur [Journal of Optimization Theory and Applications, 173 (2017) 227-2254] proposed an efficient randomized coordinate descent method to solve this type of optimization problems and presented an appealing convergence analysis. In th…
Inexact Bregman Proximal Gradient Method and its Inertial Variant with Absolute and Partial Relative Stopping Criteria
针对BPGM子问题精确求解不现实的问题，本文开发带绝对/部分相对停止准则的不精确版本iBPGM及惯性变体v-iBPGM，建立收敛速率并通过数值实验验证其在不同不精确性设置下的收敛行为。
Major-Minor Mean Field Game of Stopping: An Entropy Regularization Approach
本文针对离散时间 major-minor 平均场博弈停时问题，提出熵正则化方法构造辅助问题，证明松弛均衡存在性，并推广至次要参与者选最优控制的情形。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。