Stochastic control up to a hitting time: optimality and rolling-horizon implementation

Some new results on sample path optimality in ergodic control of diffusions

We present some new results on sample path optimality for the ergodic control problem of a class of non-degenerate diffusions controlled through the drift. The hypothesis most often used in the literature to ensure the existence of an a.s. sample path optimal stationary Markov control requires finite second moments of the first hitting times $τ$ of bounded domains over all admissible controls. We show that this can be considerably weakened: ${\mathrm E}[τ^2]$ may be replaced with ${\mathrm E}[τ\ln^+(τ)]$, thus reducing the required rate of convergence of averages from polynomial to logarithmic…

Optimality of the final model found via Stochastic Gradient Descent

We study convergence properties of Stochastic Gradient Descent (SGD) for convex objectives without assumptions on smoothness or strict convexity. We consider the question of establishing that with high probability the objective evaluated at the candidate minimizer returned by SGD is close to the minimal value of the objective. We compare this result concerning the final candidate minimzer (i.e. the final model parameters learned after all gradient steps) to the online learning techniques of [Zin03] that take a rolling average of the model parameters at the different steps of SGD.

Revisiting Smoothed Online Learning

In this paper, we revisit the problem of smoothed online learning, in which the online learner suffers both a hitting cost and a switching cost, and target two performance metrics: competitive ratio and dynamic regret with switching cost. To bound the competitive ratio, we assume the hitting cost is known to the learner in each round, and investigate the simple idea of balancing the two costs by an optimization problem. Surprisingly, we find that minimizing the hitting cost alone is $\max(1, \frac{2}α)$-competitive for $α$-polyhedral functions and $1 + \frac{4}λ$-competitive for $λ$-quadratic …

奏响高维音符：广义线性模型与多指标模型上 SGD 学习动力学的 ODE

Hitting the High-Dimensional Notes: An ODE for SGD learning dynamics on GLMs and multi-index models

本文研究高维广义线性与多指标模型中流式 SGD 的动力学，提出 ODE 确定性等价与均匀化 SDE，以推导稳定性阈值、收敛保证及一般统计量演化。

最优去中心化平滑在线凸优化

Optimal Decentralized Smoothed Online Convex Optimization

针对多智能体平滑在线凸优化问题，提出了首个可证明渐近最优的去中心化算法ACORD，仅依赖局部信息即可实现低计算复杂度，且性能优于现有LPC算法。

Optimal stopping time on discounted semi-Markov processes

This paper attempts to study the optimal stopping time for semi-Markov processes (SMPs) under the discount optimization criteria with unbounded cost rates. In our work, we introduce an explicit construction of the equivalent semi-Markov decision processes (SMDPs). The equivalence is embodied in the value functions of SMPs and SMDPs, that is, every stopping time of SMPs can induce a policy of SMDPs such that the value functions are equal, and vice versa. The existence of the optimal stopping time of SMPs is proved by this equivalence relation. Next, we give the optimality equation of the value …