巴拿赫空间中的自适应绍德尔随机镜像下降法

中文摘要 AI 辅助

本文将随机镜像下降（SMD）扩展到无限维巴拿赫空间，用于求解一类仅能通过采样获取随机梯度信息的风险泛函最小化问题。我们首先根据巴拿赫空间的一致凸性性质选择Bregman距离；对于非一致凸的ℒ¹空间，则构造由熵函数诱导的Bregman距离。基于巴拿赫空间的绍德尔基，我们引入一族适配样本量n的有限维子空间，在每次SMD迭代时将子问题限制在对应有限维子空间，并将随机梯度投影到相关有限维对偶空间，从而在巴拿赫空间中引入自适应正则化，该策略可高效求解SMD子问题。通过构建新的分析框架，我们证明所提算法的收敛速率为𝒪(n⁻¹/ᵖ¹)（忽略对数因子），其中p₁≥2由基础空间的凸性性质决定。在模型设定错误场景下（极小值点仅满足较弱正则性条件），我们进一步证明风险泛函仍收敛到其最小值。此外，处理n个样本仅需𝒪(n¹⁺ᵠ)的计算时间和𝒪(nᵠ)的内存，当极小值点具有足够正则性时，可选取任意小的θ>0。我们将该算法应用于求解统计逆问题，并通过数值实验验证其有效性。

英文摘要

In this paper, we introduce an adaptive regularization strategy for stochastic mirror descent (SMD) to solve a class of risk functional minimization problems in infinite-dimensional Banach spaces. This regularization strategy centers on using a Schauder basis to construct a nested family of finite-dimensional subspaces, with the dimension chosen adaptively according to the sample size $n$. We then restrict each SMD subproblem to the corresponding subspace and project the stochastic gradient onto its dual space. This yields closed-form solutions to the SMD subproblems and coordinate-wise updates of the basis coefficients, enabling an implementation with low computational and storage complexity. The subspace dimension also serves as a regularization parameter that balances approximation and optimization errors. For risk functional minimization in $\mathcal{L}^p$ spaces with $1<p<\infty$, we construct Bregman distances adapted to the geometry of the underlying Banach spaces using $\max\{2,p\}$-convex functionals induced by their uniform convexity. At the non-uniformly convex $\mathcal{L}^1$ endpoint, we instead construct a locally strongly convex functional based on the entropy function. By developing a new analytical framework, we establish a convergence rate of $\mathcal O\left(n^{-\min\{\frac12,\frac1p\}}\right)$, up to logarithmic factors. In the misspecified setting, where the minimizer satisfies only weaker regularity conditions, we prove that the risk functional still converges to its minimum value. Finally, we apply the method to statistical inverse problems and illustrate its empirical performance through numerical experiments in both settings.

发表机构

• Fudan University（复旦大学）

机构由 AI 辅助整理，请以论文原文为准。

补充信息

备注

37 pages, 7 figures