arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.05959cs.LOcs.LGcs.PLmath.PR
收藏

随机逼近定理的形式化

Formalization of a Stochastic Approximation Theorem

University of Pittsburgh(匹兹堡大学);IBM Research(IBM研究院);IHÉS(法国高等科学研究所)

Koundinya Vajjha, Barry Trager, Avraham Shinnar, Vasily Pestun

摘要:

随机近似算法是在目标未知且直接观测被噪声破坏的环境中用于近似目标值的迭代过程。这些算法是有用的,例如,当目标函数或模型不直接已知时,用于寻根和函数最小化。随机逼近最初是由Robbins和Monro在1951年的一篇论文中提出的,该领域已经有了巨大的发展,并影响了从自适应信号处理到人工智能的应用领域。作为一个例子,随机梯度下降算法是基于随机逼近理论的,它在机器学习的各个子域中普遍存在。在本文中,我们给出了一个正式的证明(在Coq证明助手)的一般收敛性定理,由于Aryeh Dvoretzky,这意味着收敛的重要经典方法,如Robbins-Monro和Kiefer-Wolfowitz算法。在这个过程中,我们建立了一个全面的Coq图书馆的测度论概率论和随机过程。

英文摘要:

Stochastic approximation algorithms are iterative procedures which are used to approximate a target value in an environment where the target is unknown and direct observations are corrupted by noise. These algorithms are useful, for instance, for root-finding and function minimization when the target function or model is not directly known. Originally introduced in a 1951 paper by Robbins and Monro, the field of Stochastic approximation has grown enormously and has come to influence application domains from adaptive signal processing to artificial intelligence. As an example, the Stochastic Gradient Descent algorithm which is ubiquitous in various subdomains of Machine Learning is based on stochastic approximation theory. In this paper, we give a formal proof (in the Coq proof assistant) of a general convergence theorem due to Aryeh Dvoretzky, which implies the convergence of important classical methods such as the Robbins-Monro and the Kiefer-Wolfowitz algorithms. In the process, we build a comprehensive Coq library of measure-theoretic probability theory and stochastic processes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑