arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1206.4627cs.LGstat.ML
收藏

稀疏Ising模型有偏随机优化的收敛速度

Convergence Rates of Biased Stochastic Optimization for Learning Sparse Ising Models

Stony Brook University(石溪大学)

Jean Honorio

摘要:

我们研究了精确(NP-困难)目标的随机优化的收敛速度,其中只有有偏估计的梯度。我们激励这个问题的背景下学习的伊辛模型的结构和参数。首先,我们提供了一个收敛速度分析的确定性错误的前向-后向分裂(FBS)。然后,我们扩展我们的分析有偏随机误差,首先表征一个家庭的采样器,并提供一个高概率的界限,不仅允许理解FBS,但也近端梯度(PG)的方法。我们得出一些有趣的结论:FBS只需要一个grammically增加随机样本的数量,以收敛(虽然在一个非常低的速度);所需的随机样本的数量是相同的确定性和有偏随机设置FBS和基本PG;加速PG不能保证收敛于有偏随机设置。

英文摘要:

We study the convergence rate of stochastic optimization of exact (NP-hard) objectives, for which only biased estimates of the gradient are available. We motivate this problem in the context of learning the structure and parameters of Ising models. We first provide a convergence-rate analysis of deterministic errors for forward-backward splitting (FBS). We then extend our analysis to biased stochastic errors, by first characterizing a family of samplers and providing a high probability bound that allows understanding not only FBS, but also proximal gradient (PG) methods. We derive some interesting conclusions: FBS requires only a logarithmically increasing number of random samples in order to converge (although at a very low rate); the required number of random samples is the same for the deterministic and the biased stochastic setting for FBS and basic PG; accelerated PG is not guaranteed to converge in the biased stochastic setting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑