solidot新版网站常见问题,请点击这里查看。
消息
本文已被查看4037次
A note on semi-groups of stochastic gradient descent and online principal component analysis. (arXiv:1712.06509v1 [math.PR])
来源于:arXiv
We study the Markov semigroups for two important algorithms from machine
learning: stochastic gradient descent (SGD) and online principal component
analysis (PCA). We investigate the effects of small jumps on the properties of
the semi-groups. Properties including regularity preserving, $L^{\infty}$
contraction are discussed. These semigroups are the dual of the semigroups for
evolution of probability, while the latter are $L^{1}$ contracting and
positivity preserving. Using these properties, we show that stochastic
differential equations (SDEs) in $\mathbb{R}^d$ (on the sphere
$\mathbb{S}^{d-1}$) can be used to approximate SGD (online PCA) weakly. These
SDEs may be used to provide some insights of the behaviors of these algorithms. 查看全文>>