solidot新版网站常见问题,请点击这里查看。
消息
本文已被查看1187次
A convergence analysis of the perturbed compositional gradient flow: averaging principle and normal deviations. (arXiv:1709.00515v2 [math.PR] UPDATED)
来源于:arXiv
We consider in this work a system of two stochastic differential equations
named the perturbed compositional gradient flow. By introducing a separation of
fast and slow scales of the two equations, we show that the limit of the slow
motion is given by an averaged ordinary differential equation. We then
demonstrate that the deviation of the slow motion from the averaged equation,
after proper rescaling, converges to a stochastic process with Gaussian inputs.
This indicates that the slow motion can be approximated in the weak sense by a
standard perturbed gradient flow or the continuous-time stochastic gradient
descent algorithm that solves the optimization problem for a composition of two
functions. As an application, the perturbed compositional gradient flow
corresponds to the diffusion limit of the Stochastic Composite Gradient Descent
(SCGD) algorithm for minimizing a composition of two expected-value functions
in the optimization literatures. For the strongly convex case, such an an 查看全文>>