solidot新版网站常见问题,请点击这里查看。
消息
本文已被查看5509次
Communication-Computation Efficient Gradient Coding. (arXiv:1802.03475v1 [stat.ML])
来源于:arXiv
This paper develops coding techniques to reduce the running time of
distributed learning tasks. It characterizes the fundamental tradeoff to
compute gradients (and more generally vector summations) in terms of three
parameters: computation load, straggler tolerance and communication cost. It
further gives an explicit coding scheme that achieves the optimal tradeoff
based on recursive polynomial constructions, coding both across data subsets
and vector components. As a result, the proposed scheme allows to minimize the
running time for gradient computations. Implementations are made on Amazon EC2
clusters using Python with mpi4py package. Results show that the proposed
scheme maintains the same generalization error while reducing the running time
by $32\%$ compared to uncoded schemes and $23\%$ compared to prior coded
schemes focusing only on stragglers (Tandon et al., ICML 2017). 查看全文>>