## Coded Distributed Computing with Node Cooperation Substantially Increases Speedup Factors. (arXiv:1802.04172v1 [cs.IT])

This work explores a distributed computing setting where \$K\$ nodes are assigned fractions (subtasks) of a computational task in order to perform the computation in parallel. In this setting, a well-known main bottleneck has been the inter-node communication cost required to parallelize the task, because unlike the computational cost which could keep decreasing as \$K\$ increases, the communication cost remains approximately constant, thus bounding the total speedup gains associated to having more computing nodes. This bottleneck was substantially ameliorated by the recent introduction of coded MapReduce techniques which allowed each node --- at the computational cost of having to preprocess approximately \$t\$ times more subtasks --- to reduce its communication cost by approximately \$t\$ times. In reality though, the associated speed up gains were severely limited by the requirement that larger \$t\$ and \$K\$ necessitated that the original task be divided into an extremely large number of subt查看全文

## Solidot 文章翻译

 你的名字 留空匿名提交 你的Email或网站 用户可以联系你 标题 简单描述 内容 This work explores a distributed computing setting where \$K\$ nodes are assigned fractions (subtasks) of a computational task in order to perform the computation in parallel. In this setting, a well-known main bottleneck has been the inter-node communication cost required to parallelize the task, because unlike the computational cost which could keep decreasing as \$K\$ increases, the communication cost remains approximately constant, thus bounding the total speedup gains associated to having more computing nodes. This bottleneck was substantially ameliorated by the recent introduction of coded MapReduce techniques which allowed each node --- at the computational cost of having to preprocess approximately \$t\$ times more subtasks --- to reduce its communication cost by approximately \$t\$ times. In reality though, the associated speed up gains were severely limited by the requirement that larger \$t\$ and \$K\$ necessitated that the original task be divided into an extremely large number of subt