## Deep Neural Network Approximation Theory. (arXiv:1901.02220v1 [cs.LG])

Deep neural networks have become state-of-the-art technology for a wide range
of practical machine learning tasks such as image classification, handwritten
digit recognition, speech recognition, or game intelligence. This paper
develops the fundamental limits of learning in deep neural networks by
characterizing what is possible if no constraints on the learning algorithm and
the amount of training data are imposed. Concretely, we consider
information-theoretically optimal approximation through deep neural networks
with the guiding theme being a relation between the complexity of the function
(class) to be approximated and the complexity of the approximating network in
terms of connectivity and memory requirements for storing the network topology
and the associated quantized weights. The theory we develop educes remarkable
universality properties of deep networks. Specifically, deep networks are
optimal approximants for vastly different function classes such as affine
systems and Gabor查看全文