Survey of Deep Neural Networks Model Compression

Abstract

In recent years, the deep neural networks have gained more and more attention with the rapid development of deep learning. It has achieved remarkable effect in many application fields. Usually, at a higher computation, the learning ability of deep neural networks is improved with the increase of depth, which makes the performance of deep learning on large datasets especially successful. However, the deep learning can??t be effectively applied to the lightweight mobile portable device due to the characteristics of large amount of calculation, high storage cost and complicated model. Therefore, compressing and simplifying the deep learning model has become the research hot spot. Currently, the main model compression methods include pruning, lightweight network design, knowledge distillation, quantization, neural architecture search, etc. This paper analyses and summarizes the performance, advantages and limitations and the latest research results of the model compression methods, and prospects the future research direction.