Advances and Challenges in Large Model Compression: A Survey

Abstract

With the development of computer technology, large-scale models have gradually evolved from early simple algorithms to today's complex deep neural networks with huge parameter scale and high computational complexity. These large-scale models have greatly advanced the artificial intelligence technology, which has realized revolutionary progress in the fields of image, speech, text and so on. However, the computational and storage requirements of large-scale models are also growing rapidly, which seriously constrains their deployment and application in computing resource-constrained environments. In order to solve this problem, model compression technology has emerged and has been greatly developed. In this paper, we systematically review the key technologies, application scenarios, challenges and future directions of large-scale model compression from the perspective of model compression technology. Our work aims to provide practitioners and researchers with detailed references on the development of the technology, and emphasizes the important role of model compression technology in promoting the development of lightweight artificial intelligence.