An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices



  • Xiaolong Ma
  • Wei Niu
  • Tianyun Zhang
  • Sijia Liu
  • Sheng Lin
  • Hongjia Li
  • Xiang Chen
  • Jian Tang
  • Kaisheng Ma
  • Bin Ren
  • Yanzhi Wang

Published on


Weight pruning has been widely acknowledged as a straightforward and effective method to eliminate redundancy in Deep Neural Networks (DNN), thereby achieving acceleration on various platforms. However, most of the pruning techniques are essentially trade-offs between model accuracy and regularity which lead to impaired inference accuracy and limited on-device acceleration performance. To solve the problem, we introduce a new sparsity dimension, namely pattern-based sparsity that comprises pattern and connectivity sparsity, and becoming both highly accurate and hardware friendly. With carefully designed patterns, the proposed pruning unprecedentedly and consistently achieves accuracy enhancement and better feature extraction ability on different DNN structures and datasets, and our pattern-aware pruning framework also achieves pattern library extraction, pattern selection, pattern and connectivity pruning and weight training simultaneously. Our approach on the new pattern-based sparsity naturally fits into compiler optimization for highly efficient DNN execution on mobile platforms. To the best of our knowledge, it is the first time that mobile devices achieve real-time inference for the large-scale DNN models thanks to the unique spatial property of pattern-based sparsity and the help of the code generation capability of compilers.

This paper has been published at ECCV 2020

Please cite our work using the BibTeX below.

      title={An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices}, 
      author={Xiaolong Ma and Wei Niu and Tianyun Zhang and Sijia Liu and Sheng Lin and Hongjia Li and Xiang Chen and Jian Tang and Kaisheng Ma and Bin Ren and Yanzhi Wang},
      journal={European Conference on Computer Vision (ECCV)},
Close Modal