Adversarial Robustness vs Model Compression, or Both?



  • Shaokai Ye
  • Kaidi Xu
  • Sijia Liu
  • Hao Cheng
  • Jan-henrik Lambrechts
  • Huan Zhang
  • Aojun Zhou
  • Kaisheng Ma
  • Yanzhi Wang
  • Xue Lin

Published on


It is well known that deep neural networks (DNNs) are vulnerable to adversarial attacks, which are implemented by adding crafted perturbations onto benign examples. Min-max robust optimization based adversarial training can provide a notion of security against adversarial attacks. However, adversarial robustness requires a significantly larger capacity of the network than that for the natural training with only benign examples. This paper proposes a framework of concurrent adversarial training and weight pruning that enables model compression while still preserving the adversarial robustness and essentially tackles the dilemma of adversarial training. Furthermore, this work studies two hypotheses about weight pruning in the conventional setting and finds that weight pruning is essential for reducing the network model size in the adversarial setting, training a small model from scratch even with inherited initialization from the large model cannot achieve both adversarial robustness and high standard accuracy.

Please cite our work using the BibTeX below.

  author    = {Shaokai Ye and
               Kaidi Xu and
               Sijia Liu and
               Hao Cheng and
               Jan{-}Henrik Lambrechts and
               Huan Zhang and
               Aojun Zhou and
               Kaisheng Ma and
               Yanzhi Wang and
               Xue Lin},
  title     = {Second Rethinking of Network Pruning in the Adversarial Setting},
  journal   = {CoRR},
  volume    = {abs/1903.12561},
  year      = {2019},
  url       = {},
  archivePrefix = {arXiv},
  eprint    = {1903.12561},
  timestamp = {Tue, 02 Apr 2019 12:29:45 +0200},
  biburl    = {},
  bibsource = {dblp computer science bibliography,}

Close Modal