S-Y. Peng, W. Xu, C. Cornelius, M. Hull, K. Li, R. Duggal, M. Phute, J. Martin, D.H. Chau
Georgia Institute of Technology, Georgia, United States
Keywords: adversarial attacks; adversarial robustness; robust architecture; robust principles
Convolutional neural networks (CNNs) are staples in computer vision research, but they are vulnerable to adversarial attacks where the attacker can induce arbitrary outputs of their choosing. Despite the large amount of research that aims to address such vulnerabilities, there are still conflicting opinions on how architectural components impact the overall robustness. We aim to unify existing works' diverging opinions on how architectural components affect the adversarial robustness of CNNs. To accomplish our goal, we synthesize a suite of three generalizable robust architectural design principles: (a) optimal range for depth and width configurations, (b) preferring convolutional over patchify stem stage, and (c) robust residual block design through adopting squeeze and excitation blocks and non-parametric smooth activation functions. Through extensive experiments across a wide spectrum of dataset scales, adversarial training methods, model parameters, and network design spaces, our principles consistently and markedly improve AutoAttack accuracy: 1-3 percentage points (pp) on CIFAR-10 and CIFAR-100, and 4-9 pp on ImageNet. The code is publicly available at https://github.com/poloclub/robust-principles.