Yash Sharma, Gavin Weiguang Ding, and Marcus A. Brubaker ... · Yash Sharma, Gavin Weiguang Ding, and Marcus A. Brubaker Borealis AI Abstract Carefully crafted, often imperceptible,

On the Effectiveness of Low Frequency Perturbations

Yash Sharma, Gavin Weiguang Ding, and Marcus A. Brubaker

Borealis AI

Abstract

Carefully crafted, often imperceptible, adversarialperturbations have been shown to cause state-of-the-art models to yield extremely inaccurate out-puts, rendering them unsuitable for safety-criticalapplication domains. In addition, recent work hasshown that constraining the attack space to a lowfrequency regime is particularly effective. Yet, itremains unclear whether this is due to generallyconstraining the attack search space or specificallyremoving high frequency components from con-sideration. By systematically controlling the fre-quency components of the perturbation, evaluatingagainst the top-placing defense submissions in theNeurIPS 2017 competition, we empirically showthat performance improvements in both the white-box and black-box transfer settings are yielded onlywhen low frequency components are preserved.In fact, the defended models based on adversar-ial training are roughly as vulnerable to low fre-quency perturbations as undefended models, sug-gesting that the purported robustness of state-of-the-art ImageNet defenses is reliant upon adver-sarial perturbations being high frequency in nature.We do find that under `∞ ε = 16/255, the competi-tion distortion bound, low frequency perturbationsare indeed perceptible. This questions the use of the`∞-norm, in particular, as a distortion metric, and,in turn, suggests that explicitly considering the fre-quency space is promising for learning robust mod-els which better align with human perception.

1 IntroductionDespite the impressive performance deep neural networkshave shown, researchers have discovered that they are, insome sense, ‘brittle’; small carefully crafted ‘adversarial’ per-turbations to their inputs can result in wildly different outputs[Szegedy et al., 2013]. Even worse, these perturbations havebeen shown to transfer: learned models can be successfullymanipulated by adversarial perturbations generated by attack-ing distinct models. An attacker can discover a model’s vul-nerabilities even without access to it.

The goal of this paper is to investigate the relationshipbetween a perturbation’s frequency properties and its effec-tiveness, and is motivated by recent work showing the effec-tiveness of low frequency perturbations in particular. Guoet al. [2018] shows that constraining the perturbation to thelow frequency subspace improves the query efficiency of thedecision-based gradient-free boundary attack [Brendel et al.,2017]. Zhou et al. [2018] achieves improved transferabilityby suppressing high frequency components of the perturba-tion. Similarly, Sharma et al. [2018] applied a 2D Gaussianfilter on the gradient w.r.t. the input image during the iterativeoptimization process to win the CAAD 2018 competition1.

However, two questions still remain unanswered:

1. is the effectiveness of low frequency perturbations sim-ply due to the reduced search space or specifically dueto the use of low frequency components? and

2. under what conditions are low frequency perturbationsmore effective than unconstrained perturbations?

To answer these questions, we design systematic exper-iments to test the effectiveness of perturbations manipulat-ing specified frequency components, utilizing the discrete co-sine transform (DCT). Testing against state-of-the-art Ima-geNet [Deng et al., 2009] defense methods, we show that,when perturbations are constrained to the low frequency sub-space, they are 1) generated faster; and are 2) more trans-ferable. These results mirror the performance obtained whenapplying spatial smoothing or downsampling-upsampling op-erations. However, if perturbations are constrained to otherfrequency subspaces, they perform worse in general. Thisconfirms that the effectiveness of low frequency perturbationsis due to the application of a low-pass filter in the frequencydomain of the perturbation rather than a general reduction inthe dimensionality of the search space.

On the other hand, we also notice that the improved effec-tiveness of low frequency perturbations is only significant fordefended models, but not for clean models. In fact, the state-of-the-art ImageNet defenses in test are roughly as vulner-able to low frequency perturbations as undefended models,suggesting that their purported robustness is reliant upon theassumption that adversarial perturbations are high frequency

1Competition on Adversarial Attacks and Defenses: http://hof.geekpwn.org/caad/en/index.html

arX

iv:1

903.

0007

3v2

[cs

.CV

] 1

Jun

201

9

in nature. As we show, this issue is not shared by the state-of-the-art on CIFAR-10 [Madry et al., 2017], as the dataset is toolow-dimensional for there to be a diverse frequency spectrum.Finally, based on the perceptual difference between the un-constrained and low frequency attacks, we discuss the prob-lem of using the commonly used `∞ norm as a perceptualmetric for quantifying robustness, illustrating the promise inutilizing frequency properties to learn robust models whichbetter align with human perception.

2 BackgroundGenerating adversarial examples is an optimization problem,while generating transferable adversarial examples is a gener-alization problem. The optimization variable is the perturba-tion, and the objective is to fool the model, while constrain-ing (or minimizing) the magnitude of the perturbation. `pnorms are typically used to quantify the strength of the per-turbation; though they are well known to be poor perceptualmetrics [Zhang et al., 2018]. Constraint magnitudes used inpractice are assumed to be small enough such that the ball isa subset of the imperceptible region.

Adversarial perturbations can be crafted in not only thewhite-box setting [Carlini and Wagner, 2017b; Chen et al.,2017a] but in limited access settings as well [Chen et al.,2017b; Alzantot et al., 2018a], when solely query access is al-lowed. When even that is not possible, attacks operate in theblack-box setting, and must rely on transferability. Finally,adversarial perturbations are not a continuous phenomenon,recent work has shown applications in discrete settings (e.g.natural language) [Alzantot et al., 2018b; Lei et al., 2018].

Numerous approaches have been proposed as defenses, tolimited success. Many have been found to be easily circum-vented [Carlini and Wagner, 2017a; Sharma and Chen, 2018;Athalye et al., 2018], while others have been unable to scaleto high-dimensional complex datasets, e.g. ImageNet [Smithand Gal, 2018; Papernot and McDaniel, 2018; Li et al., 2018;Schott et al., 2018]. Adversarial training, training the modelwith adversarial examples [Goodfellow et al., 2014; Trameret al., 2017; Madry et al., 2017; Ding et al., 2018], hasdemonstrated improvement, but is limited to the propertiesof the perturbations used, e.g. training exclusively on `∞does not provide robustness to perturbations generated underother distortion metrics [Sharma and Chen, 2017; Schott etal., 2018]. In the NeurIPS 2017 ImageNet competition, win-ning defenses built upon these trained models to reduce theirvulnerabilities [Kurakin et al., 2018; Xie et al., 2018].

3 Methods3.1 AttacksWe consider `∞-norm constrained perturbations, where theperturbation δ satisfies ‖δ‖∞ ≤ ε with ε being the maxi-mum perturbation magnitude, as the NeurIPS 2017 competi-tion bounded δ with ε = 16. The Fast Gradient Sign Method(FGSM) [Goodfellow et al., 2014] provides a simple, one-step gradient-based perturbation of `∞ ε size as follows:

δFGSM = s · ε · sign(∇xJ(x, y)) (1)

DCT_Low DCT_Mid DCT_High DCT_Random

Figure 1: Masks used to constrain the frequency space where n =128 and d = 299 (ImageNet). Red denotes frequency componentsof the perturbation which will be masked when generating the ad-versarial example, both during and after the optimization process.

Cln 1 [InceptionV3]

Cln 3[InceptionV3, InceptionV4,

ResNetV2 101]Adv 1 [AdvInceptionV3]

Adv 3[AdvInceptionV3, Ens3AdvInceptionV3,

Ens4AdvInceptionV4]

Table 1: Models used for generating black-box transfer attacks.

where x is the input image, J is the classification loss func-tion, sign(·) is the element-wise sign function2. When y is thetrue label of x and s = +1, δ is the non-targeted attack formisclassification; when y is a target label other than the truelabel of x and s = −1, δ is the targeted attack for manipulat-ing the network to wrongly predict y.

FGSM suffers from an “underfitting” problem when ap-plied to non-linear loss function, as its formulation is de-pendent on a linearization of J about x. The Basic IterativeMethod (BIM) [Kurakin et al., 2016; Madry et al., 2017], oth-erwise known as PGD (without random starts), runs FGSMfor multiple iterations to rectify this problem. The top-placingattack in the previously mentioned NeurIPS 2017 competi-tion, the Momentum Iterative Method (MIM) [Dong et al.,2017], replaces the gradient ∇xJ(x, y) with a “momentum”term to prevent the “overfitting” problem, caused by poor lo-cal optima, in order to improve transferability. Thus, we usethis method for our NeurIPS 2017 defense evaluation.

3.2 Frequency ConstraintsOur goal is to examine whether the effectiveness of low fre-quency perturbations is due to a reduced search space in gen-eral or due to the specific use of a low-pass filter in the fre-quency domain of the perturbation. To achieve this, we usethe discrete cosine transform (DCT) [Rao and Yip, 2014] toconstrain the perturbation to only modify certain frequencycomponents of the input.

The DCT decomposes a signal into cosine wave compo-nents with different frequencies and amplitudes. Given a 2Dimage (or perturbation) x ∈ Rd×d, the DCT Transform of xis v = DCT(x), where the entry vi,j is the magnitude of itscorresponding basis functions.

The numerical values of i and j represent the frequen-cies, i.e. smaller values represent lower frequencies andvice versa. The DCT is invertible, with an inverse transform

2sign = 1 if x > 0, sign = −1 if x < 0, sign = 0, if x = 0.

x = IDCT(v)3.We remove certain frequency components of the perturba-

tion δ by applying a mask to its DCT transform DCT(δ).We then reconstruct the perturbation by applying IDCT onthe masked DCT transform. Specifically, the mask, m ∈{0, 1}d×d, is a 2D matrix image whose pixel values are 0’sand 1’s, and the “masking” is done by element-wise product.

We can then reconstruct the “transformed” perturbation byapplying the IDCT to the masked DCT(δ). The entire trans-formation can then be represented as:

FreqMask(δ) = IDCT(Mask(DCT(δ))) . (2)

Accordingly in our attack, we use the following gradient

∇δJ(x+ FreqMask(δ), y) .

We use 4 different types of FreqMask to constrain theperturbations, as shown in Figure 1. DCT High only pre-serves high frequency components; DCT Low only preserveslow frequency components; DCT Mid only preserves midfrequency components; and DCT Rand preserves randomlysampled components. For reduced dimensionality n, we pre-serve n×n components. Recall that v = DCT(x), DCT Lowpreserves components vi,j if 1 ≤ i, j ≤ n; DCT Highmasks components if 1 ≤ i, j ≤

√d2 − n2; DCT Mid and

DCT Rand also preserve n×n components, the detailed gen-eration processes can be found in the appendix. Figure 1visualizes the masks when d = 299 (e.g. ImageNet) andn = 128. Note that when n = 128, we only preserve1282/2992 ≈ 18.3% of the frequency components, a smallfraction of the original unconstrained perturbation.

4 Results and AnalysesTo evaluate the effectiveness of perturbations under differentfrequency constraints, we test against models and defensesfrom the NeurIPS 2017 Adversarial Attacks and DefencesCompetition [Kurakin et al., 2018].Threat Models: We evaluate attacks in both the non-targeted and targeted case, and measure the attack successrate (ASR) on 1000 test examples from the NeurIPS 2017 de-velopment toolkit4. We test on ε = 16/255 (competition dis-tortion bound) and iterations = [1, 10] for the non-targetedcase; ε = 32/255 and iterations = 10 for the targeted case.The magnitude for the targeted case is larger since targetedattacks, particularly on ImageNet (1000 classes), are signif-icantly harder. As can be seen in Figure 5 and 6, uncon-strained adversarial perturbations generated under these dis-tortion bounds are still imperceptible.Attacks: As described in Section 3, we experiment with theoriginal unconstrained MIM and frequency constrained MIMwith masks shown in Figure 1. For each mask type, we testn = [256, 128, 64, 32] with d = 299. For DCT Rand, weaverage results over 3 random seeds.

To describe the attack settings, we specify model place-holders A and B. We call an attack white-box, when we at-tack model A with the perturbation generated from A itself.

3DCT / IDCT is applied to each color channel independently.4https://www.kaggle.com/c/6864/download/dev toolkit.zip

We call an attack grey-box, when the perturbation is generatedfrom A, but used to attack a “defended” A, where a defensemodule is prepended toA. We call an attack black-box (trans-fer), when the perturbation generated fromA is used to attackdistinct B, where B can be defended or not. Note that this isdistinct from the black-box setting discussed in [Guo et al.,2018], in which query access is allowed.Target Models and Defenses for Evaluation: We evalu-ate each of the attack settings against the top defense solu-tions in the NeurIPS 2017 competition [Kurakin et al., 2018].Each of the top-4 NeurIPS 2017 defenses prepend a tuned(or trained) preprocessor to an ensemble of classifiers, whichfor all of them included the strongest available adversariallytrained model: EnsAdvInceptionResNetV25 [Trameret al., 2017]. Thus, we use EnsAdvInceptionResNetV2to benchmark the robustness6 of adversarially trained models.

We then prepend the preprocessors from the top-4 NeurIPS2017 defenses to EnsAdvInceptionResNetV2, and de-note the defended models as D1, D2, D3, and D4, respec-tively. Regarding the preprocessors, D1 uses a trained de-noiser where the loss function is defined as the difference be-tween the target model’s outputs activated by the clean im-age and denoised image [Liao et al., 2017]; D2 uses randomresizing and random padding [Xie et al., 2017]; D3 uses anumber of image transformations: shear, shift, zoom, and ro-tation [Thomas and Elibol, 2017]; and D4 simply uses mediansmoothing [Kurakin et al., 2018].

For our representative cleanly trained model, we evaluateagainst the state-of-the-art NasNetLarge 3317 [Zoph etal., 2017]. We denote EnsAdvInceptionResNetV2 asEnvAdv and NasNetLarge 331 as NasNet for brevity.Source Models for Perturbation Generation: For white-box attacks, we evaluate perturbations generated from Nas-Net and EnsAdv to attack themselves respectively. For grey-box attacks, we use perturbations generated from EnsAdv toattack D1, D2, D3, and D4 respectively. For black-box at-tacks, since the models for generating the perturbations needto be distinct from the ones being attacked, we use 4 differentsources (ensembles) which vary in ensemble size and whetherthe models are adversarially trained or cleanly trained, asshown in Table 1. In summary, for black-box attacks, per-turbations generated from Adv 1, Adv 3, Cln 1, and Cln 3are used to attack NasNet, EnsAdv, D1, D2, D3, and D4.

4.1 Overview of the ResultsAs described, we test the unconstrained and constrained per-turbations in the white-box, grey-box, and black-box scenar-ios. Representative results are shown in Figure 2a, 2b, 2c,and 2d. In each of these plots, the vertical axis is attacksuccess rate (ASR), while the horizontal indicates the num-ber of frequency components kept (Dimensionality). Un-constrained MIM is shown as a horizontal line across thedimensionality axis for ease of comparison. In each fig-ure, the plots are, from left to right, non-targeted attack with

5https://github.com/tensorflow/models/tree/master/research/adv imagenet models

6EnsAdvInceptionResNetV2 is to be attacked.7https://github.com/tensorflow/models/tree/master/research/slim

256 128 64 32Dimensionality

0

20

40

Atta

ck S

ucce

ss R

ate

(ASR

) Non-Targeted (1)


0

25

50

75

100Non-Targeted (10)


0

20

40

60Targeted (10)

Unconstrained (MIM)DCT_Low

DCT_HighDCT_Random

DCT_Mid

(a) White-box attack on adversarially trained model, EnsAdv.


0

20

40

Atta

ck S

ucce

ss R

ate

(ASR

) Non-Targeted (1)


0

25

50

75



0

20

40

60

Targeted (10)


DCT_HighDCT_Random

DCT_Mid

(b) White-box attack on standard cleanly trained model, NasNet.


0

10

20

30

40

Atta

ck S

ucce

ss R

ate

(ASR

) Non-Targeted (1)


020406080

Non-Targeted (10)


0

10

20

30Targeted (10)


DCT_HighDCT_Random

DCT_Mid

(c) Grey-box attack on top-4 NeurIPS 2017 defenses prepended toadversarially trained model.


0

10

20

30

Atta

ck S

ucce

ss R

ate

(ASR

) Non-Targeted (1)


0

20

40

Non-Targeted (10)


0.0

2.5

5.0

7.5

10.0Targeted (10)


DCT_HighDCT_Random

DCT_Mid

(d) Black-box attack on sources (Table 1) transferred to defenses (En-sAdv + D1∼4)

Figure 2: Number of iterations in parentheses. Non-targeted with ε = 16/255, targeted with ε = 32/255.

iterations = 1, non-targeted with iterations = 10, and tar-geted with iterations = 10. From these figures, we can seethat DCT Low always outperforms the other frequency con-straints, including DCT High, DCT Mid and DCT Rand.

In the appendix, we show results where the perturba-tion is constrained using a spatial smoothing filter or adownsampling-upsampling filter (perturbation resized withbilinear interpolation). The performance mirrors that of Fig-ure 2a, 2b, 2c, and 2d, further confirming that the effective-ness of low frequency perturbations is not due to a general re-striction of search space, but due to the low frequency regimeitself. Thus, in our remaining experiments, we focus on lowfrequency constrained perturbations induced with DCT Low.

We compare ASR and relative changes across all black-box transfer pairs between standard unconstrained MIM andMIM constrained with DCT Low n = 128, on non-targetedattacks with both iterations = 1 and iterations = 10. Thiscomparison is visualized in Figure 3 and 4. We also showthat these results do not transfer to the significantly lower-dimensional CIFAR-10 dataset (d = 32, minimum n usedin ImageNet experiments), as the rich frequency spectrum ofnatural images is no longer present.

4.2 Observations and AnalysesDCT Low generates effective perturbations faster on ad-versarially trained models, but not on cleanly trainedmodels. Figure 2a and 2b show the white-box ASRs onEnsAdv and NasNet respectively. For EnsAdv, we can seethat DCT Low improves ASR in the non-targeted case withiterations = 1 and in the targeted case with iterations = 10,but not in the non-targeted case with iterations = 10. How-ever, in this case, DCT Low still outperforms other frequencyconstraints and does not significantly deviate from uncon-strained MIM’s performance. When the number of iterations

is large enough that unconstrained MIM can succeed consis-tently, constraining the space only limits the attack, but oth-erwise, the low frequency prior is effective. Therefore, lowfrequency perturbations are more “iteration efficient”, as theycan be found more easily with a less exhaustive search, whichis practically helpful computationally.

However, for white-box attacks on NasNet in Figure 2b,we see that although DCT Low still outperforms the otherfrequency constraints, it does perform worse than uncon-strained MIM. Comparing Figure 2a and 2b, it is clear thatDCT Low performs similarly against the adversarially trainedmodel as with the cleanly trained model, the difference hereis due to unconstrained MIM performing significantly betteragainst the cleanly trained model than against the adversari-ally trained model. This implies that the low frequency prioris useful against defended models, in particular, since it ex-ploits the space where adversarial training, which is necessar-ily imperfect, fails to reduce vulnerabilities.

DCT Low bypasses defenses prepended to the adversari-ally trained model. As previously mentioned, in the grey-box case, we generate the perturbations from the undefendedEnsAdv and use them to attack D1, D2, D3 and D4 (which in-clude preprocessors prepended to EnsAdv). Figure 2c showsthe ASR results averaged over D1∼4. DCT Low outperformsunconstrained MIM by large margins in all cases. ComparingFigure 2a with Figure 2c, the larger difference between un-constrained MIM and DCT Low in the grey-box case reflectsthe fact that the top NeurIPS 2017 defenses are not nearlyas effective against low frequency perturbations as they areagainst standard unconstrained attacks. In fact, DCT Lowyields the same ASR on D1, the winning defense submis-sion in the NeurIPS 2017 competition, as on the adversariallytrained model without the preprocessor prepended; the pre-

Cln Adv D1 D2 D3 D4

Cln

_1C

ln_3

Adv

_1A

dv_3

21 6.9 4.1 6.5 8.7 11

33 12 6.5 11 13 16

13 6.3 3.7 4.7 6.4 12

22 5.9 3.9 4.5 6.9 17

Unconstrained (N1)

0

8

16

24

32

40

(a)

Cln Adv D1 D2 D3 D4

Cln

_1C

ln_3

Adv

_1A

dv_3

17 14 14 15 16 14

28 21 22 21 24 21

22 22 22 21 23 22

30 30 29 28 30 29

Low Freq (N1)

0

8

16

24

32

40

(b)

Cln Adv D1 D2 D3 D4

Cln

_1C

ln_3

Adv

_1A

dv_3

30 9 7.3 9 11 12

70 18 21 20 20 24

7.9 7.8 4.1 6.1 8.6 8.4

19 13 5.8 13 16 18

Unconstrained (N10)

0

15

30

45

60

75

(c)

Cln Adv D1 D2 D3 D4

Cln

_1C

ln_3

Adv

_1A

dv_3

27 18 18 18 21 16

65 46 48 43 46 43

39 37 36 34 37 35

62 60 60 55 54 59

Low Freq (N10)

0

15

30

45

60

75

(d)

Figure 3: Transferability matrices with attack success rates (ASRs),comparing unconstrained MIM with low frequency constrainedDCT Low (n = 128) in the non-targeted case. First row is withiterations = 1, second is with iterations = 10. The column Cln isNasNet, Adv is EnsAdv.

Cln Adv D1 D2 D3 D4

Cln

_1C

ln_3

Adv

_1A

dv_3

0.0 -0.7 -0.8 -0.7 -0.6 -0.4

0.0 -0.7 -0.8 -0.7 -0.6 -0.5

0.0 -0.5 -0.7 -0.6 -0.5 -0.1

0.0 -0.7 -0.8 -0.8 -0.7 -0.2

Unconstrained (N1)

1.0

0.8

0.6

0.4

0.2

0.0

(a)

Cln Adv D1 D2 D3 D4

Cln

_1C

ln_3

Adv

_1A

dv_3

0.0 -0.1 -0.2 -0.1 -0.0 -0.1

0.0 -0.2 -0.2 -0.2 -0.2 -0.2

0.0 -0.0 -0.0 -0.0 0.0 0.0

0.0 -0.0 -0.0 -0.1 0.0 -0.0

Low Freq (N1)

1.0

0.8

0.6

0.4

0.2

0.0

(b)

Cln Adv D1 D2 D3 D4

Cln

_1C

ln_3

Adv

_1A

dv_3

0.0 -0.7 -0.8 -0.7 -0.6 -0.6

0.0 -0.7 -0.7 -0.7 -0.7 -0.7

0.0 -0.0 -0.5 -0.2 0.1 0.1

0.0 -0.3 -0.7 -0.3 -0.2 -0.1

Unconstrained (N10)

1.0

0.8

0.6

0.4

0.2

0.0

(c)

Cln Adv D1 D2 D3 D4

Cln

_1C

ln_3

Adv

_1A

dv_3

0.0 -0.3 -0.3 -0.3 -0.2 -0.4

0.0 -0.3 -0.3 -0.3 -0.3 -0.3

0.0 -0.1 -0.1 -0.1 -0.0 -0.1

0.0 -0.0 -0.0 -0.1 -0.1 -0.0

Low Freq (N10)

1.0

0.8

0.6

0.4

0.2

0.0

(d)

Figure 4: Transferability matrices with attack relative difference inASR with the Cln model (first column). Rows and columns in eachsubfigure is indexed in the same way as Figure 3.

processors are not effective (at all) at protecting the modelfrom low frequency perturbations, even in the targeted case,where success is only yielded if the model is fooled to predict,out of all 1000 class labels, the specified target label. Resultsagainst the individual defenses are presented in the appendix.

DCT Low helps black-box transfer to defended models.For assessing black-box transferability, we use Cln 1, Cln 3,Adv 1, Adv 3 in Table 1 as the source models for generat-ing perturbations, and attack EnsAdv and D1∼4, resulting in20 source-target pairs in total. The average ASR results overthese pairs are reported in Figure 2d. In the non-targeted case,we again see that DCT Low significantly outperforms uncon-strained MIM. However, in the targeted case, constraining tothe low frequency subspace does not enable MIM to succeedin transferring to distinct black-box defended models due tothe difficult nature of targeted transfer.

Next, we look at individual source-target pairs. For eachpair, we compare DCT Low (n = 128) with unconstrainedMIM in the non-targeted case with iterations = 1 anditerations = 10. Results for all frequency configurations withvaried dimensionality are provided in the appendix. Figure 3shows the transferability matrices for all source-target pairs,where for each subplot, the row indices denote source models,and the column indices denote target models. The value (andassociated color) in each gridcell represent the ASR for thespecified source-target pair. For Figure 4, the gridcell valuesrepresent the relative difference in ASR between the targetmodel and the cleanly trained model (Cln)8, using the source

8The relative difference for the target model = (ASR on the targetmodel - ASR on Cln) / ASR on Cln.

model of the corresponding row.Comparing (a) to (b) and (c) to (d) in Figure 3, it is clear

that low frequency perturbations are much more effectivethan unconstrained MIM against defended models. Specif-ically, we can see that DCT Low is significantly more effec-tive than unconstrained MIM against EnsAdv, and D1∼4 pro-vide almost no additional robustness to EnsAdv when lowfrequency perturbations are applied.DCT Low is not effective when transferring between unde-fended cleanly trained models. However, we do observethat DCT Low does not improve black-box transfer betweenundefended cleanly trained models, which can be seen bycomparing indices (Cln 1,Cln) and (Cln 3,Cln) between Fig-ure 3 (a) and (b), as well as (c) and (d). As discussed whencomparing white-box performance against cleanly trainedand adversarially trained models, low frequency constraintsare not generally more effective, but instead exploit the vul-nerabilities in currently proposed defenses.

4.3 Effectiveness of Low Frequency onUndefended Models v.s. Defended Models

In the last section, we showed that DCT Low is highly effec-tive against adversarially trained models and top-performingpreprocessor-based defenses, in the white-box, grey-box andblack-box cases. However, low frequency does not help whenonly cleanly trained models are involved, i.e. white-box onclean models and black-box transfer between clean models.To explain this phenomenon, we hypothesize that the state-of-the-art ImageNet defenses considered here do not reducevulnerabilities within the low frequency subspace, and thusDCT Low is roughly as effective against defended models as

Original Unconstrained (MIM) Low_256 Low_128 Low_64 Low_32

Figure 5: Adversarial examples generated with `∞ ε = 16/255 distortion

Original Unconstrained (MIM) Low_256 Low_128 Low_64 Low_32

Figure 6: Adversarial examples generated with `∞ ε = 32/255 distortion

against clean models, a property not seen when evaluatingwith standard, unconstrained attacks.

This can be most clearly seen in Figure 4, which presentsthe normalized difference between ASR on each of the tar-get models with ASR on the cleanly trained model. The dif-ference is consistently smaller for DCT Low than for uncon-strained MIM, and nearly nonexistent when the perturbationswere generated against adversarially trained (defended) mod-els (Adv 1,Adv 3). Thus, as discussed, defended models areroughly as vulnerable as undefended models when encoun-tered by low frequency perturbations.

4.4 Effectiveness of Low Frequency on CIFAR-10We test the effectiveness of low frequency perturbationson the much lower-dimensional than ImageNet, CIFAR-10dataset (d = 299 to d = 32), attacking the state-of-the-art adversarially trained model [Madry et al., 2017]. Ex-periment results with 1000 test examples can be seen in Ta-ble 2. Constraining the adversary used for training (non-targeted PGD [Kurakin et al., 2016; Madry et al., 2017];iterations = 10 and ε = 8/255) with DCT Low, and eval-uating both in the white-box and black-box settings (transfer

Dim White (Adv) Black (Adv) Black (Cln)

32 54.6 38.1 14.424 48.1 33.1 14.416 46.4 28.8 14.48 37.0 25.4 14.44 26.5 20.0 14.0

Table 2: Non-targeted attack success rate (ASR) with iterations =10 and ε = 8/255 of DCT Low in the white-box and black-box set-tings (transfer from distinct adversarially trained and cleanly trainedmodels of the same architecture) against adversarially trained modelwith 12.9% test error [Madry et al., 2017].

from distinct adversarially trained and cleanly trained modelsof the same architecture), we observe that dimensionality re-duction only hurts performance. This suggests that the notionof low frequency perturbations is not only constrained to thecomputer vision domain, but also only induces problems forrobustness in the realm of high-dimensional natural images.

5 DiscussionOur experiments show that the results seen in recent workon the effectiveness of constraining the attack space to lowfrequency components [Guo et al., 2018; Zhou et al., 2018;Sharma et al., 2018] are not due to generally reducing the sizeof the attack search space. When evaluating against state-of-the-art adversarially trained models and winning defense sub-missions in the NeurIPS 2017 competition in the white-box,grey-box, and black-box settings, significant improvementsare only yielded when low frequency components of the per-turbation are preserved. Low frequency perturbations are soeffective that they render state-of-the-art ImageNet defensesto be roughly as vulnerable as undefended, cleanly trainedmodels under attack.

However, we also noticed that low frequency perturbationsdo not improve performance when defended models are notinvolved, seen in evaluating white-box performance againstand black-box transfer between cleanly trained models. Lowfrequency perturbations do not yield faster white-box attackson clean models, nor do they provide more effective transferbetween clean models.

Our results suggest that the state-of-the-art ImageNet de-fenses, based on necessarily imperfect adversarial training,only significantly reduce vulnerability outside of the low fre-quency subspace, but not so much within. Against defenses,low frequency perturbations are more effective than uncon-strained ones since they exploit the vulnerabilities which pur-portedly robust models share. Against undefended models,constraining to a subspace of significantly reduced dimen-sionality is unhelpful, since undefended models share vulner-abilities beyond the low frequency subspace. Understandingwhether this observed vulnerability in defenses is caused byan intrinsic difficulty to being robust in the low frequencysubspace, or simply due to the (adversarial) training proce-dure rarely sampling from the low frequency region is an in-teresting direction for further work.

Low frequency perturbations are perceptible (under `∞-norm ε = 16/255). Our results show that the robustnessof currently proposed ImageNet defenses relies on the as-sumption that adversarial perturbations are high frequency innature. Though the adversarial defense problem is not con-strained to achieving robustness to imperceptible perturba-tions, this is a reasonable first step. Thus, in Figure 5, wevisualize low frequency constrained adversarial examples un-der the competition `∞-norm constraint ε = 16/255. Thoughthe perturbations do not significantly change human percep-tual judgement, e.g. the top example still appears to be astanding woman, the perturbations with n ≤ 128 are indeedperceptible.

Although it is well-known that `p-norms (in input space)are far from metrics aligned with human perception, exem-plified by their widespread use, it is still assumed that with asmall enough bound (e.g. `∞ ε = 16/255), the resulting ballwill constitute a subset of the imperceptible region. The factthat low frequency perturbations are fairly visible challengesthis common belief. In addition, if the goal is robustness toimperceptible perturbations, our study suggests this might beachieved, without adversarial training, by relying on low fre-

quency components, yielding a much more computationallypractical training procedure. In all, we hope our study encour-ages researchers to not only consider the frequency space,but perceptual priors in general, when bounding perturbationsand proposing tractable, reliable defenses.

ReferencesM. Alzantot, Y. Sharma, S. Chakraborty, and M. Srivastava.

Genattack: Practical black-box attacks with gradient-freeoptimization. arXiv preprint arXiv:1805.11090, 2018.

M. Alzantot, Y. Sharma, A. Elgohary, B. Ho, M. Srivastava,and K. Chang. Generating natural language adversarial ex-amples. arXiv preprint arXiv:1804.07998, 2018.

A. Athalye, N. Carlini, and D. Wagner. Obfuscated gradientsgive a false sense of security: Circumventing defenses toadversarial examples. arXiv preprint arXiv:1802.00420,2018.

W. Brendel, J. Rauber, and M. Bethge. Decision-based adver-sarial attacks: Reliable attacks against black-box machinelearning models. arXiv preprint arXiv:1712.04248, 2017.

N. Carlini and D. Wagner. Adversarial examples are noteasily detected: Bypassing ten detection methods. arXivpreprint arXiv:1705.07263, 2017.

N. Carlini and D. Wagner. Towards evaluating the robust-ness of neural networks. arXiv preprint arXiv:1608.04644,2017.

P. Y. Chen, Y. Sharma, H. Zhang, J. Yi, and C. Hsieh. Ead:Elastic-net attacks to deep neural networks via adversarialexamples. arXiv preprint arXiv:1709.0414, 2017.

P. Y. Chen, H. Zhang, Y. Sharma, J. Yi, and C. Hsieh. Zoo:Zeroth order optimization based black-box attacks to deepneural networks without training substitute models. InAISec ’17, pages 15–26. ACM, 2017.

J. Deng, W. Dong, R. Socher, J. Li, K. Li, and L. Fei-Fei. Ima-genet: A large-scale hierarchical image database. In Com-puter Vision and Pattern Recognition, 2009. CVPR 2009.IEEE Conference on, pages 248–255. IEEE, 2009.

G. W. Ding, Y. Sharma, K. Lui, and R. Huang. Max-marginadversarial (mma) training: Direct input space marginmaximization through adversarial training. arXiv preprintarXiv:1812.02637, 2018.

Y. Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, andJ. Li. Boosting adversarial attacks with momentum. arXivpreprint arXiv:1710.06081, 2017.

I. Goodfellow, J. Shlens, and C. Szegedy. Explainingand harnessing adversarial examples. arXiv preprintarXiv:1412.6572, 2014.

C. Guo, J. Frank, and K. Weinberger. Low frequency ad-versarial perturbation. arXiv preprint arXiv:1809.08758,2018.

A. Kurakin, I. Goodfellow, and S. Bengio. Adversar-ial examples in the physical world. arXiv preprintarXiv:1607.02533, 2016.

A. Kurakin, I. Goodfellow, S. Bengio, et al. Adversar-ial attacks and defences competition. arXiv preprintarXiv:1804.00097, 2018.

Q. Lei, L. Wu, P. Y. Chen, A. Dimakis, I. Dhillon, andM. Witbrock. Discrete attacks and submodular optimiza-tion with applications to text classification. arXiv preprintarXiv:1812.00151, 2018.

Y. Li, J. Bradshaw, and Y. Sharma. Are generative classi-fiers more robust to adversarial attacks? arXiv preprintarXiv:1802.06552, 2018.

F. Liao, M. Liang, Y. Dong, T. Pang, X. Hu, and J. Zhu. De-fense against adversarial attacks using high-level represen-tation guided denoiser. arXiv preprint arXiv:1712.02976,2017.

A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu.Towards deep learning models resistant to adversarial at-tacks. arXiv preprint arXiv:1706.06083, 2017.

N. Papernot and P. McDaniel. Deep k-nearest neighbors:Towards confident, interpretable and robust deep learning.arXiv preprint arXiv:1803.04765, 2018.

K Ramamohan Rao and Ping Yip. Discrete cosine trans-form: algorithms, advantages, applications. Academicpress, 2014.

L. Schott, J. Rauber, M. Bethge, and W. Brendel. Towards thefirst adversarially robust neural network model on mnist.arXiv preprint arXiv:1805.09190, 2018.

Y. Sharma and P. Y. Chen. Attacking the madry defensemodel with l1-based adversarial examples. arXiv preprintarXiv:1710.10733, 2017.

Y. Sharma and P. Y. Chen. Bypassing feature squeez-ing by increasing adversary strength. arXiv preprintarXiv:1803.09868, 2018.

Y. Sharma, T. Le, and M. Alzantot. Caad 2018: Gener-ating transferable adversarial examples. arXiv preprintarXiv:1810.01268, 2018.

L. Smith and Y. Gal. Understanding measures of uncer-tainty for adversarial example detection. arXiv preprintarXiv:1803.08533, 2018.

C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan,and I. Goodfellow. Intriguing properties of neural net-works. arXiv preprint arXiv:1312.6199, 2013.

A. Thomas and O. Elibol. Defense against adversarial attack:3rd place. https://github.com/anlthms/nips-2017, 2017.

F. Tramer, A. Kurakin, N. Papernot, D. Boneh, and P. Mc-Daniel. Ensemble adversarial training: Attacks and de-fenses. arXiv preprint arXiv:1705.07204, 2017.

C. Xie, J. Wang, Z. Zhang, Z. Ren, and A. Yuille. Mitigatingadversarial effects through randomization. arXiv preprintarXiv:1711.01991, 2017.

C. Xie, Y. Wu, L. van der Maaten, A. Yuille, and K. He. Fea-ture denoising for improving adversarial robustness. arXivpreprint arXiv:1812.03411, 2018.

R. Zhang, P. Isola, A. Efros, E. Shechtman, and O. Wang. Theunreasonable effectiveness of deep features as a perceptualmetric. arXiv preprint arXiv:1801.03924, 2018.

W. Zhou, X. Hou, Y. Chen, M. Tang, X. Huang, X. Gan, andY. Yang. Transferable adversarial perturbations. In ECCV(14), pages 471–486. Springer, 2018.

B. Zoph, V. Vasudevan, J. Shlens, and Q. Le. Learning trans-ferable architectures for scalable image recognition. arXivpreprint arXiv:1707.07012, 2017.

A Construction ProcessFor a specified reduced dimensionality n, and original dimen-sionality d, we consider the frequency subspace Rn×n. Forthe low frequency domain, DCT Low, we preserve compo-nents vi,j if 1 ≤ i, j ≤ n.

For the high frequency domain, DCT High, we do the op-posite, masking the lowest frequency components such thatn2 components are preserved: 1 ≤ i, j ≤ d(1 − rh). Thusrhd bands (rows/columns in V ) are preserved. To ensure thenumber of preserved components is equal between the differ-ently constructed masks, we specify rl = n

d , and solve thefollowing equation for rh:

d2 − (1− rh)2d2 = r2l d2

1− (1− rh)2 = r2l

2rh − r2h = r2l

r2h − 2rh + r2l = 0

(3)

Solving the quadratic equation for 0 ≤ rh ≤ 1, rh = 1 −√4(1−r2l )

2 .For the middle frequency band, DCT Mid, we would like

to ensure we mask an equal number of low and high fre-quency components. We thus solve the following equationfor rml:

r2mld2 =

d2 − r2l d2

2

rml =

√1− r2l

2

(4)

Thus, we mask components vi,j if 1 ≤ i, j ≤ rmld, andif d(1 − rmh) ≤ i, j ≤ d. Compute rmh from rml withequation 3.

For our representative random frequency mask,DCT Random, much like DCT High, rows/columnsare chosen, except in this case randomly rather than thehighest frequency bands. To ensure that n2 components arepreserved, rhd rows/columns are chosen, which are thenpreserved in both the x and y directions.

B Spatial Smoothing &Downsampling-Upsampling Filters

In the main paper, we show that DCT Low significantlyoutperforms all other frequency configurations (DCT High,DCT Mid, DCT Random) in the white-box, grey-box, andblack-box settings. Specifically, we observed that DCT Lowgenerates effective perturbations faster than without con-straint on adversarially trained models (but not so on cleanmodels), bypasses defenses prepended to the adversariallytrained model, helps black-box transfer to defended models,but is not effective when transferring between undefendedcleanly trained models. We observe mirrored results whenconstraining the perturbation with both spatial smoothing anddownsampling-upsampling filters; shown in Figure 7 and 8.

For the downsampling-upsampling filter, we resize the per-turbation with bilinear interpolation, and decrease the dimen-sionality from 299 to 32, as with DCT Low. For the spatial

smoothing filter, we smooth the perturbation with a gaussianfilter of fixed kernel size (7x7), decreasing the standard devi-ation to strengthen the constraint. As can be seen, despite thediffering parameters, the trends of each of the low frequencyperturbation methods are the same.

C Complete HeatmapWe summarize our attack success rate results with DCT Lowin Figure 9. The rows correspond to sources, and columnscorresponds to targets. The sources include [Cln, Adv, Cln 1,Adv 1, Cln 3, Adv 3], where Cln is NasNetLarge 331,Adv is EnsAdvInceptionResNetV2; Cln 1, Adv 1,Cln 3, Adv 3 are summarized in the main text. The tar-gets include [Cln, Adv, D1, D2, D3, D4], where D1∼4 aredefenses summarized in the main text. Thus (Cln,Cln) and(Adv,Adv) summarize white-box results, (Adv,D1∼4) sum-marizes grey-box results, and the rest of the entries summa-rize black-box results. Note that the low frequency configu-ration is DCT Low with n = 128.

D All PlotsFigure 10 shows white-box results using DCT Low attackingthe adversarially trained model. Figures 11-14 shows resultsagainst D1, D2, D3, and D4, respectively. Figures [15-19,20-24, 25-29, 30-34] shows results transferring from each ofthe source models [Cln 1,Cln 3,Adv 1,Adv 3] to each of thetarget defenses [EnsAdv,D1,D2,D3,D4].

Figure 35 shows white-box results attacking the cleanlytrained model. Figures 36-39 show black-box results trans-ferring from the source models to the cleanly trained model[Cln].

8 4 2 1Standard Deviation

0

20

40

Atta

ck S

ucce

ss R

ate

(ASR

) Non-Targeted (1)


0

25

50

75



0

20

40

60Targeted (10)

Unconstrained (MIM) Smoothing



0

20

40

Atta

ck S

ucce

ss R

ate

(ASR

) Non-Targeted (1)


0

25

50

75



0

20

40

60

Targeted (10)




0

10

20

30

40

Atta

ck S

ucce

ss R

ate

(ASR

) Non-Targeted (1)


020406080

Non-Targeted (10)


0

10

20

30Targeted (10)




0

10

20

30

Atta

ck S

ucce

ss R

ate

(ASR

) Non-Targeted (1)


0

20

40

Non-Targeted (10)


0.0

2.5

5.0

7.5

10.0Targeted (10)


(d) Black-box attack on sources transferred to defenses (EnsAdv +D1∼4)

Figure 7: Spatial Smoothing filter; number of iterations in parentheses, non-targeted with ε = 16/255, targeted with ε = 32/255.


0

20

40

Atta

ck S

ucce

ss R

ate

(ASR

) Non-Targeted (1)


0

25

50

75



0

20

40

60Targeted (10)

Unconstrained (MIM) Bilinear



0

20

40

Atta

ck S

ucce

ss R

ate

(ASR

) Non-Targeted (1)


0

25

50

75



0

20

40

60

Targeted (10)




0

10

20

30

40

Atta

ck S

ucce

ss R

ate

(ASR

) Non-Targeted (1)


020406080

Non-Targeted (10)


0

10

20

30Targeted (10)




0

10

20

30

Atta

ck S

ucce

ss R

ate

(ASR

) Non-Targeted (1)


0

20

40

Non-Targeted (10)


0.0

2.5

5.0

7.5

10.0Targeted (10)


(d) Black-box attack on sources transferred to defenses (EnsAdv +D1∼4)

Figure 8: Downsampling-Upsampling filter; number of iterations in parentheses, non-targeted with ε = 16/255, targeted with ε = 32/255.

Cln Adv D1 D2 D3 D4

Cln

Adv

Cln_1

Cln_3

Adv_1

Adv_3

49 9.2 5.6 8.8 10 13

13 11 7.8 6.9 8.6 29

21 6.9 4.1 6.5 8.7 11

33 12 6.5 11 13 16

13 6.3 3.7 4.7 6.4 12

22 5.9 3.9 4.5 6.9 17

Unconstrained (N1)

0

8

16

24

32

40

Cln Adv D1 D2 D3 D4

Cln

Adv

Cln_1

Cln_3

Adv_1

Adv_3

42 16 16 17 20 17

17 43 44 29 30 38

17 14 14 15 16 14

28 21 22 21 24 21

22 22 22 21 23 22

30 30 29 28 30 29

Low Freq (N1)

0

8

16

24

32

40

Cln Adv D1 D2 D3 D4

Cln

Adv

Cln_1

Cln_3

Adv_1

Adv_3

98 15 17 17 18 19

12 96 33 38 41 46

30 9 7.3 9 11 12

70 18 21 20 20 24

7.9 7.8 4.1 6.1 8.6 8.4

19 13 5.8 13 16 18

Unconstrained (N10)

0

15

30

45

60

75

Cln Adv D1 D2 D3 D4

Cln

Adv

Cln_1

Cln_3

Adv_1

Adv_3

98 35 35 36 37 34

34 94 94 77 75 91

27 18 18 18 21 16

65 46 48 43 46 43

39 37 36 34 37 35

62 60 60 55 54 59

Low Freq (N10)

0

15

30

45

60

75

Cln Adv D1 D2 D3 D4

Cln

Adv

Cln_1

Cln_3

Adv_1

Adv_3

64 0 0 0 0 0

0 42 0 0.1 0 0

0.5 0 0 0 0 0

6.8 0 0.1 0.2 0.1 0

0 0 0 0 0 0

0 0 0 0 0 0

Unconstrained (T10)

0.0

2.5

5.0

7.5

10.0

12.5

15.0

Cln Adv D1 D2 D3 D4

Cln

Adv

Cln_1

Cln_3

Adv_1

Adv_3

64 0.4 0.5 0.3 0.1 0.3

0.4 56 56 6.1 4.7 33

0.6 0.1 0.1 0.2 0.1 0

7.3 1.7 1.8 1.5 0.9 1.1

1.4 0.9 1.1 0.3 0.5 0.5

10 5 5.4 4.1 2.4 4.5

Low Freq (T10)

0.0

2.5

5.0

7.5

10.0

12.5

15.0

Figure 9: Transferability matrix comparing standard unconstrained MIM with low frequency constrained DCT Low (n = 128). First, second,and third rows are non-targeted with iterations = 1, non-targeted with iterations = 10, and targeted with iterations = 10


0

10

20

30

40

50

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid

(a) Non-targeted with ε = 16 anditerations = 1.


0

20

40

60

80

100

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid

(b) Non-targeted with ε = 16 anditerations = 10.


0

10

20

30

40

50

60

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid

(c) Targeted with ε = 32 and iterations =10.

Figure 10: White-box attack on adversarially trained model.


0

10

20

30

40

50

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

20

40

60

80

100

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

60

Atta

ck S

ucce

ss R

ate

(ASR

)Unconstrained (MIM)DCT_Low

DCT_HighDCT_Random

DCT_Mid


Figure 11: Grey-box attack on D1.


0

10

20

30

40

50

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

20

40

60

80

100

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

60

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid




0

10

20

30

40

50

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

20

40

60

80

100

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

60

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid




0

10

20

30

40

50

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

20

40

60

80

100

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

60

Atta

ck S

ucce

ss R

ate

(ASR


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

5

10

15

20

25

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid


Figure 15: Black-box attack from Cln 1 to EnsAdv.


0

5

10

15

20

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

5

10

15

20

25

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid


Figure 16: Black-box attack from Cln 1 to D1.


0

5

10

15

20

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

5

10

15

20

25

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

Atta

ck S

ucce

ss R

ate

(ASR


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

5

10

15

20

25

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

5

10

15

20

25

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

25

30

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

6

Atta

ck S

ucce

ss R

ate

(ASR


DCT_HighDCT_Random

DCT_Mid


Figure 20: Black-box attack from Cln 3 to EnsAdv.


0

5

10

15

20

25

30

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

6

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

25

30

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

6

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

25

30

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

6

Atta

ck S

ucce

ss R

ate

(ASR


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

25

30

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

6

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

25

30

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

5

10

15

20

25

30

35

40

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

6

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid


Figure 25: Black-box attack from Adv 1 to EnsAdv.


0

5

10

15

20

25

30

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

5

10

15

20

25

30

35

40

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

6

Atta

ck S

ucce

ss R

ate

(ASR


DCT_HighDCT_Random

DCT_Mid


Figure 26: Black-box attack from Adv 1 to D1.


0

5

10

15

20

25

30

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

5

10

15

20

25

30

35

40

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

6

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

25

30

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

5

10

15

20

25

30

35

40

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

6

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

25

30

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

5

10

15

20

25

30

35

40

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

6

Atta

ck S

ucce

ss R

ate

(ASR


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

25

30

35

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

60

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

2

4

6

8

10

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid


Figure 30: Black-box attack from Adv 3 to EnsAdv.


0

5

10

15

20

25

30

35

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

60

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

2

4

6

8

10

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

25

30

35

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

60

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

2

4

6

8

10

Atta

ck S

ucce

ss R

ate

(ASR


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

25

30

35

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

60

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

2

4

6

8

10

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid




0

5

10

15

20

25

30

35

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

60

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

2

4

6

8

10

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid




0

10

20

30

40

50

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

20

40

60

80

100

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

60

70

Atta

ck S

ucce

ss R

ate

(ASR


DCT_HighDCT_Random

DCT_Mid


Figure 35: White-box attack on cleanly trained model.


0

5

10

15

20

25

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

5

10

15

20

25

30

35

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid


Figure 36: Black-box attack from Cln 1 to Cln.


0

5

10

15

20

25

30

35

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

60

70

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

2

4

6

8

10

12

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid


Figure 37: Black-box attack from Cln 3 to Cln.


0

5

10

15

20

25

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

5

10

15

20

25

30

35

40

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

1

2

3

4

5

6

Atta

ck S

ucce

ss R

ate

(ASR


DCT_HighDCT_Random

DCT_Mid


Figure 38: Black-box attack from Adv 1 to Cln.


0

5

10

15

20

25

30

35

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

10

20

30

40

50

60

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid



0

2

4

6

8

10

12

14

Atta

ck S

ucce

ss R

ate

(ASR

)


DCT_HighDCT_Random

DCT_Mid


Figure 39: Black-box attack from Adv 3 to Cln.

Documents

Yash Sharma, Gavin Weiguang Ding, and Marcus A. Brubaker ... · Yash Sharma, Gavin Weiguang Ding, and Marcus A. Brubaker Borealis AI Abstract Carefully crafted, often imperceptible,