© Wuhan University 2023

0 Introduction

Neural networks are now widely used in a variety of fields, including automatic driving^[1,2], medical treatment^[3,4], biology^[5,6], and finance^[7], where they have produced remarkable results. However, we must consider the security issue posed by deep neural networks. Similar to the problem in the traditional field of security, we must carefully examine user input. In the SQL(Structured Query Language) injection attack and XSS(Cross Site Scripting) attack, users can inject malicious code into the entry field to dump database and server content. The situation is identical with regard to deep neural networks. Confronted with abnormal inputs, we must pay close attention to the functionality of deep neural networks. There have been early occurrences of attacks of this type. Some attackers were able to evade system detection, for instance, in malicious email detection and intrusion detection systems employing deep learning models, due to the characteristics of the models. The attack on contemporary deep neural networks will inevitably result in a breach of privacy, identity theft, and numerous other grave issues.

The development of attack enhances the defense. If we can discover more efficient attack methods, it will undoubtedly benefit defensive abilities. Adversarial examples apply a minute perturbation to the original image, which is inconceivable to the human eye but induces massive errors in the deep learning models. There are two types of adversarial example methods: black-box attacks and white-box attacks. White-box attacks such as FGSM (fast gradient sign method)^[8], BIM(basic iterative method)^[9] and C&W(Carlini &Wagner)^[10] can be very effective when the model is known. In black-box attack, there are two prevalent methods: query and migration. The former must repeatedly access the deep neural networks. In the real world, many APIs(Application Programming Interface) of online models are not free, making them costly and easily detected. Transferability is the superior option. Szegedy et al^[8] have discovered that adversarial examples are transferable. Additionally, adversarial examples produced by one model pose a threat to other models. By showcasing transferability, we are able to conduct black-box attacks without visiting the model, making our attacks more covert. In transfer-based attack, data transformation is a common method in migratory attacks, just like DI²-FGSM(Diversity Input)^[11].

In this work, our approach seeks to explore whether better transferability can be achieved by focusing on destroying sensitive features of the image, starting from the image itself. In the real world, it is less likely that we are aware of the models our target employs. Therefore, black box attacks are more significant in reality. Our method focuses on enhancing transferability in order to achieve superior results on unexplored deep neural networks. We were motivated by the human attention mechanism. Attention mechanism is that humans will selectively focus on a portion of all information while ignoring the rest of the visible information. For example, people will pay more attention on cat's face when they try to distinguish between cats and dogs. As shown in Fig. 1, the same mechanism applies to deep neural networks for classified tasks. All models focus on the face of the cat, when all they try to classify this image. If the most sensitive features are attacked, it is more likely that all models will provide an incorrect answer.

Fig. 1 Grad-Cam feature maps generated under various networks (a) is an original image belonging to the 281st ImageNet class; (b) is generated through ResNet-50; (c) is generated through Shufflennet v2; (d) is generated through VGG19

The most closely related work is the ATA(Attention-guided Transfer Attack) method^[12]. It features attention transferability. However, it is inadequate for the conditions with limited local algorithmic power since the immediacy is highly important. Meanwhile, it fails to recognize that the sensitivity characteristics generated by various models are also distinctive. The differences may hinder transferability. As shown in Fig. 1, although all the models focus on the face of the cat, there are distinctions. ResNet-50 is more focused on the body of the cat. Shufflennet v2 is more attentive to other details of the cat.

Due to the variability of attention areas, firstly, we propose a single-step attack based on vulnerable characteristics from single network. In existing work, few people consider the effects of single-step attacks. However, single-step attacks are fast and have a low number of accesses to the model. As for the method of extracting sensitive features, we refer to the ATA method using Grad-CAM(Class Activation Maps)^[13], which extracts the areas of interest of the network for a given image. Then, we propose a one-step attack based on multiple sensitive characteristics from different networks. Finally, the idea of multi-feature fusion is applied to multi-step attacks, where the current ATA approach only considers sensitive features from one model. In conclusion, we enhance the ATA method by incorporating sensitive characteristics from other classification models. We can outperform the original ATA method by 1%-3% on the CIFAR-10 and MINST dataset.

1 Related Work

The adversarial example attacks can be divided into black-box and white-box attacks based on whether the attacker knows the model structure or not. For black-box attacks, transfer-based attacks are more commonly used because they do not require access to the target model. Next, we introduce representative methods for each of these types of attacks and the most advanced methods that related to our work.

1.1 White-Box Attack

Since we understand the architecture of deep neural networks, it is obvious that white box attacks are more efficient and dangerous. FGSM is a well-established white-box attack method^[8]. In FGSM, the loss function is maximized by taking steps in the opposite direction of the gradient of the loss function. BIM is the iterative version of FGSM^[9], whereas super parameters must be manually set in FGSM. In addition, when loss function comes to nonlinear functions, we do not know if the loss function will increase or decrease. BIM solves the aforementioned issues. It divides the FGSM attack into multiple turns in order to achieve better outcomes.

1.2 Black-Box Attack

Query-based attacks use input to learn the gradient information of the model^[14], while in reality, the effect of Query-based attacks is satisfying^[15].

1.3 Transfer-Based Attack

DI²-FGSM^[11] is transfer-based attack using data transformation. It applies a combinatorial transformation to the data with probability p before generating an adversarial sample. Wu et al^[16] further used adversarial transformations as new transformations to enhance transferability. Like mentioned before, ATA method^[12] features attention transferability. Our work also enhances ATA with sensitive features from different models.

2 Background

In this section, we destruct sensitive features to improve transferability. First, we discuss how we acquire sensitivity features. In addition, we present our baseline FGSM and BIM. Finally, we discuss the ATA method and several aspects that ATA fails to notice.

2.1 Grad-CAM

Grad-CAM^[13] can be described as formula (1) and (2). As revealed in formula (1), for specified class $c$, we find the partial derivative of class $c$ for each feature map $k$. Then we apply global average pooling to get ${\alpha }_k^c$ which represents the importance of feature map $k$ in judging class $c$. In order to visualize the result, we need to apply ReLU activation function to liner combination. ReLU can filter the pixel that has negative impact on judging class $c$.

${\alpha }_k^c=\stackrel{\mathrm{g}\mathrm{l}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{l}\mathrm{a}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{p}\mathrm{o}\mathrm{o}\mathrm{l}\mathrm{i}\mathrm{n}\mathrm{g}}{\overbrace{\frac{\mathrm{1}}{Z}\sum _i \sum _j }}\underset{\mathrm{g}\mathrm{r}\mathrm{a}\mathrm{d}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{s}\mathrm{v}\mathrm{i}\mathrm{a}\mathrm{b}\mathrm{a}\mathrm{c}\mathrm{k}\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{p}}{\underbrace{\frac{\partial y^c}{\partial A_{ij}^k}}}$(1)

$L_{\mathrm{G}\mathrm{r}\mathrm{a}\mathrm{d}-\mathrm{C}\mathrm{A}\mathrm{M}}^c=\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}\underset{\mathrm{l}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{o}\mathrm{m}\mathrm{b}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}}{\underbrace{\left(\sum _k {\alpha }_k^c{A}^k\right)}}$(2)

2.2 FGSM

FGSM^[8] is a classic white-box attack method. It is based on the gradient. Conventionally, we move toward the gradient in order to decrease the loss function. In FGSM method, we move toward the gradient in step ε so that the loss function will increase. The greater the ε is, the easier our adversarial examples are to be detected. The method can be formulated as follows:

$x^{\mathrm{a}\mathrm{d}\mathrm{v}}=x+\epsilon \mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\left({\nabla }_{x}J\left(\theta ,x,y\right)\right)$(3)

where $x^{\mathrm{a}\mathrm{d}\mathrm{v}}$ represents adversarial examples, $x$is original data.$\epsilon$ is perturbation budget, θ stands for network parameter and y is label of the original data. sign() is symbolic functions to get the sign of gradient.

2.3 BIM

In FGSM, we need to set ε mutually which may cause undesirable effect in non-linear condition. Then BIM^[9] is proposed to solve this problem. In BIM, we divide an FGSM step into multiple steps. In each step, we perform similar tasks just like we did in FGSM. We attempt to increase the loss function based on previous iteration. The method can be formulated as follows:

$X_{\mathrm{0}}^{\mathrm{a}\mathrm{d}\mathrm{v}}=X,X_{N+\mathrm{1}}^{\mathrm{a}\mathrm{d}\mathrm{v}}=\mathrm{C}\mathrm{l}\mathrm{i}{\mathrm{p}}_{X,ϵ}\left\{X_N^{\mathrm{a}\mathrm{d}\mathrm{v}}+\alpha \mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\left({\nabla }_{x}J\left(X_N^{\mathrm{a}\mathrm{d}\mathrm{v}},y_{\mathrm{t}\mathrm{r}\mathrm{u}\mathrm{e}}\right)\right)\right\}$(4)

where $y_{\mathrm{t}\mathrm{r}\mathrm{u}\mathrm{e}}$ is the true label of origin data, X is original data, α is perturbation budget which satisfies 0<α< ϵ,ϵ is also a perturbation budget.

2.4 ATA

ATA^[12] is a black-box method with good transferability^[19]. It is superior to C&W^[10], JSMA(Jacobian-based Saliency Map Attack)^[17], BIM^[9], and TAP(Transferable Adversarial Perturbations)^[18] in terms of transferability. It modifies the loss function in consideration of the sensitive features. We use L2 distance to denote the distance between two images. The greater the L2 distance is, the more differences there are between the two images. The following is the new loss function:

$\begin{array}{l}{J}_{\mathrm{G}\mathrm{r}\mathrm{a}\mathrm{d}-\mathrm{C}\mathrm{A}\mathrm{M}}\left(x^n,x^{\mathrm{o}\mathrm{r}\mathrm{i}}\right)=l\left(f\left(x^n\right),t\right)\\ +\lambda \cdot {\left(L_{\mathrm{G}\mathrm{r}\mathrm{a}\mathrm{d}-\mathrm{C}\mathrm{A}\mathrm{M}}^{y^{\mathrm{o}\mathrm{r}\mathrm{i}}}\left(x^n\right)-L_{\mathrm{G}\mathrm{r}\mathrm{a}\mathrm{d}-\mathrm{C}\mathrm{A}\mathrm{M}}^{y^{\mathrm{o}\mathrm{r}\mathrm{i}}}\left(x^{\mathrm{o}\mathrm{r}\mathrm{i}}\right)\right)}_{\mathrm{2}}^{\mathrm{2}}\end{array}$(5)

Then we can solve the following optimization issues:

$\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}{J}_{\mathrm{G}\mathrm{r}\mathrm{a}\mathrm{d}-\mathrm{C}\mathrm{A}\mathrm{M}}\left(x^n,x^{\mathrm{o}\mathrm{r}\mathrm{i}}\right)\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}{\left(x^p-x\right)}_{\mathrm{2}}\le \epsilon$(6)

As a black-box attack, it means the adversarial examples generated by ATA has outstanding transferability. ATA generates adversarial examples on ResNet V2, Inception V3 ,Inception V4 and Network with both Res and Inception blocks, respectively. Then adversarial examples generated by the local model are used to attack other models, for example, using adversarial examples generated on ResNet V2 to attack Inception V3 and Inception V4, which can also gain fine results. ATA outperforms other excellent methods like C&W^[10] and TAP^[18].

However, the ATA method does not account for sensitive features produced by other methods. In addition, ATA requires a massive number of calculations. We can apply this method to more realistic scenarios if we can devise a single step method to generate adversarial examples quickly with limited local computing power.

3 Generating Adversarial Examples Using Sensitive Features

We will introduce three new methods: one-step attack based on sensitive features from single model, one-step attack based on sensitive features fusion and multi-step attack based on sensitive features fusion, step by step. One-step attacks are more suitable under conditions where speed is crucial. Due to the fact that there are times when we require adversarial examples as quickly as possible, such as during military operations, one-step attack is also needed in real-world situation. Under the condition that we have sufficient time to plan our attacks, a multi-step attack is preferable. In general, classifier f takes input picture $x$, then a probability vector will be produced showcasing the most likely class of $x$. In adversarial example, we try to add perturbation on original picture $x$ in order to deceive the classifier. When the new image $x^p$ is the input of classifier f, the classifier will make wrong decision. Meanwhile, the distance between $x$ and $x^p$ should not be too far or our $x^p$ is easy to be detected. Formula (7) needs to be satisfied:

$\mathrm{m}\mathrm{i}\mathrm{n}{\Vert x-x^p\Vert }_{\mathrm{\infty }}$(7)

$\mathrm{a}\mathrm{n}\mathrm{d}f\left(x+p\right)=c^{\mathrm{n}\mathrm{e}\mathrm{w}}\ne c^{\mathrm{o}\mathrm{r}\mathrm{i}}$

where $c^{\mathrm{n}\mathrm{e}\mathrm{w}}$ is the new prediction of the adversarial example, $c^{\mathrm{o}\mathrm{r}\mathrm{i}}$ is the prediction of the origin clean image.

So the key is to solve the following optimization problem:

$f\left(x+p\right)=c^{\mathrm{n}\mathrm{e}\mathrm{w}}\ne c^{\mathrm{o}\mathrm{r}\mathrm{i}}\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}\mathrm{m}\mathrm{i}\mathrm{n}{\Vert x-x^p\Vert }_{\mathrm{\infty }}<ϵ$(8)

3.1 One-Step Attack

Firstly, we introduce one-step attack based on sensitive features of single model. In one-step attack, we send original images into the network to acquire the sensitive features. Then we only reserve sensitive section and send $x\text{'}$ into network again. We produce adversarial examples according to new parameters of the network. Algorithm 1 describes the process of generating $x\text{'}$. The mask in one-step attack is as the same size of the input data. In sensitive region, mask contains 1, otherwise, contains 0. Then we calculate the Hadamard product of the input images and the mask. We can reserve the sensitive features.

Algorithm 1 Generate $x\text{'}$ sensitive features of single model

$\mathbf{I}\mathbf{n}\mathbf{p}\mathbf{u}\mathbf{t}:x$: original image, $\theta :$ parameters of the network. $\mathbf{O}\mathbf{u}\mathbf{t}\mathbf{p}\mathbf{u}\mathbf{t}:x\text{'}$ $\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}=\mathrm{g}\mathrm{r}\mathrm{a}\mathrm{d}\_\mathrm{C}\mathrm{A}\mathrm{M}(\theta )$ # Reserving sensitive section $x\text{'}=x\mathrm{*}\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}$ $\mathbf{r}\mathbf{e}\mathbf{t}\mathbf{u}\mathbf{r}\mathbf{n}{x}^{\text{'}}$

The aforementioned method only takes into account sensitive features generated by one model. Then, we propose a one-step attack method that combines multiple sensitive features generated by various models. Algorithm 2 describes the process of generating $x\text{'}$ using multiple sensitive features.

The α,β,γ are the probability that selects the pixels of three different section. The probability equal to 1 denotes that we choose every pixel in the area. The probability equal to 0 denotes that no pixels should be selected in the section. Clearly, if α,β,γ are too large, the $x^{\text{'}}$ will be too similar to the original image $x$, which can be formulated as follows:

Algorithm 2 Generate $x\text{'}$ using multiple sensitive features generated by various models

$\mathbf{I}\mathbf{n}\mathbf{p}\mathbf{u}\mathbf{t}:x$ :original image, ${\theta }_1:$ parameters of the network1, ${\theta }_2:$ parameters of the network2, ${\theta }_3:$ parameters of the network3 $\mathbf{O}\mathbf{u}\mathbf{t}\mathbf{p}\mathbf{u}\mathbf{t}:x^{\text{'}}$ $\mathrm{m}\mathrm{a}\mathrm{s}{\mathrm{k}}_1=\mathrm{g}\mathrm{r}\mathrm{a}\mathrm{d}\_\mathrm{C}\mathrm{A}\mathrm{M}({\theta }_1)$ $\mathrm{m}\mathrm{a}\mathrm{s}{\mathrm{k}}_2=\mathrm{g}\mathrm{r}\mathrm{a}\mathrm{d}\_\mathrm{C}\mathrm{A}\mathrm{M}({\theta }_2)$ $\mathrm{m}\mathrm{a}\mathrm{s}{\mathrm{k}}_3=\mathrm{g}\mathrm{r}\mathrm{a}\mathrm{d}\_\mathrm{C}\mathrm{A}\mathrm{M}({\theta }_3)$ # Reserving sensitive section $x\text{'}=x\mathrm{*}(\alpha *(\mathrm{m}\mathrm{a}\mathrm{s}{\mathrm{k}}_1\bigcap \mathrm{m}\mathrm{a}\mathrm{s}{\mathrm{k}}_2\bigcap \mathrm{m}\mathrm{a}\mathrm{s}{\mathrm{k}}_3)\bigcup$$\beta \mathrm{*}$($\mathrm{m}\mathrm{a}\mathrm{s}{\mathrm{k}}_{\mathrm{2,3}}\bigcap \mathrm{m}\mathrm{a}\mathrm{s}{\mathrm{k}}_{\mathrm{1,3}}$$\bigcap$$\mathrm{m}\mathrm{a}\mathrm{s}{\mathrm{k}}_{\mathrm{1,2}}$) $\bigcup$$\gamma \mathrm{*}(\mathrm{m}\mathrm{a}\mathrm{s}{\mathrm{k}}_{1\mathrm{o}\mathrm{n}\mathrm{l}\mathrm{y}}\bigcap \mathrm{m}\mathrm{a}\mathrm{s}{\mathrm{k}}_{2\mathrm{o}\mathrm{n}\mathrm{l}\mathrm{y}}\bigcap \mathrm{m}\mathrm{a}\mathrm{s}{\mathrm{k}}_{3\mathrm{o}\mathrm{n}\mathrm{l}\mathrm{y}})$ $\mathbf{r}\mathbf{e}\mathbf{t}\mathbf{u}\mathbf{r}\mathbf{n}{x}^{\text{'}}$

$\underset{\alpha ,\beta ,\gamma \to \mathrm{1}}{\mathrm{l}\mathrm{i}\mathrm{m}}{x}^{\text{'}}=x$(9)

After generating the $x^{\text{'}}$, we feed it into the neural network again. The attack is based on new neural network parameters. The concept of attack is comparable to that of FGSM, which moves in the opposite direction of the gradient. Formula (10) is the method of attack.

$x^p=x+\epsilon \cdot \mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\left({\nabla }_{x^{’}}J\left(x^{‘},y\right)\right)$(10)

3.2 Multi-Step Attack

In the multi-step attack, we modify the loss function by factoring in the sensitive features generated by different models. The objective of the new loss function is to destroy sensitive features based on the focus of the three models.

$\begin{array}{l}{J}_{\mathrm{G}\mathrm{r}\mathrm{a}\mathrm{d}-\mathrm{C}\mathrm{A}\mathrm{M}}\left(x^n,x^{\mathrm{o}\mathrm{r}\mathrm{i}}\right)=l\left(f\left(x^n\right),t\right)\\ +\alpha \cdot {\Vert L_{{\theta }_{\mathrm{1}}}^{y^{\mathrm{o}\mathrm{r}\mathrm{i}}}\left(x^n\right)-L_{{\theta }_{\mathrm{1}}}^{y^{\mathrm{o}\mathrm{r}\mathrm{i}}}\left(x^{\mathrm{o}\mathrm{r}\mathrm{i}}\right)\Vert }_{\mathrm{2}}^{\mathrm{2}}\\ +\beta \cdot {\Vert L_{{\theta }_{\mathrm{2}}}^{y^{\mathrm{o}\mathrm{r}\mathrm{i}}}\left(x^n\right)-L_{{\theta }_{\mathrm{2}}}^{y^{\mathrm{o}\mathrm{r}\mathrm{i}}}\left(x^{\mathrm{o}\mathrm{r}\mathrm{i}}\right)\Vert }_{\mathrm{2}}^{\mathrm{2}}\\ +\gamma \cdot {\Vert L_{{\theta }_{\mathrm{3}}}^{y^{\mathrm{o}\mathrm{r}\mathrm{i}}}\left(x^n\right)-L_{{\theta }_{\mathrm{3}}}^{y^{\mathrm{o}\mathrm{r}\mathrm{i}}}\left(x^{\mathrm{o}\mathrm{r}\mathrm{i}}\right)\Vert }_{\mathrm{2}}^{\mathrm{2}}\end{array}$(11)

The α,β,γ enable us to assign weight so that we can decide which sensitive feature should be pay more attention to.

BIM can be utilized to address the optimization issue. We increase the loss function with each iteration in order to destroy the sensitive features and deceive the classifier. BIM can be regarded as a multi-step version of FGSM. Algorithm 3 is our multi-step method. Figure 2 shows the flowchart of this method.

Fig. 2 The process of the multi-attack based on feature fusion

Algorithm 3 Multi-step attack

$\mathbf{I}\mathbf{n}\mathbf{p}\mathbf{u}\mathbf{t}:x$: original image, ${\theta }_1:$ parameters of the network1, ${\theta }_2:$ parameters of the network2, ${\theta }_3:$ parameters of the network3, ϵ: budget $\mathbf{O}\mathbf{u}\mathbf{t}\mathbf{p}\mathbf{u}\mathbf{t}:x^p$ ${ϵ}^{\text{'}}=\frac{ϵ}{\mathrm{i}\mathrm{t}\mathrm{e}\mathrm{r}}$ $x^0=x$ $\mathbf{f}\mathbf{o}\mathbf{r}k=0\mathbf{t}\mathbf{o}\mathrm{i}\mathrm{t}\mathrm{e}\mathrm{r}-1\mathbf{d}\mathbf{o}$ $x^{k+1}=\mathrm{c}\mathrm{l}\mathrm{i}{\mathrm{p}}_{x,\mathit{ϵ}}\left\{x^k+{ϵ}^{\text{'}}\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\left(\frac{\partial J_{\mathrm{G}\mathrm{r}\mathrm{a}\mathrm{d}-\mathrm{C}\mathrm{A}\mathrm{M}}\left(x^k,x,{\theta }_1,{\theta }_2,{\theta }_3\right)}{\partial x}\right)\right\}$ $\mathbf{e}\mathbf{n}\mathbf{d}\mathbf{f}\mathbf{o}\mathbf{r}$ $\mathbf{r}\mathbf{e}\mathbf{t}\mathbf{u}\mathbf{r}\mathbf{n}{x}^p=x^{\mathrm{i}\mathrm{t}\mathrm{e}\mathrm{r}}$

4 Experiment

Experimentally, we demonstrate that adversarial examples generated by our method are more transferable. Meanwhile, the differences between adversarial examples and original image of our method remain constant with other methods.

4.1 Setup

Dataset: We use CIFAR-10^[19] and MINST^[20] as our Dataset. CIFAR-10 includes 60 000 images with a resolution of 32×32×3. There are ten categories. These were used as the training set, which consisted of 50 000 items, and the test set consisted of 10 000 items. MINST also included 60 000 images but with a resolution of 28×28×1. These were used as the training set, which consisted of 50 000 items, and the test set consisted of 10 000 items.

Models: As local and attack models, we selected VGG19^[21], ResNet-50^[22], Inception^[23], SENet^[24] FcaNet^[25] and a transformer-based model SimpleViT^[26] .These models include both the most traditional models and the most advanced models, such as SENet and FcaNet. The use of these models as attack and local models is highly relevant; if our adversarial examples exhibit good transferability in these models, it is likely that good transferability will also be exhibited in a realistic black box. All of these models are trained with Adam optimizer with learning rate 0.001.

Baseline: In one-step attacks, FGSM is used as the baseline. In a multi-step attack, BIM and ATA serve as baselines. Our main goal is to design a more transferable method in order to achieve better results in black-box settings. Szegedy et al^[8] have discovered that adversarial examples are transferable. Although white-box attacks like FGSM is not designed for black-box settings, the adversarial examples it produced can work in black-box settings with limited effect. Therefore, we also choose FGSM as baseline to show how effective our design is.

Metrics: We use the accuracy of model to evaluate the effectiveness of our attack. The lower this metric is, the more successful our attack proves to be. We will skip the data originally misclassified by the model in order to make sure that there are no other factors that lead to wrong outputs. To evaluate the size of the perturbation, we use the L2 distance between original images and adversarial examples. L2 distance can be described as Formula (11). $x$ is the clean image and y is the adversarial example. Larger distance represents a larger generated adversarial perturbation.

${\Vert x,y\Vert }_{\mathrm{2}}=\sqrt[]{{\displaystyle \sum _i^n} {\left(x_i-y_i\right)}^{\mathrm{2}}}$(12)

4.2 The Effectiveness of One-Step Attack

Tables 1 and 2 show the transferability of a one-step attack with sensitive local model features. Clearly, our method outperforms FGSM by 3%-8%. We tested it with hyperparameters of 0.1, 0.05, and 0.01, respectively. The local model is represented in the first column. Ours represents the method we propose, while the plain in the table represents the accuracy of attacked models when they are not attacked.

Next, we discuss transferability performance under various feature maps. We apply the feature maps generated by various models to the same local network and discover that transferability performs differently. As shown in Table 3, we have highlighted the items with the highest mobility performance with bold font. We initially hypothesized that the attack performance may vary due to the structure and performance of the local network, so we introduced the new FcaNet, whose performance on ImageNet and CIFAR-10 datasets was the best in this experiment. The FcaNet-generated feature maps did not aid in improving our performance. To be more specific, none of the network's feature maps achieved the best mobility. Consequently, the experiment demonstrates the necessity of feature fusion. In Table 3, the hyperparameter (perturbation budget) ε is set to 0.1.

The experiments for one-step feature fusion based attack mentioned in Section 3.2 are displayed in Tables 4 and 5. Whereas we obtain data for experiments involving multiple feature fusion by averaging the experiments five times. This is due to the fact that the parameter settings may lead to fluctuations in the experimental results. In terms of transferability, Tables 4 and 5 demonstrate that our new feature fusion method is significantly superior to the FGSM method and has a degree of improvement over single features. Few cases will exhibit transferability similar to the single-feature case.

4.3 The Effectiveness of Multi-Step Attack

The experimental results of multi-step attacks based on feature fusion are presented in Tables 6 and 7. We can see that the ATA method performs better than BIM method, whereas our method performs even better than ATA method. It demonstrates an increase in the transferability of the adversarial examples we generate. The perturbation budget size adopted here is 0.01. When the perturbation is too great, the distance between the adversarial example and the original image becomes too great. Table 6 is the result on CIFAR-10 dataset. The transferability of our method is steadily better than that of ATA. Table 7 is the result on MINST dataset. All the models perform greatly in MINST dataset so we increase the perturbation budget to 0.2 to increase the effect of attack.

Attacked models also include a transformer-based model, ViT(Vision Transformer). Table 8 shows a discussion of adversarial examples' invisibility. Invisibility means adversarial examples should not be discovered by the human eye. If human can notice the difference between original picture and adversarial examples, the attack will be detected. L2 distance can be used to measure the distance between the original image and the adversarial example. A greater distance indicates a larger perturbation. A larger perturbation means the adversarial example can be easily detected. Our method is approximately equivalent to the distance between the FGSM and the original graph. This proves that our method performs similarly to other methods in terms of concealment. In fact, if the perturbation budget $\epsilon$ is set to 0.01, the distance between the final adversarial example and the original image is not too far from the FGSM.

5 Conclusion

Based on the fact that Grad-Cam is effective at extracting sensitive features from a network and that the sensitive features extracted by different models are primarily similar, this paper first proposes a one-step attack method against sensitive features .On the basis of the limitation that the one-step attack against sensitive features only considers the feature maps generated by the local model, a more transferability one-step attack is proposed by fusing the feature maps generated by various deep learning models. It is also demonstrated to be more transferable than FGSM and is guaranteed to be just effective. This paper also finds that the ATA multi-step attack highly resembles the ideas presented in this paper, and ATA also does not fuse multiple features. Therefore, we propose a multi-step attack method that combines multiple features. It is experimentally more transferable than the adversarial examples generated by ATA by 1%-3%.

References

All Tables

All Figures

	Fig. 1 Grad-Cam feature maps generated under various networks (a) is an original image belonging to the 281st ImageNet class; (b) is generated through ResNet-50; (c) is generated through Shufflennet v2; (d) is generated through VGG19
In the text

	Fig. 2 The process of the multi-attack based on feature fusion
In the text