Explaining ResNet50 ImageNet Classification Using the CAM Explainer

Objective

The goal of this notebook is to explore various CAM methods for image classification models. For now, we only support XGradCAM method, which is the state-of-the-art CAM method.

Loading Intel XAI Tools PyTorch CAM Module

from intel_ai_safety.explainer.cam import pt_cam as cam

Loading Notebook Modules

import torch
import numpy as np
from torchvision.models import resnet50, ResNet50_Weights
import matplotlib.pyplot as plt

Using XGradCAM

Loading the input image

Load the input image as a numpy array in RGB order.

from PIL import Image
import requests
from io import BytesIO

response = requests.get("https://raw.githubusercontent.com/jacobgil/pytorch-grad-cam/master/examples/both.png")
image = np.array(Image.open(BytesIO(response.content)))
plt.imshow(image)

Loading the Model

Load the trained model depending on how the model was saved. If you have your trained model, load it from the model’s path using ‘torch.load()’.

model = resnet50(weights=ResNet50_Weights.IMAGENET1K_V2) # Let's use ResNet50 trained on ImageNet as our model

We need to choose the target layer (normally the last convolutional layer) to compute CAM for. Simply printing the model will give you some idea about the name of layers and their specifications. Here are some common choices: - FasterRCNN: model.backbone - Resnet18 and 50: model.layer4 - VGG and densenet161: model.features

target_layer = model.layer4

We need to specify the target class as an integer to compute CAM for. This can be specified with the class index in the range [0, NUM_OF_CLASSES-1] based on the training dataset. For example, the index of the class ‘tabby cat’ is 281 in ImageNet. If targetClass is None, the highest scoring category will be used.

target_class = 281

Visualization

image_dims = (224, 224)
xgc = cam.x_gradcam(model, target_layer, target_class, image, image_dims, 'cpu')
xgc.visualize()

References

pytorch-grad-cam GitHub Project - https://github.com/jacobgil/pytorch-grad-cam

Loading Intel XAI Tools TensorFlow CAM Module

from intel_ai_safety.explainer.cam import tf_cam as cam

Explaining Image Classification Models with TensorFlow

%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
from urllib.request import urlopen
from tensorflow.keras.applications.resnet50 import ResNet50

from PIL import Image
import requests
from io import BytesIO

response = requests.get("https://raw.githubusercontent.com/jacobgil/pytorch-grad-cam/master/examples/both.png")
image = np.array(Image.open(BytesIO(response.content)))
plt.imshow(image)

model = ResNet50()
target_layer = model.get_layer("conv5_block3_out")
target_class = 281

tfgc = cam.tf_gradcam(model, target_layer, target_class, image)
tfgc.visualize()

https://github.com/ismailuddin/gradcam-tensorflow-2/blob/master/notebooks/GradCam.ipynb