Pre-trained-Models with PyTorch

In this lab, you will use pre-trained models to classify between the negative and positive samples; you will be provided with the dataset object. The particular pre-trained model will be resnet18; you will have three questions:

• change the output layer
• train the model
• identify several misclassified samples

You will take several screenshots of your work and share your notebook.

Table of Contents

• Download Data
• Imports and Auxiliary Functions
• Dataset Class
• Question 1
• Question 2
• Question 3

Estimated Time Needed: 120 min

---

Download Data

Download the dataset and unzip the files in your data directory, unlike the other labs, all the data will be deleted after you close the lab, this may take some time:

!wget https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0321EN/data/images/Positive_tensors.zip 

--2024-01-12 17:17:05--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0321EN/data/images/Positive_tensors.zip
Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196
Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2598656062 (2.4G) [application/zip]
Saving to: ‘Positive_tensors.zip’

Positive_tensors.zi 100%[===================>]   2.42G  29.1MB/s    in 71s     

2024-01-12 17:18:16 (35.0 MB/s) - ‘Positive_tensors.zip’ saved [2598656062/2598656062]

!unzip -q Positive_tensors.zip 

! wget https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0321EN/data/images/Negative_tensors.zip
!unzip -q Negative_tensors.zip

--2024-01-12 17:20:14--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0321EN/data/images/Negative_tensors.zip
Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196
Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2111408108 (2.0G) [application/zip]
Saving to: ‘Negative_tensors.zip’

Negative_tensors.zi 100%[===================>]   1.97G  36.4MB/s    in 58s     

2024-01-12 17:21:13 (34.6 MB/s) - ‘Negative_tensors.zip’ saved [2111408108/2111408108]

We will install torchvision:

!pip install torchvision

Requirement already satisfied: torchvision in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (0.15.2)
Requirement already satisfied: numpy in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from torchvision) (1.23.5)
Requirement already satisfied: requests in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from torchvision) (2.31.0)
Requirement already satisfied: torch in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from torchvision) (2.0.1)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from torchvision) (10.0.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from requests->torchvision) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from requests->torchvision) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from requests->torchvision) (1.26.18)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from requests->torchvision) (2023.11.17)
Requirement already satisfied: filelock in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from torch->torchvision) (3.9.0)
Requirement already satisfied: typing-extensions in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from torch->torchvision) (4.4.0)
Requirement already satisfied: sympy in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from torch->torchvision) (1.11.1)
Requirement already satisfied: networkx in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from torch->torchvision) (2.8.4)
Requirement already satisfied: jinja2 in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from torch->torchvision) (3.1.2)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from jinja2->torch->torchvision) (2.1.1)
Requirement already satisfied: mpmath>=0.19 in /opt/conda/envs/Python-RT23.1/lib/python3.10/site-packages (from sympy->torch->torchvision) (1.3.0)

Imports and Auxiliary Functions

The following are the libraries we are going to use for this lab. The torch.manual_seed() is for forcing the random function to give the same number every time we try to recompile it.

# These are the libraries will be used for this lab.
import torchvision.models as models
from PIL import Image
import pandas
from torchvision import transforms
import torch.nn as nn
import time
import torch 
import matplotlib.pylab as plt
import numpy as np
from torch.utils.data import Dataset, DataLoader
import h5py
import os
import glob
torch.manual_seed(0)

<torch._C.Generator at 0x7f0e441d3690>

from matplotlib.pyplot import imshow
import matplotlib.pylab as plt
from PIL import Image
import pandas as pd
import os

Dataset Class

This dataset class is essentially the same dataset you build in the previous section, but to speed things up, we are going to use tensors instead of jpeg images. Therefor for each iteration, you will skip the reshape step, conversion step to tensors and normalization step.

# Create your own dataset object

class Dataset(Dataset):

    # Constructor
    def __init__(self,transform=None,train=True):
        directory="/home/wsuser/work"
        positive="Positive_tensors"
        negative='Negative_tensors'

        positive_file_path=os.path.join(directory,positive)
        negative_file_path=os.path.join(directory,negative)
        positive_files=[os.path.join(positive_file_path,file) for file in os.listdir(positive_file_path) if file.endswith(".pt")]
        negative_files=[os.path.join(negative_file_path,file) for file in os.listdir(negative_file_path) if file.endswith(".pt")]
        number_of_samples=len(positive_files)+len(negative_files)
        self.all_files=[None]*number_of_samples
        self.all_files[::2]=positive_files
        self.all_files[1::2]=negative_files 
        # The transform is goint to be used on image
        self.transform = transform
        #torch.LongTensor
        self.Y=torch.zeros([number_of_samples]).type(torch.LongTensor)
        self.Y[::2]=1
        self.Y[1::2]=0
        
        if train:
            self.all_files=self.all_files[0:30000]
            self.Y=self.Y[0:30000]
            self.len=len(self.all_files)
        else:
            self.all_files=self.all_files[30000:]
            self.Y=self.Y[30000:]
            self.len=len(self.all_files)     
       
    # Get the length
    def __len__(self):
        return self.len
    
    # Getter
    def __getitem__(self, idx):
               
        image=torch.load(self.all_files[idx])
        y=self.Y[idx]
                  
        # If there is any transform method, apply it onto the image
        if self.transform:
            image = self.transform(image)

        return image, y
    
print("done")

done

We create two dataset objects, one for the training data and one for the validation data.

train_dataset = Dataset(train=True)
validation_dataset = Dataset(train=False)
print("done")

done

Question 1

Prepare a pre-trained resnet18 model :

Step 1: Load the pre-trained model resnet18 Set the parameter pretrained to true:

# Step 1: Load the pre-trained model ResNet18 with pretrained=True
pretrained_resnet18 = models.resnet18(pretrained=True)

Step 2: Set the attribute requires_grad to False. As a result, the parameters will not be affected by training.

# Step 2: Set the parameter cannot be trained for the pre-trained model

# Set requires_grad to False for all parameters
for param in pretrained_resnet18.parameters():
    param.requires_grad = False

resnet18 is used to classify 1000 different objects; as a result, the last layer has 1000 outputs. The 512 inputs come from the fact that the previously hidden layer has 512 outputs.

Step 3: Replace the output layer model.fc of the neural network with a nn.Linear object, to classify 2 different classes. For the parameters in_features remember the last hidden layer has 512 neurons.

# Step 3: Replace the output layer with a new nn.Linear layer
in_features = pretrained_resnet18.fc.in_features
pretrained_resnet18.fc = nn.Linear(in_features, 2)

Print out the model in order to show whether you get the correct answer.
(Your peer reviewer is going to mark based on what you print here.)

# Print the modified ResNet18 model
print(pretrained_resnet18)

ResNet(
  (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (relu): ReLU(inplace=True)
  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
  (layer1): Sequential(
    (0): BasicBlock(
      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
    (1): BasicBlock(
      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (layer2): Sequential(
    (0): BasicBlock(
      (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (downsample): Sequential(
        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): BasicBlock(
      (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (layer3): Sequential(
    (0): BasicBlock(
      (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (downsample): Sequential(
        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): BasicBlock(
      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (layer4): Sequential(
    (0): BasicBlock(
      (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (downsample): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): BasicBlock(
      (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
  (fc): Linear(in_features=512, out_features=2, bias=True)
)

Question 2: Train the Model

In this question you will train your, model:

Step 1: Create a cross entropy criterion function

# Step 1: Create a cross-entropy criterion function
criterion = nn.CrossEntropyLoss()

Step 2: Create a training loader and validation loader object, the batch size should have 100 samples each.

# Step 2: Create training and validation loader objects with a batch size of 100
batch_size = 100

train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
validation_loader = DataLoader(validation_dataset, batch_size=batch_size, shuffle=False)

Step 3: Use the following optimizer to minimize the loss

optimizer = torch.optim.Adam([parameters  for parameters in pretrained_resnet18.parameters() if parameters.requires_grad],lr=0.001)

Complete the following code to calculate the accuracy on the validation data for one epoch; this should take about 45 minutes. Make sure you calculate the accuracy on the validation data.

n_epochs = 1
loss_list = []
accuracy_list = []
correct = 0
N_test = len(validation_dataset)
N_train = len(train_dataset)
start_time = time.time()

Loss = 0
start_time = time.time()

for epoch in range(n_epochs):
    for x, y in train_loader:

        pretrained_resnet18.train() 
        # clear gradient 
        optimizer.zero_grad()
     
        # make a prediction 
        z = pretrained_resnet18(x)
   
        # calculate loss 
        loss = criterion(z, y)
    
        # calculate gradients of parameters 
        loss.backward()
        
        # update parameters 
        optimizer.step()
        
        loss_list.append(loss.data)

    correct = 0
    with torch.no_grad():
        pretrained_resnet18.eval()
        for x_test, y_test in validation_loader:
            # make a prediction 
            z = pretrained_resnet18(x_test)
        
            # find max 
            _, yhat = torch.max(z.data, 1)
       
            # calculate misclassified samples in mini-batch 
            correct += (yhat == y_test).sum().item()

    accuracy = correct / N_test
    accuracy_list.append(accuracy)

# Print the final accuracy
print(f"Accuracy on validation data: {accuracy * 100:.2f}%")

Accuracy on validation data: 99.56%

Print out the Accuracy and plot the loss stored in the list loss_list for every iteration and take a screen shot.

accuracy

0.9956

plt.plot(loss_list)
plt.xlabel("iteration")
plt.ylabel("loss")
plt.show()

Question 3:Find the misclassified samples

Identify the first four misclassified samples using the validation data:

# Set the model to evaluation mode
pretrained_resnet18.eval()

misclassified_samples = []

with torch.no_grad():
    for i, (x_test, y_test) in enumerate(validation_loader):
        z = pretrained_resnet18(x_test)
        _, yhat = torch.max(z.data, 1)

        # Identify misclassified samples
        misclassified_indices = (yhat != y_test).nonzero().flatten()
        for mis_index in misclassified_indices:
            misclassified_samples.append((x_test[mis_index], yhat[mis_index].item(), y_test[mis_index].item()))

        if len(misclassified_samples) >= 4:
            break

# Print details of the first four misclassified samples
for i, (mis_x, mis_yhat, mis_y) in enumerate(misclassified_samples[:4]):
    print(f"Misclassified Sample {i + 1}: Predicted={mis_yhat}, Actual={mis_y}")

Misclassified Sample 1: Predicted=1, Actual=0
Misclassified Sample 2: Predicted=0, Actual=1
Misclassified Sample 3: Predicted=1, Actual=0
Misclassified Sample 4: Predicted=0, Actual=1

CLICK HERE Click here to see how to share your notebook.

About the Authors:

Joseph Santarcangelo has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.

Change Log

Date (YYYY-MM-DD)	Version	Changed By	Change Description
2020-09-21	2.0	Shubham	Migrated Lab to Markdown and added to course repo in GitLab

---

##

© IBM Corporation 2020. All rights reserved.

Copyright © 2018 cognitiveclass.ai. This notebook and its source code are released under the terms of the MIT License.