PyTorch: Visualizing Loss Functions>

<PyTorch: Visualizing Loss Functions> 

 

Date: 2023.03.22               

 

* The PyTorch series will mainly touch on the problem I faced. For actual code, check out my github repository.

 

[Visualization (Code)]

[Added tracking variables for epoch, train loss, and test loss]

[Visualizing Loss Functions for each Train and Test Data]

Later we will also automate plotting the loss functions. For now, we hard-coded this manually to see the results shown below. Before reviewing the results, I would like to review the data type change.

 

In lines 7 and 8, we change PyTorch’s tensor to a Numpy array. What exactly is happening?

 

The loss function represents how well your model can predict the output compared to the actual output. The output you provided is a list of tensor values, each tensor representing the loss value for each epoch or iteration of the training process. Each tensor includes a gradient function, which PyTorch uses to optimize the model parameters. When you tried to plot the loss values using pyplot, you encountered an error because pyplot doesn't recognize PyTorch tensors. Therefore, you had to convert the tensors into numpy arrays, which pyplot can recognize and plot.

 

The difference between the initial input and the fixed one is that the fixed one includes a conversion of the PyTorch tensors into numpy arrays using the numpy() method. Specifically, this code np.array(torch.tensor(loss_values).numpy()) converts the list of PyTorch tensors loss_values into a list of numpy arrays. The numpy() method converts the PyTorch tensor into a numpy array.

 

By converting the PyTorch tensors into numpy arrays, pyplot can recognize and plot the loss function values. To use the .numpy() method, the loss_values tensor needs to be a PyTorch tensor object. If loss_values is already a PyTorch tensor object, then calling loss_values.numpy() will work fine. However, if loss_values is a list or some other Python object that is not a PyTorch tensor, then calling loss_values.numpy() will raise an AttributeError because Python lists do not have a numpy() method.

 

In the code you provided, loss_values appears to be a list of PyTorch tensors, so calling torch.tensor(loss_values).numpy() works to convert the list of tensors to a NumPy array, which can then be used with Matplotlib's plot function. However, if loss_values is already a PyTorch tensor object (and not a list of tensors), then calling loss_values.numpy() directly will work just as well.

 

In our case, loss_values is a list, as you can see below.

 

[List of Tensors]

Since we have a Python List type, we need to convert the list into PyTorch tensor types to apply the Numpy conversion function.

 

[Visualization (Graph feat. pyplot)]

That’s it for complications. The loss function will look something like the following.

[Loss Function for 200 epochs. Data is collected every ten epochs.]