Image Visualization and Primary Networks (feat. LeNet, AlexNet)

Image Visualization and Primary Networks (feat. LeNet, AlexNet)

 

Date : 2022.10.25

 

*The contents of this book is heavily based on Stanford University’s CS231n course.

 

[What’s Happening in the Conv Layer?]

We’ve seen ‘why’ conv layers are so crucial. But ‘what’ is it exactly doing? More specifically, what is the conv layer looking for?

 

The weight (filter) is the main parameter that guides the image processing network. Thus, as the CNN gets smarter, the weights improve and adjust as well. Let’s check.

 

 

[Before (above) and After (below)]

The images represent the outcome after the first hidden layer. The primary images on the left are pixels in the form of 0 ~ 255 (bright to darker). The first layer adjusted its weights depending on the brightness of the input image. The white and black areas are called blobs which is the guideline for the first hidden layer. As the network gets smarter, the layer detect patterns from the inputs depending on its guidelines.

 

The filter below reacts to vertical and horizontal edges, which is a different feature (pattern) compared to the figure above.

 

[Different Filter, Different Feature]

As the number of layers increases, each layer recognizes more complex yet abstract features of images.

 

[Each layer extracts a different feature]

The figure above shows that the network continues to ‘understand’ the image from different perspectives. Let’s take a look into two major CNN’s.

 

[LeNet]

 

[LeNet Architecture]

LeNet is the first CNN introduced to recognize hand-written numbers. It mainly uses sigmoid as the activation function. The structure consists multiple conv layers and sub-sampling (similar to pooling).

 

[AlexNet]

 

[AlexNet Architecture]

AlexNet is different from LeNet by the following properties.

1. Uses ReLu as the main activation function.
2. Uses Local Response Normalization (LRN) as the normalization layer.
3. Implements Dropouts.