Tech Development/Deep Learning (CNN)16 A Recipe for Applying Neural Networks to a Novel Problems A Recipe for Applying Neural Networks to a Novel Problems Date : 2022.12.25 *The original contents of this post is from Andrej Karpathy’s blog. [A Recipe for Training Neural Networks] While building my own CNN, I had numerous encounters with the huge gap in between “here is how a convolutional layer works” and “our network achieves state of the art results.” Thankfully Andrej has some words of w.. 2022. 12. 29. CNN: The Afterwork The Afterwork Date : 2022.12.25 *The original contents of this post is from Andrej Karpathy’s blog. [Additional Thoughts After Completing Deep CNN] After completing a convolutional neural network I couldn’t help but read additional blogs and research related to computer vision. It is truly amazing how we can program the computer to understand, categorize, and produce images as humans do. Andrej .. 2022. 12. 29. Deep Convolutional Neural Network Deep Convolutional Neural Network Date : 2022.10.27 *The contents of this book is heavily based on Stanford University’s CS231n course. [The Deeper We Go] Everything up to this point sums up into building a deep CNN. The deep version will contain the following properties. 3x3 filters for conv layer Use He weight initialization Use ReLu as activation function Use Adam as weight optimization Imple.. 2022. 12. 25. Image Visualization and Primary Networks (feat. LeNet, AlexNet) Image Visualization and Primary Networks (feat. LeNet, AlexNet) Date : 2022.10.25 *The contents of this book is heavily based on Stanford University’s CS231n course. [What’s Happening in the Conv Layer?] We’ve seen ‘why’ conv layers are so crucial. But ‘what’ is it exactly doing? More specifically, what is the conv layer looking for? The weight (filter) is the main parameter that guides the imag.. 2022. 12. 25. Completing the CNN Completing the CNN Date : 2022.10.23 *The contents of this book is heavily based on Stanford University’s CS231n course. [Implementing Conv & Pooling Layers] Like every other function (layer), the conv and pooling layer requires both forward and backward propagations. However, 4 dimensional matrix applications are no walk in the park. So, we’re going to implement a technique called ‘im2col’ (ima.. 2022. 12. 25. Convolutional & Pooling Layers Convolutional & Pooling Layers Date : 2022.10.22 *The contents of this book is heavily based on Stanford University’s CS231n course. [Convolutional Neural Network] CNN is a widely used technique in Computer Vision. CNN is similar to the network we’ve already built. We need to add the Convolution and Pooling layer to make the multilayer network into a CNN. The structure looks something like the f.. 2022. 12. 25. Batch Initialization, Overfitting, Dropout, and Optimization Batch Initialization, Overfitting, Dropout, and Optimization Date : 2022.10.16 *The contents of this book is heavily based on Stanford University’s CS231n course. [Batch Normalization] In the previous post, we explored various methods for weight initialization. The purpose of weight initialization was to evenly spread the activation outputs among all nodes. Batch normalization is a method to spr.. 2022. 12. 16. Weight Initialization, Xavier Weights, Dropout, and Setting Hyperparameters Weight Initialization, Xavier Weights, Dropout, and Setting Hyperparameters Date : 2022.10.14 *The contents of this book is heavily based on Stanford University’s CS231n course. [Weight Initialization] We’ve explored gradient descents designed to optimize the weights. Now lets focus on the initialization. “What value shall we start with?” So far we used weight decay in order to prevent overfitti.. 2022. 12. 16. SGD, Momentum, AdaGrad, and Adam Stochastic Gradient Descent, Momentum, AdaGrad, and Adam Date : 2022.10.11 *The contents of this book is heavily based on Stanford University’s CS231n course. Optimization is the process of finding the optimal variable value. We will explore different methods of optimization to initialize hyperparameters and input variables. The purpose of these “methods” is to increase both efficiency and accur.. 2022. 12. 16. Neural Network with Backward Propagation Neural Network with Backward Propagation Date : 2022.10.11 *The contents of this book is heavily based on Stanford University’s CS231n course. In the previous post, we programmed forward and backward propagations for each layer (Affine, ReLU, Sigmoid, Softmax) as separate classes. Now we only need to import and build the CNN. The benefit of coding each layer as separate classes is that we can bu.. 2022. 12. 16. Computational Graphs & Backward Propagation Date : 2022.10.09 *The contents of this book is heavily based on Stanford University’s CS231n course. [Improving Efficiency] So far, we've used multivariable calculus techniques such as differentiation and gradients to derive the “slope of the loss function for weights.” Implementing a new method called “Backward Propagation of Errors (backpropagation)” will increase efficiency in the SGD proces.. 2022. 10. 24. SGD, Epochs, and Accuracy Testing Date : 2022.09.28 *The contents of this book is heavily based on Stanford University’s CS231n course. [Implementing SGD] The overall procedure is as the following: Create a batch of randomly selected data for training. Use the loss function to find optimal inputs for weights. Repeat the above to minimize prediction error. In step 2, we will be applying the gradient descent method to a random bat.. 2022. 10. 22. Loss Function and Stochastic Gradient Descent Date : 2022.09.23 *The contents of this book is heavily based on Stanford University’s CS231n course. [Data Modeling] The data modeling process can be separated into 2 major steps: Learning and Testing. In the learning process our goal is to establish a neural network with high precision. We can control a few variables that will affect the model. The first control variable is the Weight Variable.. 2022. 10. 20. Activation Functions (Sigmoid, ReLU, Step) & Neural Networks Date : 2022.05.20 *The contents of this book is heavily based on Stanford University’s CS231n course. [퍼셉트론의 상위 개념 : 신경망. Neural Networks.] 신경망은 가중치 매개변수를 데이터로부터 학습할 수 있는 능력이 있다 특징을 가지고 있다. 신경망은 입력층, 은닉층, 출력층으로 구성되어 있다. 은닉층은 다충 퍼셉트론의 숨겨진 여러 개의 층이라고 생각하면 된다. 신경망에서는 활성화 함수가 있다. 퍼셉트론에서도 0과 1 중에서 무엇을 출력할지 정할 때 활성화 함수의 개념을 사용했었다. 활성화 함수는 뉴런 내부에 존재한다. 뉴런 안으로 들어가보자. 퍼셉트론의 대표적인 활성화 함수는 계단 함수이다. 0 혹은 1. .. 2022. 10. 14. Perceptrons, Equations, and Gates Date : 2022.04.24 *The contents of this book is heavily based on Stanford University’s CS231n course. [퍼셉트론 Perceptron] 퍼셉트론은 딥러닝 신경망의 기원이 되는 개념이다. 퍼셉트론 : 인공뉴런. 다수의 신호를 입력으로 받아 하나의 신호를 출력한다. 여기서의 신호는 전류와 같이 흐름이 있는 것이다. 퍼셉트론 신호는 2 가지의 상태를 띌 수 있다. 0과 1, 혹은 안흐른다/흐른다로 구분된다. 퍼셉트론의 환경에는 대개 입력 신호 x1, x2, x3, … , xn, 그리고 출력 신호 y1, y2, y3, … , yn, 그리고 가중치 w1, w2, w3, … , wn의 변수들이 있다. 각각의 신호 (입력/출력)는 뉴런 혹.. 2022. 10. 8. 이전 1 2 다음 728x90 반응형
