The convolution operation

This article covers the content discussed in The Convolutional Operation module of the Deep Learning course and all the images are taken from the same module.

So far, we have seen fully connected neural networks. In this article, we will discuss the convolution operation which is used in the Convolutional Neural Networks.

Before that, let’s see the key points of the Feed Forward Neural Network:

• Universal Approximation Theorem(UAT) says that Deep Neural Networks(DNN) are powerful function approximators.
• DNNs can be trained using backpropagation.
• However, Fully Connected DNNs(fully connected network means that any neuron in any of the layers is connected to all the neurons in the previous layer) are prone to overfitting as the network is very deep and the no. of parameters are very large which might result in the overfitting of the model.
• And the second problem with the fully connected networks is that some gradients might vanish due to long chains. Since the network is very deep, the gradients in the few of the starting layers might get vanished when flowing back and therefore resulting in no training of the weights.
• So, the objective is to have a network that is a complex network(having non-linearities) as we know that in most of the real-world problems the output is going to be a complex function of the input but has fewer parameters and is, therefore, less prone to overfitting. And CNN's belong to the family of…