Basics: Probability Theory

This article covers the content discussed in the Probability Theory module of the Deep Learning course and all the images are taken from the same module.

Introduction

The probability of any event A is always ≥ 0 and it will always be ≤ 1. So, probability values lie between 0 and 1 and that’s the intuition behind using the output of Sigmoid Neuron as the probability value.

And if we have ’n’ disjoint events, the sum of the probability of the union of those events is equal to the sum of the probability of individual events.

Random Variable

Let’s suppose we have some students and those students can have a grade from either of ‘A’ or ‘B’ or ‘C’. So, one way of looking at this is that we have 3 possible events: a student gets an ‘A’ grade or ‘B’ grade or ‘C’ grade. And now we can ask questions like what is the probability of a student getting an ‘A’ grade. The way we compute this is just the number of students with an ‘A’ grade divided by the total number of students.

We could have another context as well, for example, we can have two events which tell whether the student is Tall or Short. And we compute the probability of this in the same way as discussed above for Grade. So, that’s one way of looking at this situation.