Why do we need counting principles and Probability theory in Data Science?

In this previous article, we discussed Descriptive Statistics and the different measures of centrality and spread, and how to compute them. In this article, we discuss the need for counting principles that are used in the probability theory.

Why do we need counting?

The simple answer is that counting plays a very important role in Probability Theory. Let’s first see why we need probability theory.

So, we need the probability theory as we deal with samples and not the entire population and we estimate different measures(based on the domain and task at hand) from a sample.

Now we are interested in the probability that a statistic value as computed from a sample is close to the value computed from a population. Say we choose a sample randomly, then we want to know the probability that the statistic value as computed from a sample is how much close to the same parameter if computed from a population.

To answer such questions, we need probability theory.

We also need probability theory in machine learning, say we are given an image as the input and we try to predict/estimate its class and in many classification problems, we are interested in knowing the probability say if an image belongs to Class A or Class B and so on. So, another way of looking at the classification problem is as probability estimation problems.

So, that’s why probability is very very fundamental to statistics and machine learning.

Now let’s understand why the concept of counting is important. Let’s start with an example of tossing a fair coin and we are interested in knowing the probability of heads.

We know that the probability of getting heads is 50%. The way we compute this probability is as follows:

We know that there are two possible outcomes and since the coin is fair, we have no reason to believe that one of the outcomes is more likely than the other, so there is a 100% chance the coin is going to land on one of the two sides and since none of the sides is more likely than the other, we say that there is 50% chance of landing on heads and 50% chance of landing on tails.

Similarly, say we roll a fair dice and we are interested in knowing the probability of getting a 6, and using the same reasoning we can say that there are 6 possible outcomes all of which are equally likely.

So, it’s important to count the total number of possible outcomes. Let’s take one more example.

Say we have a standard deck of 52 cards and we are interested in knowing the probability of getting an ace of a spade when a card is drawn and we might say that there are 52 possible outcomes and all are equally likely and therefore the probability of getting an ace of a spade would be ‘1/52’.

Now suppose we are drawing 4 cards and we are interested in knowing the probability of getting 4 aces and in this case, every time we shuffle the deck of cards and pick 4 cards from it, we get a different outcome and there are many many outcomes possible when we draw 4 cards, each outcome is one unique set of 4 cards that we draw.

If we know the total number of possible outcomes, then we can easily tell the probability of getting 4 aces.

And over here the concept of counting and the principles are used.

And in this article, we learn how to count the number of outcomes of an experiment.

Let’s start with very simple cases where counting the total number of experiments is straightforward.

Counting

Let’s start with a simple problem and whatever learning we get from the simple, we’ll apply it to the complex problems as well.

So, if we have a sequence starting from 1 and ending at ’n’, then we know how to count the number of elements in the sequence of consecutive numbers, it will always be ‘n’.

Let’s look at other problems where this principle will be used:

This sequence is a bit different from the one we discussed earlier, we know how to count the number of elements in a sequence of consecutive numbers if the sequence starts from 1.

So, we convert it to a problem that we know how to solve. To convert the given sequence of consecutive numbers so that it starts from 1, we subtract 72 from the numbers in the sequence

And since now the sequence starts from 1, we can simply say that the number of elements in this sequence is 286.

And here is an alternative way of solving the same problem:

We know how to count the number of elements in a sequence starting from 1, so why not add the numbers from 1 to 72 in this sequence, this way we have a total of 358 elements but out of these 358 elements, we added 72 elements (numbers 1 to 72) so we can subtract 72 from 358 to get the number of elements in the actual sequence.

So, in general, if a sequence of consecutive numbers starts at ‘k’ and ends at ’n’, we can say that the total number of elements in the sequence is ‘n-k+1’.

Let’s take a little more complex problem:

So, we figure out the first three numbers and the last three numbers in the required sequence.

Now we have a problem, so far we have seen how to compute the number of elements in a sequence of consecutive numbers.

To deal with this, we know that every element in the sequence is divisible by 7, so we divide all the elements in the sequence by 7 and this way the sequence is converted to a sequence of consecutive numbers and the number of elements remains the same.

And from these transformed values(new sequence), we know how to count the number of elements using the ‘n-k+1’ where n equals 51, and k equals 11.

Let’s look at one more sequence:

In this case, all the numbers are not divisible by a common number but consecutive numbers have a difference of 4, so that gives us the idea that if so some simple arithmetic manipulations, then every element should become divisible by 4.

So, we take every element and add 1 to it, now in the new sequence(with number shifted by 1), we see that every element is divisible by 4

Let’s take one more example wherein we have the fraction value as part of our sequence

So, firstly we want to convert the sequence to a proper fraction and also ensure that the denominator is the same.

This is how we ensure that the denominator is the same.

then we convert the numbers to a proper fraction and then multiply each element by 12, now we observe that the difference between consecutive elements is 5 but none of the elements is divisible by 5, so we add the appropriate quantity such that all the elements are divisible by 5 and then we divide all the elements by 5

And the sequence we get at the end is something we know to deal with and we can compute the number of elements as ‘n-k+1’ where n equals 54 and k equals 23.

So, the basic idea is to use the same set of principles(as we use for an easy problem) to a complex problem.

References: PadhAI