Bayes' Theorem

Probability rules our lives - we use it every day without being aware of it. In this article, we discuss one of its most important theorems: Bayes' theorem.

Bayes’ theorem is one of the pillars of probability. Its namesake comes from Thomas Bayes (1702 – 1761), who proposed the theory in the eighteenth century. But what exactly was the scientist trying to explain? According to the Meriam-Webster dictionary, probability is ‘the ratio of the number of outcomes in an exhaustive set of equally likely outcomes that produce a given event to the total number of possible outcomes’.

Many probability theories govern the world. For example, when you go to the doctor, they prescribe what’s most likely to cure you. Additionally, advertisers focus their campaigns on the people who are more likely to buy the product they’re promoting. Furthermore, you choose your daily commute by the route that’s more likely to take less time.

Law of total probability

One of the most famous probability laws is the law of total probability. It’s important to analyze what the law of total probability is. To understand it, we’ll give you an example.

Let’s say that, in a random country, 39% of the citizens are women. We also know that 22% of women and 14% of men don’t have a job. So, what’s the probability P that a person chosen at random from the active population in this country is unemployed P (U)?

According to the theory of probability, this is how we’d express the probability:

• The probability that the person was a woman: P (W)
• The probability that the person was a man: P (M)

Since we know that 39% of the citizens are women, then we can deduce that P (W) = 0.39

Thus, we can infer that P (M) = 1 – 0.39 = 0.61.

Also, the stated problem gives us the conditional probabilities:

• The probability that a woman is unemployed: P (U | W) = 0.22
• The probability that a man is unemployed: P (U | M) = 0.14

Therefore, by using the law of total probability, we get:

P (U) = P (W) P (U | W) + P (M) P (U | M)

P (U) = 0.22 × 0.39 + 0.14 × 0.61

P (U) = 0.17

Thus, the probability that a person chosen at random will not have a job P (U) will be 0.17. You can see that the result lies between the two conditional probabilities (0.14 < 0.17 < 0.22).

Bayes’ theorem

Now, suppose that you chose a random adult to fill out a form and you realized that they don’t have a job. In this case, and taking into account the previous example, what’s the probability that this person you chose at random is a woman [P (W | U)]?

To solve this problem, you must apply Bayes’ theorem. Specifically, you use this theorem to calculate the probability of an event considering the previous information you have about this event. You can calculate the probability of an event A, also knowing that this event A fulfills certain characteristics (B) that affect its probability.

In this case, we’re talking about the probability that the person you chose at random to fill out a form is a woman. In addition, however, the probability won’t be independent of whether the person has a job or not.

The Bayes’ theorem formula

Like any other theorem, we need a formula to calculate the probability:

Although it seems difficult, everything has an explanation:

• To begin, B is the event we have prior information on.
• On the other hand, the term A (n) refers to the different conditional events.
• We have conditional probability in the numerator. This refers to the probability that something (an event A) will occur, knowing that another event (B) also occurs. We define this as P (A | B) and express it as ‘the probability of A given B’.
• In the denominator, we have the equivalent of P (B).

An example

Going back to the previous example, suppose that you chose a random adult to fill out a questionnaire and realized that they don’t have a job. What is the probability that this person is a woman [P (W | U)]?

Well, taking into account the previous example, we know that 39% of the active population are women. We know then that the rest are men. In addition, we know that the percentage of unemployed women is 22% and the percentage of unemployed men is 14%.

Finally, we know that the probability of choosing a random, unemployed person is 0.17. So, if we apply Bayes’ theorem, the result we’ll get is 0.5 probability that a person chosen at random, out of all those who are unemployed, will be a woman.

P (W | U) = (P (W) * P (U | W) / P (U)) = (0.22 * 0.39) / 0.17 = 0.5

We’ll conclude this article by referring to one of the most frequent confusions regarding probability. It can only range between 0 and 1. If the probability of an event is 0, then it’s impossible for it to happen. On the other hand, if the probability is 1, then it’ll happen for sure.