Base Rate Fallacy

by Craig Shrives

What Is the Base Rate Fallacy?

In Critical Thinking, the Base Rate Fallacy is failing to factor in an earlier base rate or statistic.

(Help! I'm still confused. Look at the examples below and then read the definitions above again. Hopefully, they will become clearer.)

An Example of Base Rate Fallacy

The vaccines are useless because more vaccinated are dying than non-vaccinated

In July 2021, Public Health England (PHE) announced that more than 60% of the COVID deaths in the first half of 2021 were of vaccinated people. In other words, more vaccinated people were dying of COVID than non-vaccinated people.

This statistic was quickly seized upon by the "anti-vaxxers" (those opposed to the coronavirus vaccine), who presented it as clear evidence that the vaccines did not work.

However, the "anti-vaxxers" were guilty of committing the base-rate fallacy.

The clearest way to think about this is to imagine a scenario when 99% of people are vaccinated and 1% of people aren't. Even though the vaccine dramatically reduces deaths, it is now easy to understand why there would be more deaths among the vaccinated than the non-vaccinated. Put simply, if 99% of people have a small chance of dying, then that is likely to produce more deaths than 1% of people with a greater chance of dying.

In July 2021, the situation was slightly more complicated than that though. The vast majority of the vulnerable (predominantly those over 80) were vaccinated while the majority of the less vulnerable (those under 30) were not. Even though the vaccine cut the risk of death by over 96%, the vaccinated vulnerable were still the most vulnerable, and they outnumbered the non-vaccinated. This last point is key. If literally everybody were vaccinated, then 100% of deaths would be of vaccinated people. Conversely, if literally no one were vaccinated, then 100% of deaths would be of non-vaccinated people. So, more deaths among the vaccinated speaks more about how many people have been vaccinated than it does about the efficacy of the vaccines.

Put another way, and possibly counter-intuitively for some, the efficacy of the vaccine cannot be calculated by comparing all the vaccinated with all the non-vaccinated. It could only be calculated by comparing an infected vaccinated group with an infected non-vaccinated group of equal size and equal vulnerability. In July 2021, those conditions were far from present.

In summary, the fact that more vaccinated people died than non-vaccinated told us nothing about the efficacy of the vaccines. The fact that the national COVID death rate dropped dramatically following the vaccination of the vulnerable told us everything about the efficacy of the vaccines. The "anti-vaxxers" were guilty of committing the Base-rate Fallacy because they did not account for the ratio of the vaccinated versus the non-vaccinated or the vulnerabilities of each group. They didn't factor these base facts into their conclusion.

Read about Confirmation Bias and COVID-19
Read about Availability Bias and COVID-19

Another Example of Base Rate Fallacy

This machine is useless because it's only 99% accurate

Imagine you have a machine that can detect whether coins are real or fake.

Your machine is pretty good at this. When it checks a coin, it only gets it wrong 1% of the time. 99% of the time it makes the right decision.

Of course, there are two types of error the machine can make:

• It can say that a false coin is real (a false positive).
• It can say that a real coin is false (a false negative).

Nevertheless, your machine will only make an error once for every 100 coins it checks. So, your machine is 99% accurate.

If you ran the machine until it spurted out a coin it believed to be a fake, what would be the chance of that coin actually being a fake? If you said 99%, that's almost certainly wrong. It could be much lower. The truth is you just don't know. Here's why:

Imagine your machine checked 10,010 coins, 10 of which were fake.

In checking the 10,000 real coins, the machine would wrongly identify 1% of them as fakes. In other words, it would get 100 coins wrong, and place them in its fake pile. Let's imagine the machine found all 10 of the fake coins, and correctly put them in the fake pile. We would now have 110 coins in the fake pile, but only 10 of them would be actual fakes. In this example, the chance of any coin being an actual fake would be less than 10%, even though your machine is "99% accurate".

If you originally thought 99%, your answer was derived through fallacious reasoning because an earlier premise (the tiny probability of the coins being fake in the first place, i.e., the base rate) was not taken into account.

A Practical Application for Base Rate Fallacy

Know what those cameras will do for you

Does knowing about Base Rate Fallacy have a practical application? Well, it could. Face-detection software is starting to appear in all sorts of places, and you might think it's the silver bullet to your particular problem. It could well be, but don't forget it depends on what you're testing for and the percentage of those things among everything else you're testing.

For example, if your shop buys a "99% accurate" face-detection camera to spot known shoplifters, your store detectives might find themselves following hundreds of people around the store who have no intention of shoplifting. The usefulness of such a system depends as much on the proportion of shoplifters in our society as it does on the accuracy of the system.

What about cameras to spot terrorists? What proportion of the population are terrorists? Minuscule. There would be thousands of false arrests if face-detection cameras that weren't 100% accurate were deployed at places like airports. Of course, this doesn't mean the cameras wouldn't be useful. It just means complementary procedures would need to be introduced. And that would be useful to know before spending millions on equipping an airport with such a capability.

Another Practical Application for Base Rate Fallacy

Give them 33% and tell them it's 50%

Lots of food companies exploit the Base Rate Fallacy on their packaging. When something says "50% extra free," only a third (33%) of what you're looking at is free. If you think half of what you're looking at is free, then you've committed the Base Rate Fallacy.

For example, when you buy six cans of Coke labelled "50% extra free," only two of the cans are free, not three. (It's because the original pack had four cans, and 50% of the original amount is two cans.)

If you thought three of the cans were free, then you failed to account for an earlier premise (i.e., there were four cans originally), and you committed the Base Rate Fallacy.

See Also