Techniques to Deceive with Statistics
Statistics is the science that deals with the collecting, classifying, analysing and interpreting numerical information. The findings from all that activity are also called statistics. So, statistics (the science) creates statistics (the results). Both the data-gathering process and interpreting the data can be manipulated to present the answer you want.Here are some common techniques used to manipulate statistics:
Cherry picking the data
Cherry picking is the act of selecting only the test results – or an interpretation of the results – that support the claim.
Omitting confounding variables
Omitting a confounding variable is the act of failing to consider a variable that distorts the relationship between two other variables.
Ignoring the baseline
Ignoring the baseline is the act of performing a comparison with unequal starting points for the elements being compared.
Using relative language not absolute language
While using relative language not absolute language might be appropriate (e.g., relative: "a 100% increase"; absolute: "from 5 to 10"), relative comparisons can be shocking if the absolute figures are not presented (e.g., relative: "a 300% increase!" absolute: "from 0.000001 to 0.000003").
Using deceptive images
Deceptive images are images (usually graphs) designed to support the writer's perspective. For example, a writer might truncate the y axis of a graph to make a difference look more stark.
Using small samples
If you are presented with a claim derived from a small sample, there is a high risk that chance affected the result. The greater the sample, the lower the risk of chance being a factor. (This is called the "Law of Large Numbers".) Therefore, a claim derived from a small sample ought to be challenged.
Read about the Law of Large Numbers.
Presenting emotive comparisons
Presenting emotive comparisons is the act of using disingenuous "yardsticks" that make a claim seem more powerful.
Using the different types of average
There are three types of average (mean, mode, and median). Their values can vary greatly. Be mindful that those seeking to influence might use the word "average", having deliberately chosen the type of average that best supports their claim.
Presenting biased questions
When presented with a question, we are more inclined to "agree" than "disagree" because of an inner desire to avoid confrontation. This means that statistics (e.g., a poll result) are affected by the wording of the question.
Failing to do research
The quickest way to get some facts and figures to support a breaking news story is to make them up or to concoct a high-level statement that feels accurate. For too many journalists, inventing facts is preferable to missing a deadline.
Statistics are the lifeblood of private companies, politicians and governments. They all use statistics to influence you and to manipulate your actions. For example, they will use them to persuade you to make investment decisions, to encourage you to buy their products, or to show you they are tough on crime or skilled at managing the economy.However, manipulative companies and individuals have become highly adept at spinning statistics to make the "facts" look more favourable to them.
You Can't Trust the Experts
Never forget that lots of the statistics you see are not designed to inform you but to influence you to buy something or to adopt someone else's point of view. And also bear in mind that some of the worst offenders are the respectably titled experts who operate in fields you know nothing about. These people are actively spinning statistics to influence you. It's their job. So, we all need to be far more sceptical about statistics. (Even if the statistics are not outright lies or obviously biased, there's a fair chance they're being presented by people ignorant of or indifferent to their accuracy.)
(Sir Winston Churchill 1874–1965)
(British statistician and banker Josiah Stamp, 1880–1941)
Scrutinise the Language
Let's look at the words of a motor-insurance advert that I saw recently. The ad featured a car mangled from a crash and a claim that car crashes cause 2,538 deaths per year. It also included: "1 in 200 people are killed in car crashes".
I thought that the "200" figure looked a bit low, so I went trawling for some statistics to back it up. I couldn't find any. So, did they just make up that "1 in 200 people are killed in car crashes"? Possibly. But, in my experience, these people are usually a little bit more conscientious than just inventing figures. So, were they spinning the truth with their language? Probably, and this is my suspicion.Let's look at those words again "1 in 200 people are killed in car crashes." What does that mean? Does it mean the cause of death for 1 in 200 of us will be a car crash? Well, it could mean that. But, it could also mean that 1 in 200 of us will die if we are in a car crash. The English is ambiguous.
- "Scientists have proven it will make your skin smoother." (Whose scientists? This is an example of an appeal to an unnamed authority.)
- "It contains polypeptides that help the cells communicate." (This is an example of obfuscation fallacy.)
- "Formulated with active Dead Sea minerals, our vitamin E-infused nourishing cream is enriched with ivory-coloured Shea butter and organic jojoba oil for continuous hydration." (Coupled with a high price tag this is an example of the creating the expectation effect (i.e., when something is considered better because you expect it to be better).)
See Also
- Do you disagree with something on this page?
- Did you spot a typo?
- Do you know a bias or fallacy that we've missed?
