Understanding Cherry Picking in Statistics
Cherry-picking occurs when a person makes a claim about a subject and then selects only the tests that support the claim or ignores any translation of the data that could refute the claim. In other words, it's about selecting your data to prove your claim.
This technique is particularly prevalent in the world of nutritional supplements, where commercially driven scientists have become adept at finding evidence to support the marketeers who peddle potions and supplements designed, or – more correctly – marketed, to cure all ills.
"A common cold if given proper treatment will be cured in seven days but left untreated it will hang around for a week."
Seemingly Reputable Scientists Will Engage in Cherry PickingMarketeers have been using cherry-picked tests for years to sell us stuff, ranging from the notion that Vitamin C prevents the common cold (in spite of the numerous medical studies proving that it doesn't) to the notion that root vegetables and lemon rind prevent HIV. I'm always up for a bit of blackcurrant-flavoured "Lemsip" on the off chance that Vitamin C does actually help a cold, but clearly not all the marketeers' assertions are so harmless. The root-vegetables-and-lemon-rind "cure" and other similar quack "remedies" have been responsible for hundreds of thousands of deaths from AIDS in South Africa over the last 10 years. When I say "quack", don't imagine some backstreet snake-oil merchant. It seems some of the big drug companies have been cherry-picking too. GlaxoSmithKline (one of the world's largest pharmaceutical companies) was fined a whopping $1.9 billion in July 2012 for misselling drugs, and numerous pharmaceutical companies have been caught bending or outright falsifying statistical data in an attempt to prise your cash from your grasp.
Cherry Picking Can Be Very UnderhandedSome of the tricks employed in spinning statistics are blatantly underhand. An obvious one is only reporting test results that support your claim. That's pretty bad, but companies have also been caught duplicating positive data in different scientific papers to make it look as if several studies have reached the same conclusion. Another ploy they use is to stop a study as soon as they've gained sufficient positive data. In other words, they're not letting the Law of Large Numbers kick in. They know that by continuing the trial, the likelihood of their positive result being down to chance would increase.
This stuff is all pretty dishonest, but the worst manipulation of all is suppressing negative data on drugs shown to be harmful – even to the point of suing doctors attempting to highlight a problem.
So, how prevalent is cherry picking? Well, back in 2003, it was found that trials run by drug companies were four times more likely to give a favourable outcome than a study carried out independently.
Globally, pharmaceutical industries are worth £150 billion a year. And some of that is because they've been looking at the data from their trials through their "profit first" goggles.