Understanding Biased Questions in Statistics


It is possible to get the answer you want by the way you ask the question. Let's start with an extreme example of a biased or unfair question:

Answer this question with either "yes" or "no":

"Have you stopped shoplifting these days?"

It doesn't matter how you answer this question, you're condemned as either an active shoplifter or an ex-shoplifter. (Next time you're at the pub, shout across the bar and ask your mate if he's all clear at the clinic. It's an old one, but it's still a fairly useful line to have in your back pocket during a round of banter.)

Now, the shoplifting and clinic questions were obviously unfair, but some trials and polls feature questions which are more subtly biased. Asking biased questions is another way to twist a result. Look at these two questions:

Version 1 of the Question: "Do you think the UK should leave the European Union?"

Version 2 of the Question:"Do you think the UK should remain part of the European Union?"

People Like to Avoid Confrontation

Because of an inner desire to avoid confrontation, we are far more inclined to "agree" than "disagree" when presented with questions of this nature. In other words, we are more likely to answer "yes" to these questions. This means that Version 1 is far more likely to see the UK leaving than Version 2.

I think the human desire to "agree" with such questions is so strong it will trump the British people's thoughts on whether they want to be in or out not. In other words, the result of the referendum will depend on which question is asked.

Why do I think that? Well, on balance, I think I want the UK to remain part of the EU, but I also really want to answer "yes" to the first question. (Of course, the bias could be negated by posing both questions, requiring people to answer "yes" to one and "no" to the other. But that's just messy. It's one to watch.)
Help Us To Improve

  • Do you disagree with something on this page?
  • Did you spot a typo?
  • Do you know a bias or fallacy that we've missed?
Please tell us using this form

See Also

Understanding the basics of reading body language