Pay close attention to the language of your surveys; both questions and answer options are critical. Sometimes, we inadvertently phrase questions in a way that confuses or misleads people into making answer choices that are incorrect. Avoid this common survey design mistake by a careful review.
Here’s a quick test: What’s wrong with the questions below?
As you may be able to see here, a few common issues we see in survey question development include overlapping answer options, double-negative questions, and double-barreled questions.
What are overlapping answer options?
A sloppy mistake that survey designers make is including overlapping answer options. This is a common oversight and can confuse the participants who may fall in multiple categories.
Answer options in Question 1 provides an example of a set of overlapping answer options. If a participant is 24, should she choose the second or third option? If a participant is 34, should he choose the third or fourth? The inconsistency with which participants are likely to decide means that the data collected is unhelpful.
What is a double-negative question?
A double-negative question includes two negative words, potentially confusing or misleading the participant completely. If a participant can’t understand the question, of course, their answer will be meaningless and the resulting data will be useless.
Question 2 is an example of a double-negative question. If you know what that question is asking, good for you! Each participant is likely to interpret the question differently — not helpful for data validity.
What is a double-barreled question?
Including a double-barreled question is one of the most common issues users have in building their surveys, and it can be easy for inexperienced survey designers to overlook and fail to correct. In this type of mistake, the participant is asked about two separate things simultaneously, which means that it will be unclear whether the answer given applies to one or both parts. Again, this issue means that the answers to this question cannot be used in meaningful analysis.
Question 3 is a classic double-barreled question — ambitious but misguided. What is actually being measured here?
If someone agrees with this statement, are they saying that the service was both timely and courteous? What if the service I received was timely but not courteous, or vice versa? A simple point on a rating scale cannot possibly represent the whole range of options.
Check, double check, and check again
Once you become familiar with these common errors, you will be able to avoid them more easily. These are easy checks that can be made at the time of reviewing the survey questions. While it might take time to review your questions multiple times, it’s certainly more efficient than asking the wrong questions and receiving unhelpful or meaningless results — and then having to start again from the top.
