Questions are the life and soul of any survey, without questions there is no survey. Good questions are essential to the success of any survey while poorly written questions can condemn your survey to failure.

I have put together some ideas and considerations on how to write well structured questions.

In this article I focus on:

1. Social Desirability
2. Non-attitudes
3. Inaccurate Estimates

Social Desirability

Social desirability bias is the inclination to present oneself in a manner that will be viewed favorably by others. It is commonly accepted that people are generally inclined to seek some degree of social acceptance and it is not uncommon for survey results to be skewed by people choosing the more desirable answer.

The great thing about online surveys is that people generally give more honest answers when answering questions on a computer where the feeling of anonymity is increased. This is opposed to face-face interviews or telephone interviews where anonymity may be guaranteed but it is obvious that at least the interviewer will be able to associate the answer to the respondent.

While online surveys have some clear advantages they are not exempt from this issue so it is important when designing questions to consider the social desirability bias.

• How to reduce Social Desirability Bias

Reiterate the fact that your survey is anonymous and confidential

i.e. “Please remember that all responses to this survey are completely confidential and your anonymity is guaranteed.

State that the behaviour the questions focuses on is not unusual.

e.g. “Depression affects up to 1 in 5 people, have you ever suffered from depression?”

Use face-saving techniques.

e.g. “Everyone feels low occasionally. How many times last week did you find yourself on a low or depressed mood?”

Non-attitudes

While the questions you are asking are very near and dear to you, you cannot expect your respondents to share your enthusiasm. Often respondents may feel ambivalent to the topic you are researching, if this is the case it can impact on the quality of your responses. If someone does not have an opinion on a topic you cannot expect someone to answer a question credibly.

• One of the first things you can do is make it acceptable for the respondent to say they are unaware of the topic.

“Some people are interested in rugby and some are not, would you say you are interested in rugby?”

• You can use screening or filter questions

e.g. Do you have an opinion about fluoride in water?, Have you thought much about fluoride in water?

• Provide an explicit opt outchoice’ as a response option.

e.g. Should fluoride be placed in public water?

Strongly Agree, Agree, No opinion, Disagree, Strongly Disagree

Inaccurate Estimates

When respondents are asked to provide information about past behaviour or events they are almost always relying on an estimate rather than an actual value. An example is if you ask the respondent how many hours they spend reading each day, most people would have to guess. This does not meant that you can’t ask people about their past behaviours. There are a few tricks you can use to help the respondent frame their answer.

• For example, you want to find out how many hours of television the respondent watches. The key here is to reference the question around a limited recent time period.

Not so good: “How many hours of television have you watched in the last month?”

Much Better: “How many hours of television have you watched in the last week?”

• Ask respondents to think about an actual event as opposed to a generic category of events.

Not so good: “In an average month, how much do you spend on toll calls?”

Much Better: “How much did you spend on toll calls last month?”