One of the benefits of online surveys is that you can make them as long as you like without exhausting Amazon quantities of paper. As you can imagine this benefit can quickly become a curse. I completed a survey recently which took over 25 minutes to complete asking my opinion about a range of software options.

In my mind long surveys show an inherent misunderstanding and disrespect to respondents and should be avoided at all costs. In all likelihood by the time the respondent is halfway through the survey they are likely to either give up or quickly whiz through the survey in order to claim whatever incentive is available.

You can imagine the effect of the long survey on your data quality. In most cases you might as well disregard the questions towards the end as the impact of survey fatigue takes its hold on the respondents judgement and overall attitude towards your survey. In addition your survey abandonment rate will increase dramatically.

So, the questions is raised – How long is too long for your online survey (and any other survey for that matter).  Mark Kupferman has come up with four key factors which I think are worth considering for anyone developing a survey.

1. What does the respondent get for taking the survey?
2. How interesting is the survey topic to the respondent?
3. How much fun is it to complete the survey?
4. What impact will completing the survey have on the respondent?

I have added one more of my own.

1. What happens to the survey results?

I have expanded on Mark’s factors below as well as my own. Read the rest of this entry »

Over the last few weeks I have been invited to take part in a number of surveys on a number of websites I have visited. The survey service is run by Crowd Science.

What Crowd Science seeks to do is to bridge the knowledge gap between traditioanl website analytic tools and your website visitors. This is achieved by asking your website visitors a series of questions with a core focus on geographic and demographics.

There is nothing really new here, website surveys seeking to understand the profile of visitors have been around for some time. What seems to make Crowd Science different is the ease of installation and reporting functionality.

This morning I signed up for an account and had the survey operating in under five minutes. The basic account is free and once you place the survey tag onto your site the survey is working. While it is too early to say whether or not Crowd Science is a useful tool I look forward ot posting a full review of this promising and useful looking service.

I see that Apple has sent out a survey to conduct a customer satisfaction with its iPhone 3G customers.

While it seems that iPhone customers are the most satisfied what I am interested in is the possibilities that the new wave of smart phones will have on the market research industry and how surveys are deployed.

With more and more people using their mobiles to check their emails the possibilities that your survey will be completed on a phone increases. This can only mean faster response rates and opens up the possibilities for instant feedback.

There are a number of limiting factors though. Data plans in NZ are currently far from generous and while completing a survey does not generally take a lot of data the survey will be completed by the respondent at considerable expense to themselves.

Also, with the Smart phone war developing rapidly a number of browsers are being used across platforms. The survey needs to be able compatible with those platforms. I suspect it is early days with this technology but I believe that market research and online surveys on mobile phones can only increase.

One of the ways to spruce up your survey and make it more interesting to your respondents is to incorporate multimedia into the survey. Incorporating video clips into your survey is a cinch and something which is quite impractical with more traditional survey methods. i.e. can you imagine the cost of sending out a DVD with every mail survey.

There are a number of options to consider when incorporating video into your survey. One of the easiest and most straightforward ways is to use an external video hosting service like YouTube Another option is to host the video on your own server.

Recently when conducting a survey we had to make a decision on how to incorporate a video into a survey and we decided on Google Video The reason for this was we found that Google Video allowed the most flexibility. We were able to restrict access to the video and it gave a far cleaner user experience.

YouTube failed the test due to the fact that if the user clicked on the video they were then taken through to the YouTube site, hardly ideal halfway through the survey. Also at the end of the YouTube video it gives the viewer options of related videos to watch – also not ideal, we don’t want to distract our respondents while they are taking the survey. Now it may be possible that some of these YouTube functions can be managed (i,e turned off) but I couldn’t figure out how to do it.

So, Google Video in my mind is a pretty safe choice for getting videos into your surveys.  I would love to hear of others though.

Asking someones age is generally through of  the height of rudeness, yet in surveys the age of your respondents is critical demographic data. Data accuracy around this question is pretty important and recent research shows that the way you ask this question has a strong influence on how it will be answered.

Philip Gendall and Benjamin Healey (my old market research lecturer and backwards market research guru) have done a bit of digging round to see who researchers can best pose this question to respondents without having 80% of your respondents saying they have just turned 21. Read the rest of this entry »

In a slightly confusing transaction software giant Microsoft has purchased survey and shopping company Greenfield Online. As part of the deal Microsoft will sell of the survey arm of Greenfield Online and retain the European shopping comparison component.

Greenfield’s Internet survey solutions (ISS) business collects, organizes, and sells consumer and business survey data to market research firms as well as offers real-time survey sampling. But the service does not fit in Microsoft’s game plan for its Live Search.

“It was very important to Microsoft to find the right partner to champion the continued growth of the Greenfield Online ISS business, which is not a fit within Microsoft’s strategy for our Live Search platform,” Tami Reller, chief financial officer of Windows and Online Services at Microsoft, said in a statement.

I came across an interesting paper that reports the possibility that online questionnaires might, all things being equal, produce slightly longer responses to open-ended questions than their paper-based counterparts.

Martyn Denscombe in his paper ‘The Length of Responses to Open-Ended Question – A Comparison of Online and Paper Questionnaires in Terms of a Mode Effect‘ (2007) investigates whether or not there is any difference between the responses of open ended questions to paper based surveys compared to online surveys.

Denscombe undertook an experiment survey was conducted that involved the random allocation of respondents into two groups who then completed a questionnaire either as a web-based online version or in an 2 Social Science Computer Review “optical mark recognition” (OMR) paper format. The research was designed to enable near identical groups to respond to near-identical questionnaires delivered in different modes.

Read the rest of this entry »

Poor survey design can create a poor user experience and increase drop out rates of participation. The design of your online survey can have a large impact on the results that your survey generates so it pays to get it right. Historically online surveys have looked very much like mail surveys simply transposed onto a web page. While technically anything can go with surveys there are some basic design principles that are worth while adhering to.

Read the rest of this entry »

Time for researchers to reinvent research itself?, Jonathan Dodd raises this questions in his article “Online surveys don’t tick all the boxes “

The key issue raised is with consumer panels and the detrimental effects that ‘professional’ respondents can have on research results. I have to agree with the arguments raised in this article and this is why Rock Research has to date stayed away from consumer panels.

One type of panel which we have implemented with success is the Rock Research Citizens Panel This is an online Panel which randomly recruits members from the community in order to provide feedback to Local Government. While a small incentive is used the main focus is to allow community members to have a say in key topical issues that are affecting the community. To date the Citizens Panels have been very successful at providing timely and accurate feedback from the community to Council decision makers.

Palmerston North City Council have recently successfully implemented a Rock Research Citizens Panel which has surveyed residents on issues from rates rises to recreation strategies.

In Thomas F. Stafford’s and Dennis Gonier’s article “The Online Research “Bubble” – Seeking to improve the commonly used online
survey sampling approaches” they seek to offer some improvements to the current online survey sampling methods which are in vogue with market researchers. The practice of large online consumer panels is posing a serious threat to the industry due to the fact that there is an emerging phenomenon of quasi-professional survey takers. Research from one large panel has shown that 1% of the panel was responsible for 30% of the online surveys completed. These survey takers are less likely to be fully employed and spend a disproportionate amount of time online.

The solution that Stafford & Gonier suggest is to look beyond the panel. They make the point that it is worthwhile considering other alternatives. They argue that To get your online research one quickly, you’ll either have to sacrifice cost or quality. It’s an easy choice: two of three important characteristics are available. You just don’t get it all, because the free lunch” of cheap, quick, and valid results doesn’t exist in online studies any more than it did in the world of offline research. Good sampling takes time or money; take your choice.

One thing that I do think that is worthwhile considering is designing purpose built panels. Custom online panels can can offer a pretty neat solution and at a reasonable price. While not suitable for all research problems instances they do particularly well within closed groups i.e. customer research, citizens panels.