survey design

Why won?, Why lost?, Why missed?-Surveys for Greater Customer Insight

September 29th, 2009 by Jared Bothwell


Customer satisfaction surveys tend to dominate thinking when it comes to surveying customers. Yet, there are plenty of other ways to gain insights into your customer base by asking a few quick questions. ‘Why won?’, ‘Why lost?’ and ‘Why missed?’ surveys are just one example. And if you don’t have any customers they are also quite useful for dating.

‘Why won?’, ‘Why lost?’ and ‘Why missed?’ surveys are a series of customer insight surveys that are practically guaranteed to give you the information and insights that any business requires in order to retain existing customers, win back old customers and win new business.

I first came across ‘Why won?’, ‘Why lost?’ and ‘Why missed?’ surveys in my first job as a market researcher when I was straight out of university. I was immediately stuck by the simplicity of the concept but like most things often it is the simple ideas that are the best (excluding  factor and conjoint analysis).

Why Won?

Gaining a new client may mean the popping of champagne corks and a flurry if high fives all around the office.

Often little thought is put to why you may have been successful in winning the new client. A ‘Why won?’ survey seeks to answer this important question.  The key reason for doing a Why won” survey is that the information gleaned can be valuable in winning other new clients .

The underlying assumption being that if newly won customer liked feature A, then it is likely that other prospects may like feature A too. This could lead to greater promotion and emphasis being made of feature A and consequently more new customers.

The timing of a Why won? survey is important. It is best to leave some time between the customer coming on board and asking the new customer to complete the survey. Off course this depends on the type of interaction you may have with the customer but six weeks is a good rule of thumb.

The Questions
While it is not quite as simple as “Hey Bob, Why won?”, but there isn’t too much more to it than that.

The main objective of the survey is to identify the key factors why your new customer decided to go with you  and not your competition. If you have left it for six weeks you can also use the opportunity to see how your new customers are settling in. This can identify any early teething problems and enable you to stop your new customer becoming a lost customer.

The Methodology
The methodology of your survey deployment depends on a number of factors. The value of the new account comes into play along with how you will continue to communicate with your new customer. For higher value accounts I like the idea of a personalised telephone call. The call can play an integral part in the induction process of the new customer.

The person who makes the call should ideally be a neutral individual who has no role in the management of the account. In most circumstances it is ideal to use a market researcher to undertake the interview. Off course, the researcher can be calling from your organisation. The value in using an independent interviewer is that you are more likely to receive more candid and there fore more valuable information.

Online surveys can also be used instead of telephone interviews. I would be reluctant to use online surveys for high value accounts though. Responce rates are likely to be low and they will do little to strengthen the relationship or make your new customer feel welcome.

Why Lost?

Losing customers or clients is a drain on any business and often the temptation is to simply make excuses on why customers leave without capturing any data on the key reasons why they have decided to cease doing business with your organsation.

This is the purpose of the ‘Why lost?’ survey. Questions tend to focus on the key reasons why the client has made the decision to leave. It can also be useful to ask if there is anything you could do to win there business back. Often there isn’t but if there is you really want to know.

The Methodology
The methodology to use is similar to the why won surveys. The key factor is really the value of the account. Spitting out an online survey to a million dollar client will justify their decision. You just don’t care. A telephone survey would be more suitable.

Often the hardest thing with ‘Why Lost?’ surveys is actually knowing when you have lost a client. It is human nature to avoid conflict so if you client can just slip off in the dark – they will. In some industries it is fairly obvious that your customer is leaving e.g. “Hi, I want to cancel my subscription”.

Why Missed?

Often an enormous  amount of resources and energy can go into pitching for new business and unless your operating in a monopoly it is likely that you will come across business defeat in a competitive pitch.

While it is tempting in these situations to just put it down to a “numbers game” and move on if you want to learn from your defeats it is critical that you take the time to find out why your pitch was unsuccessful.

Methodlogy and process should work the same as the ‘Why won?’ and ‘Why lost” surveys.

Data Collection

With all these surveys it is essential that the data is collected in a useful and meaningful manner. It is of no value if you have various forms floating around the office.

While the data collected is incredibly useful to make quick improvements and gains to your business processes it is the longitudinal data that provides fantastic insights into how your business is tracking over time.

Competitive Intelligence

‘Why won?’, ‘Why lost?’ and ‘Why missed?’ surveys will also provide you with a goldmine of competitive intelligence. This data should be tagged appropriately as it enables a complete profile of your competitors activity to constantly developed. There is no need to engage in covert espionage when you have a wealth of knowledge lurking within your customers.

Posted in Customer Satisfaction, Ideas, survey design, Uncategorized | 2 Comments »

NZ Post Market Research Survey Causes Public Concern

July 14th, 2009 by Jared Bothwell

A current market research survey being undertaken by NZ Post has caused some concern which has led to the Privacy Commissioner making a statement that people should not feel compelled to complete the survey.

It seems that some people had believed that they survey was compulsory since it was being sent by NZ Post. While I have not seen the survey invitation the story does highlight the valid issue that surveys sent by some state owned enterprises or Government departments may make people concerned due to the perceived authority of the sender. Most survey senders are loathe to stress the point that the respondent does not have to complete the survey (this would do nothing to help survey response rates) but in this instance it would have been worthwhile stating.

What seems to have worried people most is the level of detailed personal data that the survey asked for. While this information is seen by marketers as marketing gold I think the general public has wised up about simply giving away their details to anyone that asks. The risks are too high (i.e. identity theft or just receiving a flood of irritating targeted direct mail) so you should really just ask for what you need, otherwise you risk ending up with a low response rate as well as alienating the people you want to get close too.

Posted in survey design | 3 Comments »

Saving Excel Charts as an Image

June 19th, 2009 by Jared Bothwell

Excel is pretty good at producing charts and shifting them into a MS Word or PowerPoint report is simple as copy and paste. Saving a chart as an image can be a little more tricky. One quick and easy way is to copy and paste the chart into MS paint and then save as a tiff image (or something similar).

I think the most straight forward method I have found is to introduce this quick and easy Excel add-in featured on the  PTS Blog. The add-in can be found as a zip file and follow these instructions so how to install a excel add-in (which is pretty easy). A great solution!

Posted in survey design | 3 Comments »

Survey Questions – Mandatory or Optional

June 11th, 2009 by Jared Bothwell

One of the great things about online surveys is that you can create a world where free will no longer exists. Or in other words you can choose to make your survey questions mandatory or optional. This power should be used wisely though as although you can make your survey questions mandatory it is a far harder exercise to make your survey mandatory and making survey questions mandatory can be a great way of increasing your survey dropout rates. I thought it was worthwhile investigating then when questions should be mandatory and when caution should be applied.

To begin with it is probably useful in determining what a mandatory question actually is. A mandatory question is when the survey respondent is forced to respond to the question.  If no response is received then the respondent is not able to proceed with the survey. Generally this is achieved by a simple piece of javascript which in practive is as simple as ticking a box marked “Mandatory”. So far it sounds pretty good. Mandatory questions mean that all your survey questions will be answered – what could be wrong with that? In practice a lot -the major downside is that making questions mandatory can bug the hell out of your respondents and cause them to simply dropout of your survey.

A good rule of thumb is the more mandatory questions your survey has the higher your survey drop-out rate will be. All that is achieved by a high drop-out rate is a high non-response error, which is something which should be avoided at all costs. It pays then to seriously consider when you should make questions mandatory.

  • Screening Questions

Often when conducting your survey you have a target segment in mind. You may wish to research all females aged  between 18 – 40. It makes sense then that any screening questions are made mandatory. In this example gender and age are required to be mandatory to ensure you get to speak to the right people. i.e. females aged between 18-40

  • Branching Questions

Branching questions are when the respondents answer to a question sends them off in a particular direction whilst a different response will send then in a different direction of the survey and asked another question. It is essential to make branching questions mandatory otherwise the survey will not know where to send them and your survey respondents will be stuck in survey limbo for ever (not a nice place). 

  • Essential Questions

Essential questions are those questions that you really need an answer to. You really need to take a step back on this one. If you wrote the survey you are likely to believe that all questions are essential. Before you make all questions mandatory take a long hard look at the survey questions and ask yourself what if any damage will be done if the respondent chose not to answer the question. If you can live without the data then give the respondent the choice (another note, if you can live without the data ask yourself if the question is required at all).

The three tips above are pretty good guidelines to keep in mind when considering if you should make your questions mandatory. But like most thing there are a couple of mitigating factors that should also be considered when making this decision.

  • Length of Survey

If your survey is short you are more likely to be able to get away with mandatory questions than if your survey is long.

  • Relationship with respondent

If you have a close relationship with your respondent’s i.e. if they are staff members then you are more likely to be able to get away with mandatory questions.

  • Incentives

If you have some generous incentives for respondents the above rules for mandatory questions. The rule of thumb with incentives and respondents is the greater the level of incentive the more your respondents will be willing to put up with.

So while mandatory questions are pretty useful to ensuring you get the data you need they also have the potential to really bug your respondents. In a way mandatory questions are a lot like drinking – moderation is the key!

Posted in Marketing Research, Online Market Research Tools, survey design | Comments Off

Selecting a Winner for your Survey Prize

May 7th, 2009 by Jared Bothwell


Providing an incentive to survey respondents is a great way to increase response rates and one of my favourites is the prize draw.

Running a prize draw as an incentives raises the question on how you are going to select your prize winner. I laugh when I think about my first research job where I would spend hours making little numbered tickets, placing them in a box and trying to find someone potentially impartial in the office to make the prize draw.

A far quicker and more efficient method is to use a random number selector. The key thing is to attribute a sequential numerical indeifiter to each response. One of the easiest ways to do this is to place your survey data into excel and use the row number as a guide.

The next step is to select a random number selector. Google ‘random number selector’ and you will find a few. My personal choice is The easiest way to get your random number is to use the ‘True Random Number Generator’ on the home page. Enter in the range of numbers, push the button and hey presto! you have a random number and your prize draw.

While it may sound a little weird I typically don’t like to go with the first number it gives me. I start of by letter the generator select a number between 1 and 10. The response from this query then gives me how many attempts I will make before I go with the final generated number. I.e. the ‘N’ th query will be the prize winner.

Posted in Marketing Research, survey design | 2 Comments »

What do your customers really want?

March 31st, 2009 by Jared Bothwell


Finding out what your customers really want is one of the key objectives of market research.

There are a lot of things in the market researchers tool box which help a researcher find the answer to this dynamic question. One of the question types most used by researchers are preference scales. Although they differ in name and length the basic structure of preference scales is universal.

When using preference scales respondents are asked to express their level of preference along a incremental scale. i.e. 1 = very unimportant – 10 = Very important. The issue with this type of scale is that respondents are often likely to express that most things have some degree of importance. This issue often translates itself into products that have a wide range of features as it is difficult to establish which features should be left out.  What then emerges is a over complicated product which often confuses customers. Enter stage left -  ‘Maximum Difference Scaling’.

Maximum difference scaling (or MaxDiff to those in the know) is a statistical method (an example of the science in market research) which forces respondents to make trade-offs between multiple options. By forcing the respondents to make these trade-offs it is possible to find out what is ‘really’ important to your customers and therefore what they really want.

The results of MaxDiff make it quite clear which features are the preferred features and they can then be incorporated into the final product offering.

Posted in survey design | 3 Comments »

Seven things to avoid when designing your market research survey

March 30th, 2009 by Jared Bothwell

surveyfortunecookie1While there is plenty of advice on the net on how to design survey, I want to focus on seven things to avoid when designing your market research survey.

Why make the mistakes that most professional market researchers have already made.

My seven things to avoid are:

  1. Avoid long surveys
  2. Avoid convoluted & technical language
  3. Avoid multiple questions within one question
  4. Avoid asking respondents date of birth
  5. Avoid going live without testing your survey
  6. Avoid dull surveys
  7. Avoid roads to no-where

Avoid long surveys

I cannot emphasise this one enough. No matter how interesting you think your survey is to your respondent a long survey will kill any-one’s enthusiasm.

 A long survey will see a high dropout rate and poor data quality. Just because someone sticks around to the end of the survey does not mean they have answered your survey with quality responses. In all likelihood they have probably just ticked any old box in an effort to get out of your survey as quickly as possible. Remember, people are smart and they will find the quickest way out of any situation.

Avoid convoluted & technical language

Use language in your survey that is easy to understand. Your respondent is not going to run to find their dictionary if they have trouble with the language in your survey. When designing your survey keep in mind all ages, genders and education levels. This doesn’t mean you have to dumb down your survey, just make it understandable to your sample group.

Avoid multiple questions within one question

I call these double barrelled shotguns. i.e. Do you like green eggs and ham?, where would you eat them, in a box of with a fox?

Multiple questions within one question can lead to questions being on partially answered or not answered at all.

Avoid asking respondents date of birth

Avoid at all costs asking for the respondents specific date of birth i.e. DD/MM/YYYY.

If you really need specific birth data leave it at the year i.e. What year were you born? even better provide a year range. i.e What is your age? 16-20, 21-24 etc.

Avoid going live without testing your survey

Testing your survey is crucial for ensuring it works as it should.

When testing you are trying to find out if the questions work, i.e do people understand the question? and does the survey work – very important for online surveys and the like. This is your chance to ensure that your research project will be successful and you can now make any final changes. Internal testing with your peers and internal stakeholders enables everyone to get on board with the survey. A small pre-test of your sample group will ensure that the survey works in the ‘real world’

Avoid dull surveys

Make your survey fun. Keep the language light and breezy. With online surveys consider using video, images and other multimedia. This can help keep your respondents engaged.

Avoid roads to no-where

This is most important with electronically deployed surveys i.e. online surveys. Skip-logic and piping mean it is easy to direct your respondent in different directions. Just don;t lead them down a dead end street. Testing helps with this.

Keep these seven points in mind when designing your next survey and you should be on the right track.

Happy Surveying!!

Posted in Marketing Research, survey design | Comments Off